Good news, everyone! (aka articles/posts worth reading according to me)

Sometimes it seems impossible to find useful, creative or entertaining information in the huge pile of trash that the Internet has become. But there are plenty of good articles/posts out there. I've decided to post here some of the ones that caught my attention lately and I've decided to call this column "Good news, everyone!", in honor of professor Farnsworth :) (Futurama, in case you didn't get the reference).

Deep Learning in the Stock Market

https://medium.com/@TalPerry/deep-learning-the-stock-market-df853d139e02

A Mathematician's Secret: We're Not All Geniuses

https://www.bloomberg.com/view/articles/2017-06-07/a-mathematician-s-secret-we-re-not-all-geniuses

How to read and understand a scientific paper: a guide for non-scientists

http://blogs.lse.ac.uk/impactofsocialsciences/2016/05/09/how-to-read-and-understand-a-scientific-paper-a-guide-for-non-scientists/

How long should peer review take?

https://scientistseessquirrel.wordpress.com/2015/08/31/how-long-should-peer-review-take/

The Limits of the CAP Theorem

https://www.cockroachlabs.com/blog/limits-of-the-cap-theorem/

GitHub Secrets

https://github.com/blog/967-github-secrets

What papers should everyone read?

https://cstheory.stackexchange.com/questions/1168/what-papers-should-everyone-read

Learn to Read Code

https://kylesletten.com/2017/06/26/learn-to-read-code.html

Oracle character sets and VARCHAR size

A few weeks ago one of my work mates contacted me with a problem that seemed particularly weird for him. A database column had a limit of 200 characters but updates would fail sometimes with "value too large for column" even though the application was also limiting the input to 200 characters.

The problem is actually quite simple, but it can be confusing if you are not familiar with 2 concepts: database character sets and the way Oracle limits a VARCHAR column.

First, character sets. Oracle has 2 types of character sets based on the type of encoding used:

• Single-byte character sets (e.g. US7ASCII and WE8ISO8859P1)
• Multibyte character sets (e.g. AL32UTF8)

A single-byte character set, as the name implies, uses only one byte to represent each of the supported characters. A multibyte character set, on the other hand, can use more than one byte to represent one character.

Now the problem is clear. The application was limiting the input to 200 characters but these characters used up more than 200 bytes in the database, causing the error. But why is Oracle limiting the column to 200 bytes instead of 200 characters as expected?

As a matter of fact Oracle can limit the size of your VARCHAR columns both ways. If you want to limit the size in bytes:

    mycolumn VARCHAR2(10 BYTE)

If you want to limit the size in characters:

    mycolumn VARCHAR2(10 CHAR)

The catch here is that the default behavior is limit in bytes, contrary to what one would expect. So, if you don't specify which way to limit your VARCHAR2 columns Oracle will limit them in bytes. Beware of that!

    mycolumn VARCHAR2(10) is equivalent to mycolumn VARCHAR2(10 BYTE)

Obs: this default behavior can be changed by setting the NLS_LENGTH_SEMANTICS parameter.

I find this behavior a bit weird and completely understands why my work mate was a bit confused. 

If you want more information on this, here's a link to a good Oracle document:

https://docs.oracle.com/cd/B19306_01/server.102/b14225/ch2charset.htm

Some interesting articles I have read in the last few days:

New technology is eroding your right to tinker with things you own
http://www.economist.com/news/science-and-technology/21723679-digital-technology-eroding-peoples-right-tinker-things-they-own-end-ownership

Power Causes Brain Damage
https://www.theatlantic.com/magazine/archive/2017/07/power-causes-brain-damage/528711/?single_page=true

How is Ruby Different in Japan?
https://appfolio-engineering.squarespace.com/appfolio-engineering/2017/5/24/how-is-ruby-different-in-japan

Why is cycling so popular in the Netherlands?
http://www.bbc.com/news/magazine-23587916

impdp remap to a different schema on target machine

If you have a full schema exported using expdp and want to import it in a different schema on the target machine, you have to use the remap_schema option.

Example:

impdp <user>/<passwd> DUMPFILE=my_dmp_file.dmp remap_schema=<schema_exported>:<schema_where_I_want_to_pump_the_data_into>

It's not necessary to create the user <schema_where_I_want_to_pump_the_data_into> beforehand, the impdp utility will create it for you.

ORA-39087: directory name MY_DATA_PUMP_DIRECTORY is invalid

When performing a database export using the expdp utility you can run into an error like:

expdp <user>/<passwd> schemas=<schema_to_be_exported> dumpfile=my_dmp_file.dmp logfile=my_log_file.log

ORA-39002: invalid operation 
ORA-39070: Unable to open the log file. 
ORA-39087: directory name MY_DATA_PUMP_DIRECTORY is invalid

That happens because the current user default output directory is not defined.

To solve that, grant “create any directory” privilege to your user:

SQL> grant create any directory to <user>;

Grant succeeded.

And then create a directory to be used by the expdp utility:

SQL> create directory my_data_pump_directory as '/tmp/db_dmp';

Directory created.

Hash deep find and compact in rails

One of the applications I've worked recently has a lot of tree like structures represented internally as hashes. This is a very convenient way of handling such structures and rails provides good support to convert between object, hash and string (json) representations of these structures.

Two problems that are recurrent when working with these structures are finding a key in the tree and compacting (removing null or empty values) from the tree. Below there are 2 code fragments that do that.

def hash_deep_find(obj, key)
  if obj.respond_to?(:key?) && obj.key?(key)
    return obj[key]
  elsif obj.respond_to?(:keys)
    obj.keys.each do |child_key|
      return hash_deep_find(obj[child_key], key)
    end  endend

def hash_deep_compact(obj)
  if obj.is_a?(Hash)
    obj = obj.delete_if { |k, v| v.blank?}
    obj.keys.each do |key|
      hash_deep_compact(obj[key])
    end  endend

As to not change the application structure, these methods were implemented as methods of an Utils class already present in the application. Alternatively, one could simply extend the behavior of Hash (either in the Hash class itself or a class that extends Hash) adding these 2 methods.

Solving ORA-28001: the password has expired

It always happens when you are in a hurry and it makes you really upset. It's nothing to get desperate, though. To solve it, follow the steps below:

Connect as sysdba to the database:

> sudo su - oracle

> sqlplus sys as sysdba

Set password life time to unlimited in Oracle:

SQL> ALTER PROFILE DEFAULT LIMIT PASSWORD_LIFE_TIME UNLIMITED;

Profile altered.

Reset the password for the locked user:

SQL> ALTER USER <user_name> IDENTIFIED BY <password>;

User altered.

Unlock the user account:

SQL> ALTER USER user_name ACCOUNT UNLOCK;

User altered.

Check if the user account is actually unlocked:

SQL>  SELECT USERNAME,ACCOUNT_STATUS FROM DBA_USERS WHERE USERNAME='<user_name>'

ACCOUNT_STATUS should be OPEN.