Hash deep find and compact in rails

One of the applications I've worked recently has a lot of tree like structures represented internally as hashes. This is a very convenient way of handling such structures and rails provides good support to convert between object, hash and string (json) representations of these structures.

Two problems that are recurrent when working with these structures are finding a key in the tree and compacting (removing null or empty values) from the tree. Below there are 2 code fragments that do that.

def hash_deep_find(obj, key)
  if obj.respond_to?(:key?) && obj.key?(key)
    return obj[key]
  elsif obj.respond_to?(:keys)
    obj.keys.each do |child_key|
      return hash_deep_find(obj[child_key], key)
    end  endend

def hash_deep_compact(obj)
  if obj.is_a?(Hash)
    obj = obj.delete_if { |k, v| v.blank?}
    obj.keys.each do |key|
      hash_deep_compact(obj[key])
    end  endend

As to not change the application structure, these methods were implemented as methods of an Utils class already present in the application. Alternatively, one could simply extend the behavior of Hash (either in the Hash class itself or a class that extends Hash) adding these 2 methods.

Solving ORA-28001: the password has expired

It always happens when you are in a hurry and it makes you really upset. It's nothing to get desperate, though. To solve it, follow the steps below:

Connect as sysdba to the database:

> sudo su - oracle

> sqlplus sys as sysdba

Set password life time to unlimited in Oracle:

SQL> ALTER PROFILE DEFAULT LIMIT PASSWORD_LIFE_TIME UNLIMITED;

Profile altered.

Reset the password for the locked user:

SQL> ALTER USER <user_name> IDENTIFIED BY <password>;

User altered.

Unlock the user account:

SQL> ALTER USER user_name ACCOUNT UNLOCK;

User altered.

Check if the user account is actually unlocked:

SQL>  SELECT USERNAME,ACCOUNT_STATUS FROM DBA_USERS WHERE USERNAME='<user_name>'

ACCOUNT_STATUS should be OPEN.

Oracle Database sizing for developers

Recently, one of our biggest clients has required us to estimate the size of the database of one of our applications. What seemed a complicated task at first proved to be fairly simple.

The task was divided into 3 steps:

1) First step was to map what is persisted by each of the application's features.

2) Next step was to estimate the average use of the application, that is, how many times each of the features would be used in a month.

3) Knowing these information, we simply had to estimate the average row size for each of the tables and multiply by the number of records estimated in the previous 2 steps. In Oracle, this can be easily achieved by querying the dba_tables after analyzing the tables:

 -- Analyze the tables and generate statistics for them (including average row size)
 ANALYZE TABLE <table name> COMPUTE STATISTICS;


 -- Get the generated average row size
 select OWNER, TABLE_NAME, AVG_ROW_LEN from dba_tables where OWNER=<schema owner> order by TABLE_NAME;

If you want to be super cautious you could use the maximum row size instead of the average row size.

The average row size depends on the data you have inserted on your database, so it could be terribly wrong if your data sample is small or badly distributed. The maximum row size, on the other hand, gives you the maximum size a row can take and thus can be used to estimate the size of your database in a more conservative manner.

To get the maximum row size for all tables of an user in Oracle:

 -- MAX_ROW_SIZE
 select
   table_name ,
   sum(data_length)
 from
   user_tab_columns
 group by
   table_name

Transferring things between Virtualbox host and guest machines

If you want to transfer text or image between host and guest (or vice versa), the simplest way is good old copy and paste. 

Fortunately, Virtualbox already has a feature that allows us to do so. To enable it, first you need to install the Virtual Box Guest Additions. 

On your guest machine, go to the top menu and choose **Devices** and then **Insert Guest Additions CD Image...**.

Installing guest additions, however, isn't enough to enable copy and paste. You now need to enable it in the virtual machine settings. 

Select your machine in Virtualbox, then click on **Settings**. On the settings window, click on the **General** menu and then **Advanced**. What you are looking for is **Shared Clipboard**.

Even though copy and paste is fantastic, currently it doesn't work for files. To be able to move files between your host and guest machines you can enable **Drag'n'Drop** on the same tab. It works like charm in my linux system.

Solr update / sunspot commit way too slow

We have recently updated one of our applications to ruby 2.1.2 and rails 3.2 and, as part of the process, updated most of the gems used. One of the gems updated was sunspot, from 1.3.0 to 2.1.0.

After this update we started experiencing terrible performance in solr update when running a sunspot commit. And by terrible I mean awful. 45s average per commit.

If you are experiencing something like this too, here are some tips (and, at the end, the solution to our problem):

• Check if your system has enough free memory. If you are swapping, that could be your problem.

• If you use the sunspot-solr gem to start solr, run it in the foreground to get the full log on your screen and try to identify which step is taking long (in our case it was SpellChecker):

sunspot-solr -s /my_app/shared/solr/ -d /my_app/shared/solr/data --pid-dir=/my_app/shared/pids/ run 

will start solr in the foreground.

• In the configuration file, solrconfig.xml, check if the values of autoCommit.maxTime and autoSoftCommit.maxTime aren't to low:

         <autoCommit>

           <maxTime>60000</maxTime>

         </autoCommit>

         <autoSoftCommit>

           <maxTime>15000</maxTime>

         </autoSoftCommit>

To be honest, I don't know what are the ideal values for them, but 60s between each autoCommit and 15s between each softCommit seemed fine for our purposes.

• Check if the value of the autowarmCount property for the caches is not too high. In our case, it was 0 and so it remained.

autowarmCount="0"

• Check if the automatic building of the spellcheck dictionary on every commit is not enabled. In our case, that was the culprit! This is a very expensive operation and it was being invoked after every single commit. We simply changed the property buildOnCommit of the spellchecker to false and the average commit time dropped to 19ms. Excellent!

Performance killer:

<str name="buildOnCommit">true</str>

Way to go:

<str name="buildOnCommit">false</str>

Just as an observation, buildOnCommit = true was the default value on the configuration file generated by sunspot-solr.