Samuel Mullen

What could possibly go wrong?

Getting More Out of the Rails Console

Even at it’s most basic functionality, the console is indispensable in the Rails developer’s arsenal of tools. Whether it’s running a query, testing a chain of methods, or executing small blocks of code, the console may be the most useful tool available to the Rails developer. With that in mind, doesn’t it make sense to do what we can to get the most out of it?

IRB

.irbrc

Under the hood, the Rails Console is just IRB (Interactive Ruby), so anything you can do with IRB, you can do in the console. This means you can modify your IRB environment and .irbrc file to define methods to use at the console. Here are three methods I frequently use:

Useful IRB methods
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
\# return a sorted list of methods minus those which are inherited from Object
class Object
  def interesting_methods
    (self.methods - Object.instance_methods).sort
  end
end

\# return an Array of random numbers
class Array
  def self.test_list(x=10)
    Array(1..x)
  end
end

\# return a Hash of symbols to random numbers
class Hash
  def self.test_list
    Array(:a..:z).each_with_object({}) {|x,h| h[x] = rand(100) }
  end
end

Of course this isn’t even scratching the surface of what’s possible. Check out what other people are doing:

The Last Expression

While working in the console, have you ever typed out a bit of code to return a value and then realize you forgot to assign the returned value to a variable? You then have to go back into the console history, move your cursor to the beginning of the line, add the variable, and then execute the code again.

Ugh, what a pain!

Unbeknownst to most people, IRB places the output of the last command into the _ variable. Here, let me show you:

_ example
1
2
3
4
 > Array(1..10)
 => [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
 > _
 => [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

Now, as cool as that is, you have to understand that _ always contains the output of the last expression. This means if you try call a method on it, it will then contain the output of the method executed.

_ detailed example
1
2
3
4
5
6
7
8
 > Array(1..10)
 => [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
 > _
 => [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
 >  _.map {|i| i * 2}
 => [2, 4, 6, 8, 10, 12, 14, 16, 18, 20]
 > _
 => [2, 4, 6, 8, 10, 12, 14, 16, 18, 20]

Forking IRB

When working in the console, it’s sometimes desirable to have another instance to play with. It may be that you don’t want to lose what you were working with, or you just need another scratch area, but whatever the case, you can create a new console (IRB) session by calling irb at the prompt (note: you’ll use irb for the rails console as well).

I typically don’t use this. If I need another IRB instance, I just open a new tmux pane or window and work there.

If this sort of method fits your workflow, I highly recommend reading Gabriel Horner’s in depth post on IRB commands

The Rails Console

Models

One of the things you will want to make extensive use of in the console are your app’s models. The Rails console is a great way to play with your models and an alternative way of accessing your data.

Model Example
1
2
3
4
5
6
7
8
9
10
11
 > u = User.find 1234; nil
=> nil

 > u.name
 => "Foo Man"

 > u.email = "fooman@example.com"
 => "fooman@example.com"

 > u.save
 => true

The “app” object

The app object is used by test processes to mimic system interactions. Through this object, we can access routing information and even make requests to our app.

app Example
1
2
3
4
5
6
7
8
9
10
11
12
13
14
# displaying path information
 > app.users_path
=> "/users/

 > app.user_path(User.last)
=> "/users/42

# making app requests

 > app.get app.user_path(User.last)
=> 200

 > app.response.body
=>  => "<!DOCTYPE html>\n<html>\n  <head>\n    <title>..."

The “helper” Object

I really don’t think I can do a better job on this than what Nick Quaranto already did in his “Three Quick Rails console tips” post.

Reloading the Environment

If you make changes to your app while still in the console, you will need to reload your console session with the reload! command. You will also need to reinstantiate objects which existed prior to the reload for them to recognize your changes.

reload! Example
1
2
3
4
5
6
7
8
 > u = User.find 1234; nil

\# changes made to User model outside the console 

 > reload!
Reloading...
 => true
 > u = User.find 1234; nil

Better Output

At one time it seemed like everyone was writing a gem for improving the IRB experience, but it appears like that particular endeavor has since been largely ignored. The one project that appears to be currently active is the awesome_print gem.

I’ve used this gem in the past, and it really does improve the output and IRB experience. It also supports pry.

In a pinch, you can format the output as YAML with the y command.

y example
1
2
3
4
5
6
7
8
 > y Array.test_list(5)
---
- 1
- 2
- 3
- 4
- 5
 => nil

Avoiding Output

Running commands in the console can get pretty noisy. Output follows every command which is run. To get around this, just end your command with a semicolon and nil.

Avoiding Output
1
2
 > u = User.last; nil
=> nil

In the above example, u still contains the “last” user record, it just doesn’t print out all the output that would normally be produced.

The Sandbox

Sometimes it would be nice to open up a console session and mess around with the data to see what happens. But if you do that, the data’s messed up. The solution to that is to lunch the console with the --sandbox flag. When launched, you can handle the data, tweak it, and destroy it, all without fear of harming any of your data.

Rails Console Sandbox
1
2
3
4
5
rails console --sandbox

Loading development environment in sandbox (Rails 3.2.1)
Any modifications you make will be rolled back on exit
 >

Conclusion

In my workflow, the rails console is an indispensible tool. It allows me to test out ideas, access the database, run minor tasks, and even calculate basic math. I can’t imagine developing Rails applications without it, because I know how painful it is to be deprived of such a tool in other languages and frameworks.

What are your favorite IRB and console tricks?

Validating Booleans

I ran into an instance today wherein I needed to validate that a boolean field was either true or false and not null. I tried using validates :fieldname, :presence => true, but since :presence uses #blank? under the hood, it was reading false as not being present. (Why is false considered blank?)

Anyway, I needed a validator to test whether an attribute was either true or false and I couldn’t find anything among the standard validators, so I wrote my own.

Just plop this file in your app’s lib/validators directory.

truthiness_validator.rb
1
2
3
4
5
6
7
class TruthinessValidator < ActiveModel::EachValidator
  def validate_each(record, attribute, value)
    unless value == true || value == false
      record.errors[attribute] << "must be true or false"
    end
  end
end

In your model, add the validation like so:

some_model.rb
1
2
3
4
5
6
7
class SomeModel < ActiveRecord::Base
  ...

  validates :field_name, :truthiness => true

  ...
end

For more information on writing validators, see Getting Started with Custom Rails3 Validators.

Why It’s Okay to Write Crummy Code

There is an inherent need in good developers to write “good” code. We continually look for ways to simplify, enhance, improve, and speed up the code we - or others - have written. We also look back on our older code and shake our head in disgust.

Perhaps it is because of this love of “good” code or perhaps it is because of what we know we can do in more familiar languages, or perhaps, even, it’s because we know what we don’t know, but it seems there is a sort of barrier between learning and using a new technology (by technology I mean a language, framework, methodology, etc.).

Rather than doing, sometimes it’s easier to continue reading and researching. “Just one more section”, “just one more chapter”, “just one more book”. It makes sense, of course, we have to have some sort of foundation upon which to build; and so we read a little bit more.

But the argument that we need to learn just a little more before we begin doing is a hollow argument: we learn far more by doing than by reading. Think back upon when you’ve learned the most: was it when you were in class listening to lectures, or in lab applying what you learned; when you read the chapter, or when you completed the exercises; when you read the API, or tried to implement it?

But the “doing” is difficult, we end up hitting speed bumps and roadblocks, we go in circles in roundabouts, and occasionally we even find ourselves in dark alleyways or hit dead ends, but eventually, if we persevere, we find our way onto the right path. It’s those little success which make all the difference. Finding out what doesn’t work is a learning experience all to itself, but it is the successes, both large and small, which make all the difference. It is the successes which provide us with the energy and motivation to continue on.

With this in mind, go ahead and write crummy code. Write obtuse, ugly, poorly thought out code. Disregard methodologies and best practices, violate principles, ignore everything which hinders you from learning and playing. Fool around and explore, find the speed bumps, the roadblocks, and even the dark alleyways; it’s how you learn and how you eventually succeed.

So to Hell with being ready and writing perfect code; go have fun. If you’re not having fun, you’re doing it wrong anyway.

Evolution of an Apple Fanboy

  • 1997 - 2010: I was an anti-apple, Linux fanboy.
  • 18 months ago: I bought an iPad (16GB) instead of a Kindle at my wife’s behest
  • 17 months ago: I won a second iPad (32GB)
  • 1 year ago: I bought a MacBook Pro so I could be on the same page as my Ruby coworkers
  • Six months ago: I won a 13” MacBook Air
  • 10 days ago: I started learning Objective-C and iOS development.
  • This morning: I searched out iOS developer podcasts
  • Currently: I’m planning on getting the next iPhone, the new iPad, and the Apple TV (little black box).

Things got really weird somewhere in that mess, and I have no idea how it happened. I still see myself as a Linux guy, but I can no longer deny my appreciation for Apple products. Who says I’m closed-minded?

Compiling Objective-C Programs at the Command Line

I’ve switched my learning efforts from JavaScript frameworks to iOS development. I’m using my morning 30 minute period of time to study the ”Objective-C Programming” book from ”Big Nerd Ranch”. I’m also taking an hour out of my normal billable hours to study Big Nerd Ranch’s ”iOS Programming”.

Many of the chapters of the “Objective-C Programming” book have exercises at the end to help the student cement what he or she has learned. As I’ve been going through them, I’ve become very frustrated with needing to open a new XCode project for each exercise. It would be a lot easier for me just to code up the solutions in Vim and compile them at the command line.

I wasn’t sure it was possible, but after a quick Googling, I ran across a sample chapter from “Programming in Objective C” which had the solution.

In a nutshell:

  1. Create an objective-c implimentation file (*.m)
  2. Add the necessary code
  3. Compile it using the following command
Compiling obj-c Programs
1
$ clang -fobjc-arc –framework Foundation filename.m -o output_filename

For those who are familiar with GCC or other command-line compilers, that should be enough to get you started. For the rest of you, you may want to stick with XCode. Seriously, I’m not sure how long I’ll stick with this method.

You should note that writing and debugging Objective-C programs from the terminal is a valid approach. However, it’s not a good long-term strategy. If you want to build Mac OS X or iOS applications, there’s more to just the executable file that needs to be “packaged” into an application bundle. It’s not easy to do that from the Terminal application, and it’s one of Xcode’s specialties. Therefore, I suggest you start learning to use Xcode to develop your programs. There is a learning curve to do this, but the effort will be well worth it in the end.

As stated at the end of the linked article (chapter), this is not something you’ll want to use to build MacOS or iOS applications. I’m only doing things this way to simplify completion of exercises.

Thirty Minutes a Day

Thirty minutes doesn’t seem like much time, especially for programmers. It takes that much time just to get into the “zone”. It takes at least that much time just to research some libraries or problems. You can read a chapter of a book in thirty minutes, but can you imagine working on a project for only thirty minutes a day?

But if thirty minutes a day is the only amount of time you have, the question changes from “can you?” to “do you want to?”.

I get around thirty minutes to myself in the mornings before my family wakes up and I have to get everyone ready for their day. For a long while I used that precious amount of time to read blog posts, twitter, email, or pitter it away in some other manner. I’ve not done that this year.

Since January, I’ve used those thirty minutes to write ten blog posts and read three books. It doesn’t sound like much, but if I were to keep up that pace, I would have forty blog posts written and 12 books read by the end of the year. That’s not too shabby…If I were to keep up the pace.

I’ve been wanting to learn Backbone.js for a while now, but the free time I have during the rest of my day involves my family. The only time I have is my thirty minute window. So I’ve decided to use that time to work on a project using backbone.

I won’t have time to write - although I have found time this Saturday morning - and I won’t have as much time to read, but I really want to learn backbone.js.

I get thirty minutes a day. It’s not much, but it’s mine, and it’s not wasted.

ActiveRecord::Base.store - Pros, Cons, and Usage

In my post on ActiveRecord’s new explainmethod, I mentioned there were “two” features in ActiveRecord (introduced in Rails 3.2) I wanted to look at. This post covers the second method: ActiveRecord::Base.store.

Store provides a simple means of accessing and storing key/value pairs related to a model. The API documentation uses the example of a User model which has settings. “Settings” may not in themselves warrant their own model, but there still needs to be a means of accessing them. This is where Store comes in to play.

Behind the curtain, store is just a Hash which gets serialized and deserialized upon save and load. It has a few accessors added in to make it look less hashy, but the hash is still there nonetheless as we’ll see later.

Prior to Rails 3.2, if you needed this functionality you had three choices: 1) implement it yourself; 2) Muck up your table with a bunch of extra fields; 3) create another table to store all the fields.

For the sake of laziness, I’ll use the same project I used in my previous post: https://github.com/samullen/ar_explain

Setup

In our example, we’re going to allow users to toggle the different types of email they want to receive. The first thing we’ll need to do is add the field in which to store the settings.

Add contact_settings to Users
1
2
3
4
5
class AddContactSettingsToUsers < ActiveRecord::Migration
  def change
    add_column :users, :contact_settings, :string, :limit => 4096
  end
end

You’ll likely need some space for your settings so set the string limit to something large. Alternatively, you can use text instead of string if you like, but I tend to run more conservative.

Next, we’ll need to let the user model know we’ll be using the contact_settings field as a store.

User Model
1
2
3
4
5
class User < ActiveRecord::Base
  store :contact_settings, accessors: [ :daily_email, :weekly_email, :account_email ]

  has_many :owner_restaurants
end

Well that wasn’t too difficult.

Usage

Like I said, the field used as the store is really just a hash. You can see that in the console when I retrieve the first user record:

Retrieving the First User
1
2
3
4
5
6
7
1.9.3p125 :001 > u = User.first
  User Load (0.3ms)  SELECT `users`.* FROM `users` LIMIT 1
 => #<User id: 1, name: "John Galt", email: "john@galtsgulch.com", created_at: "2012-03-08 01:50:22", updated_at: "2012-03-08 01:50:22", contact_settings: {}> 
1.9.3p125 :002 > u.contact_settings
 => {}
1.9.3p125 :003 > u.contact_settings.class
 => Hash

Instead of accessing the attributes through contact_settings as you would a normal Hash, you access them as if they were attributes on the model itself.

Accessing Store Attributes
1
2
3
4
5
6
7
8
9
10
1.9.3p125 :005 > u.weekly_email = true
 => true
1.9.3p125 :006 > u.account_email = true
 => true
1.9.3p125 :007 > u.daily_email = false
 => false
1.9.3p125 :008 > u
 => #<User id: 1, name: "John Galt", email: "john@galtsgulch.com", created_at: "2012-03-08 01:50:22", updated_at: "2012-03-08 01:50:22", contact_settings: {:weekly_email=>true, :account_email=>true, :daily_email=>false}> 
1.9.3p125 :009 > u.contact_settings
 => {:weekly_email=>true, :account_email=>true, :daily_email=>false}

As mentioned earlier, store fields are just hashes. This means you can access them and use methods on them just like any other hash. You can even add attributes not defined in the store.

Accessing Store as a Hash
1
2
3
1.9.3p125 :010 > u.contact_settings[:foo] = "bar"
1.9.3p125 :012 > u.contact_settings
 => {:weekly_email=>true, :account_email=>true, :daily_email=>false, :foo=>"bar"}

If we were to save the record and look at it in the database (without :foo => "bar"), it would look like this.

User Record
1
2
3
4
5
6
7
8
9
10
11
mysql> select * from users;
+----+--------------+----------------------+---------------------+---------------------+-------------------------------------------------------------------+
| id | name         | email                | created_at          | updated_at          | contact_settings                                                  |
+----+--------------+----------------------+---------------------+---------------------+-------------------------------------------------------------------+
|  1 | John Galt    | john@galtsgulch.com  | 2012-03-08 01:50:22 | 2012-04-04 11:23:47 | ---
:weekly_email: true
:account_email: true
:daily_email: false
 |
|  2 | Howard Roark | howard@architect.com | 2012-03-08 01:50:22 | 2012-03-08 01:50:22 | NULL                                                              |
+----+--------------+----------------------+---------------------+---------------------+-------------------------------------------------------------------+

Pros

I think many of the “pros” for ActiveRecord.store are pretty obvious: it eliminates the need for yet another table or extra fields; simplifies adding new attributes; they work just like normal model attributes. Did I mention that validations work on Store attributes? They do, and Rafal Wrzochol has a great write up on using them

So what are the drawbacks?

Cons

There are a number of philosophical reasons against using the new Store feature. The main argument against is that the data really isn’t normalized. But we’re using Rails, which means we have a tendency to abuse normalization for the sake of the application anyway.

Another minus is that dirty attributes don’t work quite as well in the Store. For instance, you can’t call User#weekly_email_changed?. The only thing you can do is check if the Store field has changed (e.g. User#contact_settings_changed?). Again, it’s not really a huge issue and I imagine this will get resolved in future releases.

Really the main “con” with regard to using store - and this really is a big deal - is that you can’t perform efficient searches on the field. The only way to perform a search is by surrounding the search term with “%” characters.

1
SELECT * FROM users WHERE contact_settings LIKE '%weekly_email: true%';

If the percent sign was just on the end, that would be one thing, but with the leading “%” it’s now the slowest possible way of searching the database.

I really think the new Store feature in Rails 3.2 is a nice feature. They’ve done it well and made its usage fairly seamless (i.e. store attributes look and act like any other attribute). If your application’s database is fairly large or if you plan on running a lot of queries against the attributes in the Store (e.g. gathering lots of metrics) you may want to use a separate table. For most applications out there, however, this is a very safe and sane solution.

Further Reader

The Need to Refactor

When people hear the words “ceremony” and “tradition”, their first thoughts usually involve religion. Of course it makes sense, all religions have some sort of tradition and ceremony, but while some of the traditions are prescribed, many have come about from some need or requirement. Those which have come about often continue on long after the requirement no longer exists. For example, the communion plates in most Christian churches are stacked and covered to keep the flies out. Flies have ceased to be a problem in our air conditioned mega-churches of the 21st century, but we continue to protect our crackers and grape juice from them nonetheless.

Businesses have ceremonies and traditions as well, only we call them “processes” and “systems”. In the world of programming we call it “technical debt”. Regardless of the name used, the only way to reduce what is unnecessary is by refactoring. The best definition of refactoring that I am aware of was provided by Martin Fowler:

Refactoring is a disciplined technique for restructuring an existing body of code, altering its internal structure without changing its external behavior. Its heart is a series of small behavior preserving transformations. Each transformation (called a ‘refactoring’) does little, but a sequence of transformations can produce a significant restructuring. Since each refactoring is small, it’s less likely to go wrong. The system is also kept fully working after each small refactoring, reducing the chances that a system can get seriously broken during the restructuring.

Businesses refactor all the time, but again the terms are different: “streamlining processes”, “downsizing”, “pivoting”, and “reorg” are only a few examples. But in the world of programming, we don’t always have the luxury of refactoring like we want, and we really want to.

When you refactor in business, there is usually some sort of immediate, “tangible” result: Suzy, Johnny, and Sally were axed so now our expenses are x amount less. In programming, if I refactor a library of code, the end result may only be that I increase its readability. How do you quantify that? When it can’t be quantified, refactoring is only seen as an expense and something to be avoided, and so the technical debt grows.

As alluded, the results of a refactoring are not always measurable. There are certainly instances which result in measurable improvements, but improvements such as increased performance are not always important. So why allow your development staff time to refactor?

Although “increased performance [is] not always important”, it oftentimes is. With increased performance and efficiency come reduced hardware requirements, translating into either lower expenses or greater output. If you have a website, it means a better user experience, greater retention, and increased revenues. Look at some of these stats provided from Velocity 2009:

  • Bing - Bing found that a 2 second slowdown changed queries/user by -1.8% and revenue/user by -4.3%. Google Search found that a 400 millisecond delay resulted in a -0.59% change in searches/user
  • Google - One experiment increased the number of search results per page from 10 to 30, with a corresponding increase in page load times from 400 milliseconds to 900 milliseconds. This resulted in a 25% drop-off in first result page searches.
  • Shopzilla - A year-long performance redesign resulted in a 5 second speed up (from ~7 seconds to ~2 seconds). This resulted in a 25% increase in page views, a 7-12% increase in revenue, and a 50% reduction in hardware.

But application performance is only part of the picture, if your application or system has customers - and it does - then those customers are going to want the occasional change made or problems addressed. The time it takes to accommodate the request is proportional to the complexity of the system (i.e. the more complex the code, the longer it’s going to take).

By providing your development staff time to refactor, you not only shore up an application’s foundation, you reduce the cost of future development. In other words, decreased complexity equates to increased productivity, resulting in happier customers.

There is one more consideration with regard to encouraging regular refactoring, and if the truth were to be told, it’s my whole motive for writing this post: working with code that has a high technical debt is miserable and it’s a morale killer. When you know that each new feature request or bug fix that comes in will require sifting through years of spaghetti code, the sense of dread becomes palpable. Who wants to work in a system where any change, any minor tweak, could result in hours of bug hunts, shoring up crumbling foundations, and frustration? So developers avoid and procrastinate, and sometimes we start looking for a new job and a new system.

I’ve worked in shops where management allowed the development staff to either completely rewrite or perform a massive overhaul on a system, and the immediate improvement in developer morale was dramatic. We began talking about work again, how to do things right, how to make it faster, we began caring about what the customer wanted again, and we weren’t dreading coming to work each day.

Of course, I’ve also worked in shops where refactoring was too “expensive” for the company. It’s one thing to “overhaul” a project which has been neglected for too long, it’s quite another to have to build upon what is already there. It’s like the difference between cleaning a house owned by a “hoarder”, and being forced to live in one.

Technical debt creeps in to every project, and that’s okay. The point is to manage it, and the only way to manage it is by regularly refactoring your code and your design. If allowances are made for refactoring with each bug fix or new feature, keeping the technical debt low is a small investment. If allowances are not made, well, it’s kinda like the differences between patching a house’s foundation and mudjacking it. Either address the problems in the foundation as they arise or watch the house it’s supporting slowly come crashing down.

Further Reading

Understanding ActiveRecord::Base.explain in Rails 3.2

With the release of Rails 3.2, comes a host of new features. There are two in particular in ActiveRecord which seem to have attracted a lot of attention, so I thought I would look at them each more closely. This post, as you have no doubt realized from the title, is focused on ActiveRecord::Base.explain

This may come as a shock, but explain is only new in Ruby on Rails; it - or similar features - has been around in the database world forever. Stated most simple, “The EXPLAIN statement provides information about the execution plan for a SELECT statement.” (MySQL 7.8.2. EXPLAIN Output Format) In other words, when you explain a query, the database returns information about how it is going to go about finding all the data. It’s a very useful tool which can help developers uncover why queries are running slowly and what might be done to speed them up.

Let’s look at an example so we can see the value of explain.

Schema Setup

We’ll need a couple tables with data for our example. I’ll use users and user_restaurants.

The users Table
1
2
3
4
5
6
7
8
9
class CreateUsers < ActiveRecord::Migration
  def change
    create_table :users do |t|
      t.string :name
      t.string :email
      t.timestamps
    end
  end
end
The user_restaurants Table
1
2
3
4
5
6
7
8
9
10
class CreateUserRestaurants < ActiveRecord::Migration
  def change
    create_table :user_restaurants do |t|
      t.integer :user_id
      t.string :name
      t.boolean :like
      t.timestamps
    end
  end
end

The users table is self-explanatory. user_restaurants, on the other hand, is a table of restaurants a user likes (I guess that’s self-explanatory as well.)

Well need a couple of users, so let’s add them here.

Populate the users Table
1
2
3
4
5
6
class AddUsers < ActiveRecord::Migration
  def up
    User.create(:name => "John Galt", :email => "john@galtsgulch.com")
    User.create(:name => "Howard Roark", :email => "howard@architect.com")
  end
end

And our users will want to “like” or “dislike” restaurants, so we’ll need to create the liked restaurant records. I’ll name the restaurants “Restaurant n” to save myself from having to come up with 100,000 restaurant names. We’ll split those restaurants between the two users and define 20% of them to be restaurants the users did not “like”.

Populate the user_restaurants Table
1
2
3
4
5
6
7
8
9
class AddRestaurants < ActiveRecord::Migration
  def up
    users = User.all

    100000.times do |i|
      UserRestaurant.create(:user_id => i % 2 == 0 ? users.first.id : users.second.id, :name => "Restaurant #{i}", :like => i % 5 == 0 ? false : true)
    end
  end
end

Model Associations

The last thing we have to do is associate the users table to the user_restaurants table.

Model Associations
1
2
3
4
5
6
7
class User < ActiveRecord::Base
  has_many :user_restaurants
end

class UserRestaurant < ActiveRecord::Base
  belongs_to :user
end

Explain Round 1

Okay, we’ve created our tables and populated them with data, and we’ve got our models and associations. Let’s run a query to find “John Galt’s” favorite restaurants and see what’s going on.

users / user_restaurants join
1
2
User.where(:email => "john@galtsgulch.com").
  joins(:user_restaurants).where("user_restaurants.like = 1")

When I run this from the Rails Console, explain kicks in automatically and returns the table below (this is output from MySQL, PostgreSQL looks and is worded differently). “Active Record monitors queries and if they take more than [config.active_record.auto_explain_threshold_in_seconds] their query plan will be logged using warn.” (What’s new in Edge Rails: EXPLAIN)

Note: To manually execute explain on a query, just append .explain to it.

Explain Output v1
1
2
3
4
5
6
+----+-------------+------------------+------+---------------+------+---------+------+--------+--------------------------------+
| id | select_type | table            | type | possible_keys | key  | key_len | ref  | rows   | Extra                          |
+----+-------------+------------------+------+---------------+------+---------+------+--------+--------------------------------+
|  1 | SIMPLE      | users            | ALL  | PRIMARY       | NULL | NULL    | NULL |      2 | Using where                    |
|  1 | SIMPLE      | user_restaurants | ALL  | NULL          | NULL | NULL    | NULL | 100041 | Using where; Using join buffer |
+----+-------------+------------------+------+---------------+------+---------+------+--------+--------------------------------+

The columns you’re going to be most interested in are “key” and “rows”. The “key” columns lists the indexes, if any, the database uses to find data. “The rows column indicates the number of rows MySQL believes it must examine to execute the query.” 7.8.2. EXPLAIN Output Format

In the output above, we can see that the query would use no indexes, and would have to search through 100,041 rows (Hey, that’s more rows than are in the database). “For InnoDB tables, this number is an estimate, and may not always be exact.” 7.8.2. EXPLAIN Output Format

Database Refactoring

Let’s add some indexes to a couple of the key table columns and see if we can’t reduce the pain.

AddTableIndexes
1
2
3
4
5
6
7
class AddTableIndexes < ActiveRecord::Migration
  def up
    add_index :users, :email
    add_index :user_restaurants, :user_id
    add_index :user_restaurants, :like
  end
end

Now, when we run our query it still issues an explain (because it’s a really stupid example), but we now see how adding the indexes improved performance.

Explain Output v2
1
2
3
4
5
6
+----+-------------+------------------+------+------------------------------------------------------------------+-----------------------------------+---------+---------------------+-------+-------------+
| id | select_type | table            | type | possible_keys                                                    | key                               | key_len | ref                 | rows  | Extra       |
+----+-------------+------------------+------+------------------------------------------------------------------+-----------------------------------+---------+---------------------+-------+-------------+
|  1 | SIMPLE      | users            | ref  | PRIMARY,index_users_on_email                                     | index_users_on_email              | 768     | const               |     1 | Using where |
|  1 | SIMPLE      | user_restaurants | ref  | index_user_restaurants_on_user_id,index_user_restaurants_on_like | index_user_restaurants_on_user_id | 5       | ar_explain.users.id | 25010 | Using where |
+----+-------------+------------------+------+------------------------------------------------------------------+-----------------------------------+---------+---------------------+-------+-------------+

Improvements

Looking at the new explain output, we see two things: 1) the query is now using the indexes we created; and 2) the number of potential rows that have to be searched through has been quartered.

It may not seem like a big deal that we went from two to one for the search by email, but imagine if you had 10,000 users, and by using the index here it was still able to go directly to that one record. And if the data we were searching for was something other than a boolean, as it is with the “like” column, we would be able to further reduce the number of potential records through which were searched.

Of course, if we didn’t have our handy dandy explain tool we might not even have realized it was a database problem to begin with.

As I alluded to in the beginning, there is nothing new about explain. It’s an excellent tool which provides insight into what course of action the database will take to find the data you are searching for. I’ve been using explain for some time directly from the database command line; it’s nice to finally have it accessible from the Rails Console.

I’ve purposely ignored the various implementations and configurations of explain in order to focus more on why it’s useful. If you are interested in configuration and implementation, check out What’s new in Edge Rails: EXPLAIN.

Further Reader

Left Pad Zeroes in JavaScript

I was recently working on a project wherein I needed to present a number padded on the left with zeroes. Most languages have some sort of formatter method which would allow you to do that, but not JavaScript. Instead of following the path of other languages, JavaScript opts to give the developer opportunities to be creative.

After a quick search, I ran across a very elegant solution from Pau Sanchez which I have turned into a the following function and added some smarts to allow the padding to be changed as necessary.

lpad
1
2
3
4
5
6
7
var lpad = function(value, padding) {
    var zeroes = "0";

    for (var i = 0; i < padding; i++) { zeroes += "0"; }

    return (zeroes + value).slice(padding * -1);
}

Note that this doesn’t account for numbers which are longer than the padding value; they get cut off. Most of the time you deal with left padded numbers, you’re dealing with a fixed width and this isn’t going to be a problem. I’ll leave it as an exercise for the user, if this is not the current case for you.

Here it is again in CoffeeScript:

lpad
1
2
3
4
5
lpad = (value, padding) ->
  zeroes = "0"
  zeroes += "0" for i in [1..padding]

  (zeroes + value).slice(padding * -1)