Samuel Mullen What could possibly go wrong?

Validating Presence of Associations and Foreign Keys in Rails

by Samuel Mullen

Posted on Dec 31, 2013


The Problem

When working with ActiveRecord models, there are occasions where using either a foreign key or an object to denote an association is desirable. It may be the only available information about the foreign record is the key, or it may be that only a “new” (i.e. not persisted) object has been created. To ensure the association is present, you need to validate the presence of both the association (i.e. the Object) and the foreign key.

Assuming we have a User model with a has_many association to a Post model, the solution might look something like this:

class Post < ActiveRecord::Base
  belongs_to :user, :inverse_of => :posts
  
  validates :user, :presence => {:if => proc{|o| o.user_id.blank? }}
  validates :user_id, :presence => {:if => proc{|o| o.user.blank? }}
end

Here, we’re validating the existence of the User association: first by checking the presence of the object if the foreign key is blank; second by checking the presence of the foreign key if the User association is blank.

This isn’t a great solution, because it results in two error messages if the validation fails.

A New Validation

A better solution is to create a validator. To do that in Rails, the validator file must be created under lib/validators, and the filename must be the snake case version of the class name (i.e. ExistenceValidator => existence_validator.rb).

class ExistenceValidator < ActiveModel::EachValidator
  def validate_each(record, attribute, value)
    if value.blank? && record.send("#{attribute}_id".to_sym).blank?
      record.errors[attribute] << I18n.t("errors.messages.existence")
    end
  end
end

lib/validators/existence_validator.rb

As you can see, we subclass ActiveModel::EachValidator and override the validate_each method.

Inside the method, we add to the errors list if either the value of what we are testing is #blank? or if the foreign key is #blank? Just add the desired error message to your config/locales/en.yml file (adjust for language).

en:
  errors:
    messages:
      existence: "must exist as an object or foreign key"

config/locales/en.yml

Aside: #send

The above solution may be a little tricky if you’re not too familiar with metaprogramming in Ruby, so we’ll look at it a little more closely here.

record.send("#{attribute}_id".to_sym).blank?

In Ruby, the #send method takes a symbol as its first argument, and it “sends” that symbol as a message to an object, (i.e. it calls a method of the same name as the symbol).

We are defining the message to be sent with the "#{attribute}_id".to_sym bit – in our use case, it will end up being user_id. That message gets sent to the model instance the validation is defined within and determined if it is blank?.

When fully translated, it looks like this:

record.user_id.blank?

Usage

Now that we’ve defined our new validation, let’s use it. We can remove the two presence validators we had before and replace them with the more succinct existence validator.

class Post < ActiveRecord::Base
  belongs_to :user, :inverse_of => :posts
  
  validates :user, :existence => true
end

Post model using new validator

Where It Doesn’t Work

The solution is pretty good, but it assumes the foreign key is the same name as the association, only with an appended _id (e.g. user and user_id).

For the occasion when you need to get around this, you’ll want to create a sort of “in class” validator for your odd association.

class Post < ActiveRecord::Base
  belongs_to :author, :class_name => User, :inverse_of => :posts

  validate :must_have_author

  private

  def must_have_author
    if author.blank? && user_id.blank?
      self.errors.add(:author, I18n.t("errors.messages.existence")) 
    end
  end
end

Post model using specific validator

At this point you should have a solution for 90% of what you’re going to run into with regard to validating associations. If you see something I missed, or if there’s a way I can improve anything above, please leave a comment and I’ll update the post accordingly.

Read More

The Problem with Rails Callbacks

by Samuel Mullen

Posted on May 7, 2013


If you search StackOverflow for “Rails callbacks”, a large number of the results pertain to seeking means to avoid issuing the callback in certain contexts. It almost seems as though Rails developers discover a need to avoid callbacks as soon as they discover their existence.

Normally, this would be a cause for concern, that perhaps the feature should be avoided altogether or even removed, but callbacks are still part of Rails. Maybe the problem goes deeper.

What is a Callback?

As you likely already know, callbacks are just hooks into an ActiveRecord object’s life cycle. Actions can be performed “before”, “after”, or even “around” ActiveRecord events, such as save, validate, or create. Also, callbacks are cumulative, so you can have two actions which occur before_update, and those callbacks will be executed in the order they are occur.

Where Trouble Begins

Developers usually start noticing callback pain during testing. If you’re not testing your ActiveRecord models, you’ll begin noticing pain later as your application grows and as more logic is required to call or avoid the callback.

I say, “developers usually start noticing callback pain during testing” because in order to speed up tests or to get them to pass, it becomes necessary to “stub out” the callback actions. If you don’t stub out the action, then you must add the supporting data structure, class, and/or logic to each test in order for it to pass.

Here’s an example of what I mean:

class Post < ActiveRecord::Base
  has_many :followers
  after_save :notify_followers
  
  def publish!
    self.published_at = Time.now
    save
  end
  
  private
  
  def notify_followers
    Notifier.post_mailer.deliver
  end
end

describe "publishing the article" do
  it "saves the object with a defined published_at value" do
    Post.any_instance.stub(:notify_followers) # Codey McSmellsalot
    post = Post.new(:title => "The Problem with Callbacks in Rails")
    post.publish!
    expect(post.published_at).to be_an_kind_of(Time)
    expect(post).to_not be_a_new_record
  end
end

In order to get that example code to pass, notify_followers must be “stubbed out”. If it isn’t, and if followers are used within the mailer, the test will fail because it’s not able to execute the delivery (i.e. it’ll error out due to nil values).

What About Observers?

Rails developers who’ve begun moving into a more Object Oriented mindset might ask, “What about using observers instead of callbacks?” It’s the right direction: by creating an observer, you move responsibilities which don’t belong in the object being observed to the observer.

The problem is that observers in Rails are kind of like ninja callbacks: they perform the same function as callbacks, they just work in the shadows. Unless you look at the file system, you are very likely to forget Observers even exist in your application.

Furthermore, observers are assigned to their appropriate class when Rails starts up, and Rails starts up when you run your tests. Once again, you’ll start feeling pain in your tests first, because in order to avoid observer calls in your tests, you will need to either create all the dependent objects or install a gem such as no_peeping_toms. Just like callbacks, observers run every time their condition is met.

Aside: Herman Moreno wrote a good post on undocumented observer usage: Fun with ActiveRecord::Observer.

Why Are Callbacks So Problematic?

In his post on ActiveRecord, Caching, and the Single Responsibility Principle, Joshua Clayton noticed “after_* callbacks on Rails models seem to have some of the most tightly-coupled code, especially when it comes to models with associations.”

It’s no coincidence. “before_” callbacks are generally used to prepare an object to be saved. Updating timestamps or incrementing counters on the object are the sort of things we do “before” the object is saved. On the other hand, “after_*” callbacks are primarily used in relation to saving or persisting the object. Once the object is saved, the purpose (i.e. responsibility) of the object has been fulfilled, and so what we usually see are callbacks reaching outside of its area of responsibility, and that’s when we run into problems.

Solving the Problem

Jonathan Wallace, over at the Big Nerd Ranch, ran into to same problems and came up with one simple rule: “Use a callback only when the logic refers to state internal to the object.” (The only acceptable use for callbacks in Rails ever)

If we can’t use callbacks which extend responsibility outside their class, what do we do? We make an object whose responsibility is to handle that callback.

Before Example

Let’s look at a hypothetical example. This is what we might originally have:

class Order < ActiveRecord::Base
  belongs_to :user
  has_many :line_items
  has_many :products, :through => :line_items
  
  after_create :purchase_completion_notification
  
  private
  
  def purchase_completion_notification
    Notifier.purchase_notifier(self).deliver
  end
end
class Notifier < ActionMailer...
  def purchase_notifier(order)
    @order = order
    @user = order.user
    @products = order.products

    rest of the action mailer logic
  end
end

In the above example we can see that when an order is saved, it’s going to shoot off an email to the customer. That Mailer is going to use the order object to retrieve the ordering user and the products which were purchased and likely use them in the email. Pretty simple, right?

In a test, however, any time an order is saved to the database, user, line_items, and products will need to be created, or the purchase_notifier method will need to be stubbed out – Order.any_instance.stub(:purchase_notifier) (Ugh).

After Example

Here’s what happens when we move some responsibilities:

Our Order model is much simpler.

class Order < ActiveRecord::Base
  belongs_to :user
  has_many :line_items
  has_many :products, :through => :line_items
end

Here’s our new class:

class OrderCompletion
  attr_accessor :order
  
  def initialize(order)
    @order = order
  end
  
  def create
    if self.order.save
      self.purchase_completion_notification
    end
  end
  
  def purchase_completion_notification
    Notifier.purchase_notifier.deliver(self.order)
  end
end

What we’ve done above is moved the process of saving the order and sending the notification out of the Order model and into a PORO (Plain Old Ruby Object) class. This class is responsible for completing an order. Y’know, saving the order and letting the customer know it worked.

By doing this, we no longer have to stub out the notification method in our tests, we’ve made it a simple matter to create an order without requiring an email to be sent, and we’re following good Object Oriented design by making sure our classes have a single responsibility (SRP).

It’s a simple matter to use this in our controller too:

def create
  @order = Order.new(params[:order])
  @order_completion = OrderCompletion.new(@order)
  
  if @order_completion.create
    redirect_to root_path, notice: 'Your order is being processed.'
  else
    @order = @order_completion.order
    render action: "new"
  end
end

Wrapping up

As much as I complain about callbacks, they’re really not bad as long as you remember the rule: “Use a callback only when the logic refers to state internal to the object.” And really, that can be applied to any method.

When you start to feel those first twinges of pain from your tests, whether from callbacks or otherwise, consider if what you are trying to do exceeds your class’s responsibility. Creating a new class is a simple matter, especially compared to the pain and frustration caused by not doing it.

Many thanks to Pat Shaughnessy for proof-reading and providing feedback.

Read More

Step By Step: Setting Up minitest and Cucumber with Rails

by Samuel Mullen

Posted on Nov 10, 2012


I’ve tried a couple times in the past to successfully set up a Rails application with both minitest and Cucumber; each time, I ran in to issues. I’m not entirely sure why I ran into the issues I did, but I just couldn’t seem to get things working. Recently, I managed to get everything going with almost no effort. Here’s what I did.

Step 1: Creating and Configuring a New Rails App with minitest

For the sake of simplicity, we’ll assume that we’re creating a new Rails application. To do so, we’ll want to skip the standard usage of Test::Unit

rails new <app_name> --skip-test-unit

Immediately after this, we’ll need to install and initialize the minitest-rails gem

group :test, :development do
  ...
  gem 'minitest-rails', '~> 0.3.0` # as of this writing
  ...
end

bundle install and then initialize the gem in the app.

rails g mini_test:install

The last thing we’ll do is make sure our generators behave accordingly. To do that, add the following logic to the config/application.rb config block:

config.generators do |g|
  g.test_framework :mini_test, :spec => true, :fixture => false
end

This turns on minitest’s spec functionality and turns off fixture generation. If you don’t want to use minispec or if you do want to use fixtures, just alter the values accordingly.

Step 2: Setting up Cucumber

Okay, at this point we have our foundational test suite set up. Now we need to add cucumber to the mix.

Add cucumber-rails to your Gemfile as you normally would.

group :test do
  gem 'cucumber-rails', "~> 1.3.0"
  gem "database_cleaner", "~> 0.9.1"
end

Call bundle install and then call the cucumber Rails generator:

rails g cucumber:install

Now, add the following lines of code to features/support/env.rb. (I placed mine near the top.)

require 'minitest/spec'
World(MiniTest::Assertions)
MiniTest::Spec.new(nil)

For greater separation, the previous lines of code above can be added to a different support file.

Step 3: Running the Tests

At this point, everything should work. Let’s find out with a couple simple tests.

minispec Test

Create the following test file and model:

require_relative "../minitest_helper"

describe Foo do
  it "must be valid" do
    Foo.must_respond_to :new
  end
end

And the corresponding model file:

class Foo
end

Notice in the first line of foo_test.rb that we’re using require_relative instead of a normal require call. We do this to avoid the need to use rake. Now, rather than running bundle exec rake minitest:models and running all the models, we can execute bundle exec ruby test/models/foo_test.rb.

Cucumber Test

Create the following Cucumber “feature” file:

Feature: test
  In order to test cucumber
  a user
  wants to successfully land on the home page

  Scenario: foo
    Given I am on the home page
     Then I should see "Ruby on Rails: Welcome aboard"

And an associated step file:

Given /^I am on the home page$/ do
  visit "/"
end

Then /^I should see \"([^"]+)"$/ do |string|
  page.has_content?(string).must_equal true
end

Run the test and everything should be green.

bundle exec cucumber features/test.feature

Final Thoughts

I like minitest a lot, mainly because it’s fast, it’s simple, and it’s included in Ruby 1.9+, but there are drawbacks; the two main ones for me are its readability and the lack of integration with the vim-rails plugin. Neither of these “drawbacks” are deal breakers, however: the vim-rails plugin will eventually support it, and there are gems to make the suite more “civilized” (listed below).

So if you like the idea of a really simple and fast test suite, and you don’t mind the syntax, give minitest a go on your next project. Oh, and let me know how it works out for you.

Further Reading

Read More

Getting More Out of the Rails Console

by Samuel Mullen

Posted on Jul 11, 2012


Even at it’s most basic functionality, the console is indispensable in the Rails developer’s arsenal of tools. Whether it’s running a query, testing a chain of methods, or executing small blocks of code, the console may be the most useful tool available to the Rails developer. With that in mind, doesn’t it make sense to do what we can to get the most out of it?

IRB

.irbrc

Under the hood, the Rails Console is just IRB (Interactive Ruby), so anything you can do with IRB, you can do in the console. This means you can modify your IRB environment and .irbrc file to define methods to use at the console. Here are three methods I frequently use:

\# return a sorted list of methods minus those which are inherited from Object
class Object
  def interesting_methods
    (self.methods - Object.instance_methods).sort
  end
end

\# return an Array of random numbers
class Array
  def self.test_list(x=10)
    Array(1..x)
  end
end

\# return a Hash of symbols to random numbers
class Hash
  def self.test_list
    Array(:a..:z).each_with_object({}) {|x,h| h[x] = rand(100) }
  end
end

Of course this isn’t even scratching the surface of what’s possible. Check out what other people are doing:

The Last Expression

While working in the console, have you ever typed out a bit of code to return a value and then realize you forgot to assign the returned value to a variable? You then have to go back into the console history, move your cursor to the beginning of the line, add the variable, and then execute the code again.

Ugh, what a pain!

Unbeknownst to most people, IRB places the output of the last command into the _ variable. Here, let me show you:

 > Array(1..10)
 => [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
 > _
 => [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

Now, as cool as that is, you have to understand that _ always contains the output of the last expression. This means if you try call a method on it, it will then contain the output of the method executed.

 > Array(1..10)
 => [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
 > _
 => [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
 >  _.map {|i| i * 2}
 => [2, 4, 6, 8, 10, 12, 14, 16, 18, 20]
 > _
 => [2, 4, 6, 8, 10, 12, 14, 16, 18, 20]

Forking IRB

When working in the console, it’s sometimes desirable to have another instance to play with. It may be that you don’t want to lose what you were working with, or you just need another scratch area, but whatever the case, you can create a new console (IRB) session by calling irb at the prompt (note: you’ll use irb for the rails console as well).

I typically don’t use this. If I need another IRB instance, I just open a new tmux pane or window and work there.

If this sort of method fits your workflow, I highly recommend reading Gabriel Horner’s in depth post on IRB commands

The Rails Console

Models

One of the things you will want to make extensive use of in the console are your app’s models. The Rails console is a great way to play with your models and an alternative way of accessing your data.

 > u = User.find 1234; nil
=> nil

 > u.name
 => "Foo Man"

 > u.email = "fooman@example.com"
 => "fooman@example.com"

 > u.save
 => true

The “app” object

The app object is used by test processes to mimic system interactions. Through this object, we can access routing information and even make requests to our app.

# displaying path information
 > app.users_path
=> "/users/

 > app.user_path(User.last)
=> "/users/42

# making app requests

 > app.get app.user_path(User.last)
=> 200

 > app.response.body
=>  => "<!DOCTYPE html>\n<html>\n  <head>\n    <title>..."

The “helper” Object

I really don’t think I can do a better job on this than what Nick Quaranto already did in his “Three Quick Rails console tips” post.

Reloading the Environment

If you make changes to your app while still in the console, you will need to reload your console session with the reload! command. You will also need to reinstantiate objects which existed prior to the reload for them to recognize your changes.

 > u = User.find 1234; nil

\# changes made to User model outside the console 

 > reload!
Reloading...
 => true
 > u = User.find 1234; nil

Better Output

At one time it seemed like everyone was writing a gem for improving the IRB experience, but it appears like that particular endeavor has since been largely ignored. The one project that appears to be currently active is the awesome_print gem.

I’ve used this gem in the past, and it really does improve the output and IRB experience. It also supports pry.

In a pinch, you can format the output as YAML with the y command.

 > y Array.test_list(5)
---
- 1
- 2
- 3
- 4
- 5
 => nil

Avoiding Output

Running commands in the console can get pretty noisy. Output follows every command which is run. To get around this, just end your command with a semicolon and nil.

 > u = User.last; nil
=> nil

In the above example, u still contains the “last” user record, it just doesn’t print out all the output that would normally be produced.

The Sandbox

Sometimes it would be nice to open up a console session and mess around with the data to see what happens. But if you do that, the data’s messed up. The solution to that is to lunch the console with the --sandbox flag. When launched, you can handle the data, tweak it, and destroy it, all without fear of harming any of your data.

rails console --sandbox

Loading development environment in sandbox (Rails 3.2.1)
Any modifications you make will be rolled back on exit
 >

Conclusion

In my workflow, the rails console is an indispensible tool. It allows me to test out ideas, access the database, run minor tasks, and even calculate basic math. I can’t imagine developing Rails applications without it, because I know how painful it is to be deprived of such a tool in other languages and frameworks.

What are your favorite IRB and console tricks?

Read More

Validating Booleans

by Samuel Mullen

Posted on Jun 14, 2012


I ran into an instance today wherein I needed to validate that a boolean field was either true or false and not null. I tried using validates :fieldname, :presence => true, but since :presence uses #blank? under the hood, it was reading false as not being present. (Why is false considered blank?)

Anyway, I needed a validator to test whether an attribute was either true or false and I couldn’t find anything among the standard validators, so I wrote my own.

Just plop this file in your app’s lib/validators directory.

class TruthinessValidator < ActiveModel::EachValidator
  def validate_each(record, attribute, value)
    unless value == true || value == false
      record.errors[attribute] << "must be true or false"
    end
  end
end

In your model, add the validation like so:

class SomeModel < ActiveRecord::Base
  ...

  validates :field_name, :truthiness => true

  ...
end

For more information on writing validators, see Getting Started with Custom Rails3 Validators.

Read More

ActiveRecord::Base.store - Pros, Cons, and Usage

by Samuel Mullen

Posted on Apr 2, 2012


In my post on ActiveRecord’s new explainmethod, I mentioned there were “two” features in ActiveRecord (introduced in Rails 3.2) I wanted to look at. This post covers the second method: ActiveRecord::Base.store.

Store provides a simple means of accessing and storing key/value pairs related to a model. The API documentation uses the example of a User model which has settings. “Settings” may not in themselves warrant their own model, but there still needs to be a means of accessing them. This is where Store comes in to play.

Behind the curtain, store is just a Hash which gets serialized and deserialized upon save and load. It has a few accessors added in to make it look less hashy, but the hash is still there nonetheless as we’ll see later.

Prior to Rails 3.2, if you needed this functionality you had three choices: 1) implement it yourself; 2) Muck up your table with a bunch of extra fields; 3) create another table to store all the fields.

For the sake of laziness, I’ll use the same project I used in my previous post: https://github.com/samullen/ar_explain

Setup

In our example, we’re going to allow users to toggle the different types of email they want to receive. The first thing we’ll need to do is add the field in which to store the settings.

class AddContactSettingsToUsers < ActiveRecord::Migration
  def change
    add_column :users, :contact_settings, :string, :limit => 4096
  end
end

You’ll likely need some space for your settings so set the string limit to something large. Alternatively, you can use text instead of string if you like, but I tend to run more conservative.

Next, we’ll need to let the user model know we’ll be using the contact_settings field as a store.

class User < ActiveRecord::Base
  store :contact_settings, accessors: [ :daily_email, :weekly_email, :account_email ]
  
  has_many :owner_restaurants
end

Well that wasn’t too difficult.

Usage

Like I said, the field used as the store is really just a hash. You can see that in the console when I retrieve the first user record:

1.9.3p125 :001 > u = User.first
  User Load (0.3ms)  SELECT `users`.* FROM `users` LIMIT 1
 => #<User id: 1, name: "John Galt", email: "john@galtsgulch.com", created_at: "2012-03-08 01:50:22", updated_at: "2012-03-08 01:50:22", contact_settings: {}> 
1.9.3p125 :002 > u.contact_settings
 => {} 
1.9.3p125 :003 > u.contact_settings.class
 => Hash 

Instead of accessing the attributes through contact_settings as you would a normal Hash, you access them as if they were attributes on the model itself.

1.9.3p125 :005 > u.weekly_email = true
 => true 
1.9.3p125 :006 > u.account_email = true
 => true 
1.9.3p125 :007 > u.daily_email = false
 => false 
1.9.3p125 :008 > u
 => #<User id: 1, name: "John Galt", email: "john@galtsgulch.com", created_at: "2012-03-08 01:50:22", updated_at: "2012-03-08 01:50:22", contact_settings: {:weekly_email=>true, :account_email=>true, :daily_email=>false}> 
1.9.3p125 :009 > u.contact_settings
 => {:weekly_email=>true, :account_email=>true, :daily_email=>false} 

As mentioned earlier, store fields are just hashes. This means you can access them and use methods on them just like any other hash. You can even add attributes not defined in the store.

1.9.3p125 :010 > u.contact_settings[:foo] = "bar"
1.9.3p125 :012 > u.contact_settings
 => {:weekly_email=>true, :account_email=>true, :daily_email=>false, :foo=>"bar"} 

If we were to save the record and look at it in the database (without :foo => "bar"), it would look like this.

mysql> select * from users;
+----+--------------+----------------------+---------------------+---------------------+-------------------------------------------------------------------+
| id | name         | email                | created_at          | updated_at          | contact_settings                                                  |
+----+--------------+----------------------+---------------------+---------------------+-------------------------------------------------------------------+
|  1 | John Galt    | john@galtsgulch.com  | 2012-03-08 01:50:22 | 2012-04-04 11:23:47 | ---
:weekly_email: true
:account_email: true
:daily_email: false
 |
|  2 | Howard Roark | howard@architect.com | 2012-03-08 01:50:22 | 2012-03-08 01:50:22 | NULL                                                              |
+----+--------------+----------------------+---------------------+---------------------+-------------------------------------------------------------------+

Pros

I think many of the “pros” for ActiveRecord.store are pretty obvious: it eliminates the need for yet another table or extra fields; simplifies adding new attributes; they work just like normal model attributes. Did I mention that validations work on Store attributes? They do, and Rafal Wrzochol has a great write up on using them

So what are the drawbacks?

Cons

There are a number of philosophical reasons against using the new Store feature. The main argument against is that the data really isn’t normalized. But we’re using Rails, which means we have a tendency to abuse normalization for the sake of the application anyway.

Another minus is that dirty attributes don’t work quite as well in the Store. For instance, you can’t call User#weekly_email_changed?. The only thing you can do is check if the Store field has changed (e.g. User#contact_settings_changed?). Again, it’s not really a huge issue and I imagine this will get resolved in future releases.

Really the main “con” with regard to using store - and this really is a big deal - is that you can’t perform efficient searches on the field. The only way to perform a search is by surrounding the search term with “%” characters.

SELECT * FROM users WHERE contact_settings LIKE '%weekly_email: true%';

If the percent sign was just on the end, that would be one thing, but with the leading “%” it’s now the slowest possible way of searching the database.

I really think the new Store feature in Rails 3.2 is a nice feature. They’ve done it well and made its usage fairly seamless (i.e. store attributes look and act like any other attribute). If your application’s database is fairly large or if you plan on running a lot of queries against the attributes in the Store (e.g. gathering lots of metrics) you may want to use a separate table. For most applications out there, however, this is a very safe and sane solution.

Further Reader

Read More

Understanding ActiveRecord::Base.explain in Rails 3.2

by Samuel Mullen

Posted on Mar 7, 2012


With the release of Rails 3.2, comes a host of new features. There are two in particular in ActiveRecord which seem to have attracted a lot of attention, so I thought I would look at them each more closely. This post, as you have no doubt realized from the title, is focused on ActiveRecord::Base.explain

This may come as a shock, but explain is only new in Ruby on Rails; it - or similar features - has been around in the database world forever. Stated most simple, “The EXPLAIN statement provides information about the execution plan for a SELECT statement.” (MySQL 7.8.2. EXPLAIN Output Format) In other words, when you explain a query, the database returns information about how it is going to go about finding all the data. It’s a very useful tool which can help developers uncover why queries are running slowly and what might be done to speed them up.

Let’s look at an example so we can see the value of explain.

Schema Setup

We’ll need a couple tables with data for our example. I’ll use users and user_restaurants.

class CreateUsers < ActiveRecord::Migration                                     
  def change                                                                    
    create_table :users do |t|                                                  
      t.string :name                                                            
      t.string :email                                                                
      t.timestamps                                                                  
    end                                                                         
  end                                                                           
end
class CreateUserRestaurants < ActiveRecord::Migration                       
  def change                                                                    
    create_table :user_restaurants do |t|                                  
      t.integer :user_id                                                        
      t.string :name                                                                  
      t.boolean :like                                                                 
      t.timestamps                                                              
    end                                                                         
  end                                                                           
end    

The users table is self-explanatory. user_restaurants, on the other hand, is a table of restaurants a user likes (I guess that’s self-explanatory as well.)

Well need a couple of users, so let’s add them here.

class AddUsers < ActiveRecord::Migration                                        
  def up                                                                        
    User.create(:name => "John Galt", :email => "john@galtsgulch.com")          
    User.create(:name => "Howard Roark", :email => "howard@architect.com")      
  end                                                               
end

And our users will want to “like” or “dislike” restaurants, so we’ll need to create the liked restaurant records. I’ll name the restaurants “Restaurant n” to save myself from having to come up with 100,000 restaurant names. We’ll split those restaurants between the two users and define 20% of them to be restaurants the users did not “like”.

class AddRestaurants < ActiveRecord::Migration                                  
  def up                                                                        
    users = User.all                                                            
                                                                                
    100000.times do |i|
      UserRestaurant.create(:user_id => i % 2 == 0 ? users.first.id : users.second.id, :name => "Restaurant #{i}", :like => i % 5 == 0 ? false : true)                
    end
  end
end 

Model Associations

The last thing we have to do is associate the users table to the user_restaurants table.

class User < ActiveRecord::Base
  has_many :user_restaurants
end

class UserRestaurant < ActiveRecord::Base
  belongs_to :user
end

Explain Round 1

Okay, we’ve created our tables and populated them with data, and we’ve got our models and associations. Let’s run a query to find “John Galt’s” favorite restaurants and see what’s going on.

User.where(:email => "john@galtsgulch.com").
  joins(:user_restaurants).where("user_restaurants.like = 1")

When I run this from the Rails Console, explain kicks in automatically and returns the table below (this is output from MySQL, PostgreSQL looks and is worded differently). “Active Record monitors queries and if they take more than [config.active_record.auto_explain_threshold_in_seconds] their query plan will be logged using warn.” (What’s new in Edge Rails: EXPLAIN)

Note: To manually execute explain on a query, just append .explain to it.

+----+-------------+------------------+------+---------------+------+---------+------+--------+--------------------------------+
| id | select_type | table            | type | possible_keys | key  | key_len | ref  | rows   | Extra                          |
+----+-------------+------------------+------+---------------+------+---------+------+--------+--------------------------------+
|  1 | SIMPLE      | users            | ALL  | PRIMARY       | NULL | NULL    | NULL |      2 | Using where                    |
|  1 | SIMPLE      | user_restaurants | ALL  | NULL          | NULL | NULL    | NULL | 100041 | Using where; Using join buffer |
+----+-------------+------------------+------+---------------+------+---------+------+--------+--------------------------------+

The columns you’re going to be most interested in are “key” and “rows”. The “key” columns lists the indexes, if any, the database uses to find data. “The rows column indicates the number of rows MySQL believes it must examine to execute the query.” 7.8.2. EXPLAIN Output Format

In the output above, we can see that the query would use no indexes, and would have to search through 100,041 rows (Hey, that’s more rows than are in the database). “For InnoDB tables, this number is an estimate, and may not always be exact.” 7.8.2. EXPLAIN Output Format

Database Refactoring

Let’s add some indexes to a couple of the key table columns and see if we can’t reduce the pain.

class AddTableIndexes < ActiveRecord::Migration
  def up
    add_index :users, :email
    add_index :user_restaurants, :user_id
    add_index :user_restaurants, :like
  end
end

Now, when we run our query it still issues an explain (because it’s a really stupid example), but we now see how adding the indexes improved performance.

+----+-------------+------------------+------+------------------------------------------------------------------+-----------------------------------+---------+---------------------+-------+-------------+
| id | select_type | table            | type | possible_keys                                                    | key                               | key_len | ref                 | rows  | Extra       |
+----+-------------+------------------+------+------------------------------------------------------------------+-----------------------------------+---------+---------------------+-------+-------------+
|  1 | SIMPLE      | users            | ref  | PRIMARY,index_users_on_email                                     | index_users_on_email              | 768     | const               |     1 | Using where |
|  1 | SIMPLE      | user_restaurants | ref  | index_user_restaurants_on_user_id,index_user_restaurants_on_like | index_user_restaurants_on_user_id | 5       | ar_explain.users.id | 25010 | Using where |
+----+-------------+------------------+------+------------------------------------------------------------------+-----------------------------------+---------+---------------------+-------+-------------+

Improvements

Looking at the new explain output, we see two things: 1) the query is now using the indexes we created; and 2) the number of potential rows that have to be searched through has been quartered.

It may not seem like a big deal that we went from two to one for the search by email, but imagine if you had 10,000 users, and by using the index here it was still able to go directly to that one record. And if the data we were searching for was something other than a boolean, as it is with the “like” column, we would be able to further reduce the number of potential records through which were searched.

Of course, if we didn’t have our handy dandy explain tool we might not even have realized it was a database problem to begin with.

As I alluded to in the beginning, there is nothing new about explain. It’s an excellent tool which provides insight into what course of action the database will take to find the data you are searching for. I’ve been using explain for some time directly from the database command line; it’s nice to finally have it accessible from the Rails Console.

I’ve purposely ignored the various implementations and configurations of explain in order to focus more on why it’s useful. If you are interested in configuration and implementation, check out What’s new in Edge Rails: EXPLAIN.

##Further Reader * What’s new in Edge Rails: EXPLAIN * MySQL: EXPLAIN Output Format * MySQL: Optimizing Queries with EXPLAIN * SQLite: EXPLAIN Query Plan * PostgreSQL: Using EXPLAIN

Read More

Guidelines for Using ActiveRecord Callbacks

by Samuel Mullen

Posted on Jan 17, 2012


ActiveRecord callbacks are awesome little devices which allow you to “hook” in to the life cycle of an ActiveRecord object. But from a quick Googling of “ActiveRecord callbacks”, you might not come to the same conclusion. Within the first ten results (as of this writing), there are four which are either looking for or providing ways of circumventing the triggering of callbacks.

Why would you want to skip callbacks? I can think of lots of reasons, but every single one of them is the result of models being too tightly coupled to other objects and responsibilities. Examples: sending emails, logging, updating related or unrelated tables, etc. If your callback stays within its object’s area of responsibility it should never need to be skipped.

Generally speaking - and I do mean “generally” - an ActiveRecord model has one responsibility: interacting with the database. When your model exceeds that one mandate, it immediately becomes more difficult to maintain. I would even go so far as to say that for every responsibility added to a model or any other object, the difficulty of maintenance doubles (No hard numbers; just shooting from the hip.)

To avoid this exponential growth in model maintenance pain, I’ve come up with three guidelines to determine if using a callback is appropriate.

Guideline #1: Use callbacks only if they can be run every time and in every circumstance they are triggered.

This rule is pretty straightforward, but let’s look at it for the sake of clarity.

Callbacks are executed every time their triggering event occurs (e.g. A before_save callback will be triggered before every save event). If there is any instance in which it shouldn’t be called, then think about moving the logic to a decorator object.

Consider also, context. If a callback cannot be issued in every circumstance (e.g. development, production, and test), that is a warning the callback needs to be refactored into something less coupled.

Guideline #2: Never create callbacks which exceed the model’s responsibility.

If a class has more than one responsibility, then the responsibilities become coupled. Changes to one responsibility may impair or inhibit the class’ ability to meet the others. – Uncle Bob Martin Single Responsibility Principle

As mentioned previously, the responsibility of an ActiveRecord model is to interact with the database. When we require our models to send emails, create files, or what have you, we exceed our model’s one responsibility. This doesn’t sound like a bad thing at first, but as requirements change - and they always change - the model will be required to do more and more in order to meet the added responsibilities.

By limiting callbacks to the scope of the model’s responsibility, we simplify maintenance and testing and keep the application prepared for further growth.

Guideline #3: If you have to stub callbacks during testing, you are violating one of the previous rules of thumb.

RSpec now has a nifty little method called any_instance which it inherited from Mocha. This method allows a developer to apply a bit of logic to “any instance” of a particular class. In the past I have used any_instance to keep my models from sending emails during testing. I did this because the emails were dependent upon more than just the one model and would therefore fail during unit testing. By stubbing out the callback I was saying 1) my callback couldn’t by executed every time and in every circumstance, and 2) it was exceeding the model’s responsibility. In other words I was violating the first two rules of thumb.

I’m sure there are valid reasons for stubbing your callbacks during normal unit testing (what some call integration testing), but in general, stubbing every instance of a callback in order to get a test to pass is an implication your callbacks have exceeded their responsibility.

I have said somewhat facetiously, “callbacks are the devil.” I usually say something like this after dealing with a callback which was doing more than it should. Callbacks, like any tool, can be exceptionally useful and are no more “the devil” than any other tool. It’s when we make our tools exceed their responsibility that they get a little devilish.

Further Reading:

Read More

David Chelimsky's Answer to Testing Named Scopes

by Samuel Mullen

Posted on Mar 18, 2011


I was looking for the best way to test named scopes in Rails using Rspec and I ran across the following answer by David Chelimsky (Rspec’s creator) in the Rspec Google Group:

describe User, ".admins" do 
  it "includes users with admin flag" do 
    admin = User.create! :admin => true 
    User.admin.should include(admin) 
  end 
 
  it "excludes users without admin flag" do 
    non_admin = User.create! :admin => false 
    User.admin.should_not include(non_admin) 
  end 
end 
 
class User < ActiveRecord::Base 
  named_scope :admins, :conditions => {:admin => true} 
end 

Read the entire thread for context.

Read More

Named Date/Time Formats in Ruby on Rails

by Samuel Mullen

Posted on Sep 8, 2009


When I first began writing perl code in the mid 90’s I was amazed at how easy it was to program most tasks in contrast to programming them in C. It was fun to program again. Now, as I come to Ruby and its relevant web frameworks I am again rediscovering the joy of programming I didn’t realize I had begun to lose.

One of the tasks I always hated was date comparisons and date formatting. In perl, you grab two dates, change them to UNIX epoch seconds, compare them, and then run some calculations on that comparison to return some value. It’s easier now with the DateTime module, but for the longest time, that’s how you did it. Date formatting was similar; you had to hand-roll it.

I’m not using perl anymore - well, I am, but only until I can port things to Ruby - but I still find myself doing things the “perl way”. Until last week, I’d been using Ruby’s implementation of strftime to do most of my date formatting. And then I found this little nugget in the Rails API under ActiveSupport::CoreExtensions::Date::Conversions” and its sister “ActiveSupport::CoreExtensions::Time::Conversions

You can add your own formats to the Date::DATE_FORMATS hash. Use the format name as the hash key and either a strftime string or Proc instance that takes a date argument as the value.

Date::DATE_FORMATS[:month_and_year] = "%B %Y"
Date::DATE_FORMATS[:short_ordinal] = lambda { |date| date.strftime("%B #{date.day.ordinalize}") }

Huh? What it means is that I can add now add my own personal date/time formats and call them with a to_s on any Date object in Rails. Here’s an example from my config/initializers/datetime_formats.rb file:

datetime_formats = {
  :standard => "%b %d, %Y"
}
Date::DATE_FORMATS.merge!(
  datetime_formats.merge(
    :ymd => "%Y%m%d"
  ) 
)
Time::DATE_FORMATS.merge!(
  datetime_formats
)

In the above example datetime_formats is a list of those formats which are common to both Date::DATE_FORMATS and Time::DATE_FORMATS. To bring in Time or Date specific formats, we just merge them into the datetime_formats hash.

Now, when you need to apply a format, say to the created_at field of an ActiveRecord object, just call to_s(:format_name). Here’s an example:

x = Post.find(100)
x.created_at.to_s(:standard) # returns "Sep 08, 2009"

It’s these little things that make programming fun again.

Read More

About me


I live in the greater Kansas City area with my beautiful wife, our two great kids, and our dog. I've been programming using Open Source technologies since '97 and I'm currently an independent software developer specializing in Ruby on Rails and iOS. I am for hire.