I’ve been having a lot of conversations via email with people using Searchlogic, that go like the following:

Some person: Hi Ben, How do I order by an order’s total with Searchlogic?

Me: Cache the total in your database. Your idol, Ben

Some person: Before Searchlogic, I was just doing: Order.all(:order => “(quantity * price) ASC”). Is there a way to do this in Searchlogic?

Me: No, you shouldn’t order data like that. …long explanation of what to do…. Your idol, Ben

Some person: Thank you so much Ben, I can’t thank you enough for everything you have done for me. You truly are my idol. There should be a statue in your honor.

Maybe it didn’t go exactly like that, but you get the point. So instead of having to retype the “long explanation of what to do”, I can just link them to this post:

This can be simple or complicated depending on how you calculate your total. Let’s start with the simple:

Simple solution

Your total method looks something like to following. Dealing with only local attributes.

def total
    quantity * price
end

Your first thought might be to sort via the ruby sort method. This is wrong on so many levels, not to mention you already have a tool that is extremely good at sorting data: your database.

Awesome! We’ll just do something like this:

Order.all(:order => "(quantity * price) ASC")

Wrong again! This is bad for a number of reasons:

1. Your code is no longer DRY. What if you want to add in tax later on down the road? You’ll have to do it in multiple places.
2. (quantity * price) can not be indexed. The whole point of indexing is to sort.

Now you’re saying “WTF Ben?!?!?1!?11″. Relax, why not just cache the total in the database?

class Order < ActiveRecord::Base
    before_validation :cache_virtual_columns

    def total
        quantity * price
    end

    private
        def cache_virtual_columns
            self.total = total
        end
end

Now you can index the total column and order just like you would with any other column. Yay!

Don’t celebrate yet. Chances are your total probably isn’t that simple.

Advanced solution

Maybe your total method looks like:

def total
    line_items.sum(:total) + tax + shipping_cost
end

Oh no! What happens when the total of a line item changes? The total in the orders table is no longer accurate. Your total now depends on other models. This is where observers shine:

class OrderObserver < ActiveRecord::Observer
  observe :order, :line_item

  def before_validation(obj)
    case obj
    when Order
      obj.total = obj.total
    end
  end

  def after_save(obj)
    case obj
    when LineItem
      obj.order.update_attribute(:total, obj.order.total)
    end
  end
end

You could really set this up a number of ways. You could create a CacheObserver that handles all caching for all models. This could get messy and confusing really fast, which is why I recommend the above example. My philosophy on OO programming is that classes should be responsible for their self. If the Order class is calculating the total it should be responsible for keeping it up to date in the database too.

“But Ben, this looks messy and complicated”. I agree with you, to an extent, but there is no simple solution here, because there is no pattern. Your total method could be anything you want. For all I know, your total method could be outsourced to a team in India that calculates the total by hand.

Sometimes in programming you have to give up something to gain something. I feel like writing some simple code in an observer is worth having a cached value of the total in the database. It makes everything so much cleaner, easier, and simpler in your controllers and views. Models should do all of the heavy lifting anyways.

Obviously, this is not just limited to total. You could do this for pretty much any virtual attribute in your models.

What do you think? I am always interested in feedback.

Why modifiers are necessary?

ActiveRecord does a great job when it comes to keeping your code database agnostic. But I feel like it neglected searching when it came to that goal. What if you want to find all records that were created after 7am? Depending on your database you would have to do something like the following:

MySQL:        HOUR(created_at)
PostgreSQL:   date_part('hour', created_at)
SQLite:       strftime('%H', created_at)

All of a sudden your app is not database agnostic. Oh no!

All of that bragging to your girlfriend about how you can switch databases in a matter of minutes was all a lie. How’s she going to feel when she finds out HOUR(created_at) is a MySQL specific function? Searchlogic to the rescue!

Searchlogic modifiers

Remember how you can do this in searchlogic?

User.all(:condition => {:created_at_gt => Time.parse("Jan 1, 2008")})
# => "created_at > 2008-01-01 00:00:00

Now you can do this too:

User.all(:conditions => {:hour_of_created_at_gt => 10})
# => HOUR(created_at) > 10

or this:

User.all(:conditions => {:tan_of_sin_of_cos_of_minute_of_created_at_gt => 20})
# => TAN(SIN(COS(MINUTE(created_at)))) > 20

What the hell was that? That was me making the point that you can chain modifiers. Obviously a very unrealistic example, but cool none-the-less.

Check out the readme for a list of modifiers and check out the changelog for a complete list of changes

What do you think?

Maybe it’s not a pain in the ass for you, maybe it is, or maybe it is and you don’t know it. We’ll never know. Either way, this tutorial is pain-in-the-ass free. Guaranteed. That’s what this is all about, making your life easier when it comes to paginating, ordering, and searching. Things you do multiple times in almost every application.

My solution: Searchlogic. It has saved me a lot time, shortened my code, and ultimately given me the proper tools to paginate, order, and search my data. Hopefully it will do the same for you.

Enough talk, let’s dive in.

“Does this really work? Am I about to waste my time?”. I hope not. But just for you, I made this live example:

Live example based on this tutorial

Paginating, ordering, and searching doesn’t get any easier than this. Before we start I want to make the following assumptions about you:

1. You have a working rails application on rails edge (rake rails:freeze:edge)
2. You have the following model structure: UserGroup => User => Order (=> = has_many). Make sure you have the has_many and belongs_to relationships set up. The fields are irrelevant, just change the field names in the view to whatever you have.

1. Install Searchlogic

$ sudo gem install searchlogic

Now add the gem dependency to your rails config:

# config/environment.rb
config.gem "searchlogic"

As always you can install searchlogic as a plugin, but plugins are becoming a thing of the past. I recommend the above approach, but here is the plugin installation if you prefer to do it this way:

$ script/plugin install git://github.com/binarylogic/searchlogic.git

2. Create your controller

Let’s create a controller that searches, orders, and paginates users. Like an admin area. Create app/controllers/users_controller.rb with the following content:

# app/controllers/users_controller.rb
class UsersController < ApplicationController
    def index
        @search = User.new_search(params[:search])
        @users, @users_count = @search.all, @search.count
    end
end

What did we do here?

It’s simple actually, if you haven’t looked at Searchlogic I suggest you take a quick glance. I’m not going to get into crazy detail about what it does, because the README in the library covers that, but to make this short it “enhances” searching with ActiveRecord.

Notice that we started a new search with the @search object. This lets you search via an object, which is really handy for your view, which you will see below. It also added in some nifty methods such as page, per_page, order_by, order_as, and a plethora of conditions for each of your columns. The second line does a simple search and counts the results. The thing to remember here is that the .count method ignores pagination. It ignores the limit and offset values. So if you want to know how many users matched the search, use count, not @users.size. If the first page is limited to 10 reconds @user.size will return 10.

3. Create the view and your’re done

# app/views/users/index.html.erb
<% if @users_count > 0 %>
    <%= @users_count %> users found

    <table border="1" cellpadding="5">
        <tr>
            <th><%= order_by_link :id %></th>
            <th><%= order_by_link :user_group => :name %></th>
            <th><%= order_by_link :first_name %></th>
            <th><%= order_by_link :last_name %></th>
            <th><%= order_by_link [:email, :first_name] %></th>
        </tr>
        <% @users.each do |user| %>
            <tr>
                <td><%= user.id %></td>
                <td><%= user.user_group ? user.user_group.name : "-" %></td>
                <td><%= user.first_name %></td>
                <td><%= user.last_name %></td>
                <td><%= user.email %></td>
            </tr>
        <% end %>
    </table>

    <br />
    <br />

    Per page: <%= per_page_select %>

    <% if @search.page_count > 1 %>
        <br />Page: <%= page_select %>
    <% end %>
<% else %>
    No users were returned
<% end %>

What did we do here?

All that we did was iterate through our users, list them in a table format, and then add page and per_page controls. Notice any unusual helpers? (order_by_link, per_page_select, and page_select). I won’t go into detail about these helpers because its all in the documentation under Searchlogic::Helpers::ControlTypes.

These are only a few ways to use these helpers, this is really just the tip of the iceberg. The sky is the limit: create a select that lets users navigate through the pages, create a list of links (like flickr) that lets you navigate through pages, create a link that lets you order by any number of columns you want, etc. I know ruby has a bad rep when it comes to documentation, but I actually put some decent time into the Searchlogic documentation and I think you will find it helpful. That’s your best resource for finding out everything Searchlogic has to offer.

Adding a search form

“This is all great, but you said I could search my data”. Searching your data is just as easy. Just add this to the top of your index.html.erb

# app/views/users/index.html.erb
<% form_for @search do |f| %>
    <fieldset>
        <legend>Search Users</legend>

        <% f.fields_for @search.conditions do |users| %>
            <%= users.label :first_name_keywords %><br />
            <%= users.text_field :first_name_keywords %><br />
            <br />

            <% users.fields_for users.object.orders do |orders| %>
                <%= orders.label :total_gt, "Has orders with a total greater than" %><br />
                $<%= orders.text_field :total_gt %><br />
                <br />
            <% end %>

            <% users.fields_for users.object.user_group do |user_group| %>
                <%= user_group.label :name_starts_with, "Belongs to user group with name that starts with" %><br />
                <%= user_group.text_field :name_starts_with %><br />
                <br />
            <% end %>
        <% end %>
    </fieldset>
    <%= f.submit "Search" %>
<% end %>

As I mentioned above Searchlogic creates default conditions on your columns based on the type. Letting you use a form builder to call those conditions. When it receives these conditions on the back-end it will do its “magic” and creates the proper SQL. Don’t worry about SQL injections, Searchlogic has you covered on that (see the documentation for more info).

What’s great about this method of searching?

1. Your search logic is in one place: the view.
2. You can add conditions by doing f.text_field “[column name]_[condition]“. So when your picky client calls up and says “Hey, we need a search field for finding emails that end with…” No prob: f.text_field :email_ends_with. Done!
3. You can traverse your relationships with fields_for and set conditions on related objects. Saweet!

Ajaxified

So you’re saying “this is great, but super old school, where’s the AJAX?!?!?” So you’re all about AJAX? No problem. In the example I have 3 examples: a non AJAX example, an AJAX example using the built in rails helpers, and a jQuery AJAX example. Check them out. You can view the source of the examples on github (each example if named spaced into its own controller). On a side note, I highly recommend jQuery. I recently started using it and love it. I actually get excited to write javascript because I always find something new when digging through the documentation or plugins and unobtrusive javascript never felt so good.

Some helpful links

This tutorial is really just the tip of the ice berg with Searchlogic. Checkout these links to see all it has to offer:

• Live example based on this tutorial
• Source code for the live tutorial
• Searchlogic, the library that makes all of this magic happen
• Searchlogic documentation

I am always interested in hearing feedback. Let me know what you think, what you like, what you don’t, that you hate the name, etc. I love criticism. Searchlogic is under active development and I am always trying to improve it. I hope this tutorial was helpful to you and ultimately makes your life easier.

Enough said, let me show you some of my favorite features. If you are still interested you can just jump over to the searchlogic github page for a more detailed explanation.

For the following examples lets assume these relationships: User => Orders => LineItems

Simple Searching Example

User.all(
  :conditions => {
    :first_name_contains => "Ben",          # first_name like '%Ben%'
    :email_ends_with => "binarylogic.com"   # email like '%binarylogic.com'
  },
  :per_page => 20,                      # limit 20
  :page => 3,                           # offset 40, which starts us on page 3
  :order_as => "ASC",
  :order_by => {:user_group => :name}   # order user_groups.name ASC
)

same as above, but object based

search = User.new_search
search.conditions.first_name_contains = "Ben"
search.conditions.email_ends_with = "binarylogic.com"
search.per_page = 20
search.page = 3
search.order_as = "ASC"
search.order_by = {:user_group => :name}
search.all

In both examples, instead of using the “all” method you could use any search method: first, find(:all), find(:first), count, sum, average, etc, just like ActiveRecord.

The beauty of searchlogic, integration with rails

Using Searchlogic in rails is the best part, because rails has all kinds of nifty methods to make dealing with ActiveRecord objects quick and easy, especially with forms. So let’s take advantage of them! That’s the idea behind this plugin. Searchlogic is searching, ordering, and pagination all rolled into one simple plugin. Take all of that pagination and searching cruft out of your models and let Searchlogic handle it. Check it out:

# app/controllers/users_controller.rb
def index
    @search = User.new_search(params[:search])
    @users, @users_count = @search.all, @search.count
end

Now your view.

# app/views/users/index.html.haml
- form_for @search do |f|
 - f.fields_for @search.conditions do |users|
  = users.text_field :first_name_contains
  = users.calendar_date_select :created_after # nice rails plugin for replacing date_select
  - users.fields_for users.object.orders do |orders|
    = orders.select :total_gt, (1..100)
  = f.submit "Search"

- if @users_count > 0
  %table
    %tr
      %th= order_by_link :account => :name
      %th= order_by_link :first_name
      %th= order_by_link :last_name
      %th= order_by_link :email
    - @users.each do |user|
      %tr
        %td= user.first_name
        %td= user.last_name
        %td= user.email

  Per page:
  = per_page_select
  Page:
  = page_select
- else
  No users returned

Things to notice

1. Passing a search object right into form_for and fields_for
2. The built in conditions for each column and how you can traverse the relationships and set conditions on them
3. The order_by_link helper
4. The page_select and per_page_select helpers
5. All of your search logic is in 1 spot: your view. Nice and DRY.

See my tutorial on this example

Nifty methods for searching trees

Just like the conditions you automatically get with each column. You automatically get conditions for searching tree data structures, if you’re model is a tree data structure.

# User is a tree data structure.
conditions = User.new_conditions
conditions.child_of = User.roots.first                  # Finds all children
conditions.sibling_of = User.first                      # Finds siblings
conditions.descendant_of = User.roots.first             # Finds all children, grandchildren, etc
conditions.inclusive_descendant_of = User.roots.first   # Includes root + children, grandchildren, etc.

Any of these conditions can accept a User object or a User id.

Get Started

Check out the GitHub repository to get started:

http://github.com/binarylogic/searchlogic/

Or a tutorial I posted on using Searchlogic in rails:

http://www.binarylogic.com/2008/9/7/tutorial-pagination-ordering-and-searching-with-searchlogic