Building Cross Model Search with Ember, Rails, and Elasticsearch, Part II

Development

Reading Time: 13 minutes

In Part One of this two part series, we covered how to integrate Elasticsearch and Rails to query against multiple models. To illustrate this, we started a Rails app for an ecommerce store named Tember. It includes an Elasticsearch index made up of four different models using Toptal’s Chewy gem.

Today we’re going to pick up where we left off and build a cross model autocompleting search feature for the Tember store. We left off last time in the Rails console interacting with our new indexes, so the first thing we need to do is get our Rails app accepting HTTP requests and sending back data.

Let’s get started!

Serving Data from Rails and Elasticsearch Over HTTP

We are strictly building an API for this search endpoint, so let’s organize the controllers under the api/v1 namespace.

 $ rails g controller api/v1/searches
      create  app/controllers/api/v1/searches_controller.rb
      invoke  rspec
      create    spec/controllers/api/v1/searches_controller_spec.rb

I breathe easier when I’m confident in the test coverage, so let’s start start with some rspec specs for our search controller. Open up the searches_controller_spec.rb, and let’s add in some tests for the controller:

# spec/controllers/api/v1/searches_controller_spec.rb

require 'rails_helper'

RSpec.describe Api::V1::SearchesController, :type => :controller do

  describe 'GET #index - /v1/searches' do
    before do
      get :index, q: 'an example query string'
    end

    it { is_expected.to respond_with :ok }
    it { is_expected.to respond_with_content_type :json }
  end
end

Let’s run the tests and start moving from red to green:

$ rspec spec/controllers/api/v1/searches_controller_spec.rb

  1) Api::V1::SearchesController GET #index - /v1/searches
     Failure/Error: get :index, q: 'an example query string'
     ActionController::UrlGenerationError:
       No route matches {:action=>"index", :controller=>"api/v1/searches", :q=>"a example query string"}

As you probably noticed, we don’t have any routes defined. Let’s open up config/routes.rb and add the appropriate routes.

As I mentioned earlier, organization is crucial at any point in a project, but especially at the start of one. Organizing the routes specified for the API into an api and v1 namespace helps keep your project tidy, adds logical separation of responsibilities, and if you add constraints in the declaration, it can be used to add subdomains.

Subdomains allow you to load balance at the DNS level if necessary. Carlos Souza from Code School covers this material in depth in his Surviving APIs with Rails course.

# config/routes.rb

Rails.application.routes.draw do

  # use the constraints option to add a subdomain to the uri
  namespace :api, path: '' do # constraints: { subdomain: 'api' }
    namespace :v1 do
      resources :searches, only: [:index]
    end
  end
end

Let’s run the tests and see where we stand:

$ rspec spec/controllers/api/v1/searches_controller_spec.rb

  1) Api::V1::SearchesController GET #index - /v1/searches
     Failure/Error: get :index, q: 'an example query string'
     AbstractController::ActionNotFound:
       The action 'index' could not be found for Api::V1::SearchesController

We covered our No routes match error, but now we’re seeing another error complaining about the lack of an index action in our controller. Here’s how we fix this:

# controllers/api/v1/searches_controller.rb

class Api::V1::SearchesController < ApplicationController

  # v1/searches
  def index
    render json: []
  end

end

Now when we run rspec spec, it gives us a green light:

.

Finished in 0.01418 seconds (files took 2.24 seconds to load)
2 examples, 0 failures

Loading Elasticsearch Documents Inside Rails Controllers

This controller doesn’t gives us any results, so let’s add in some of the Chewy commands we used in the previous post to pull in data from Elasticsearch. If you recall, we had a class called StoreIndex which represented the integration of our Postgres stored data and our Elasticsearch documents.

Yes, I said “documents.” Elasticsearch is actually a NoSQL document oriented datastore. Let’s pull in the command we used last time and dissect it:

StoreIndex.query(term: {name: 'oak'})

This particular command is looking at the index as a whole and asking for any document where the name field matches oak. If we were simply interested in the name field, this command would be sufficient. Unfortunately, we’re interested in more fields and want our search to query against names, descriptions, and even the review bodies. Let’s alter the way we interface with our StoreIndex:

StoreIndex.query(query_string: {fields: [:name, :description, :body], query: ‘oak’, default_operator: 'or'})

This command gives us more control over the fields we are querying against. You also might notice the default_operator: ‘or’. This makes the search query inclusive. Other filter mode options include: :and, :or, :must, :should.

If we open up the console to test this, you’ll see additional classes are loaded by the query:

$ rails c

2.2.2 :005 > StoreIndex.query(query_string: {fields: [:name, :description, :body], query: 'oak', default_operator: 'or'}).load.to_a

StoreIndex Search (38.9ms) {:body=>{:query=>{:query_string=>{:fields=>[:name, :description, :body], :query=>"oak", :default_operator=>"or"}}}, :index=>["store"], :type=>[]}
  
Product Load (0.4ms)  SELECT "products".* FROM "products" WHERE "products"."id" IN (1, 2, 3)
Review Load (28.5ms)  SELECT "reviews".* FROM "reviews" WHERE "reviews"."id" IN (2, 1, 3)
Vendor Load (0.3ms)  SELECT "vendors".* FROM "vendors" WHERE "vendors"."id" IN (1, 3)

Let’s adjust the SearchesController to include our new query commands:

class Api::V1::SearchesController < ApplicationController

  # v1/searches
  def index
    render json: StoreIndex.query(query_string: {fields: [:name, :description, :body], query: params[:q], default_operator: 'or'}).load.to_a, each_serializer: SearchesSerializer
  end

end

We are querying across many models, so it’s important to have solid test coverage on the components of this feature. Let’s open up our controller spec and start adding more cases:

require 'rails_helper'

RSpec.describe Api::V1::SearchesController, :type => :controller do

  let(:vendor) { Vendor.new(id: 1, name: 'Example Vendor Name', description: 'Example vendor description.') }
  let(:product) { Product.new(id: 1, name: 'Example Product Name', description: 'Example product description.') }
  let(:json) { JSON.parse(response.body) }

  it { should route(:get, 'v1/searches').to(action: :index) }

  describe 'GET #index - /v1/searches' do
    before do
      StoreIndex.stub_chain(:query, :load, :to_a).and_return([vendor, product])
      get :index, q: 'oak'
    end

    it { is_expected.to respond_with :ok }

    it 'returns array of records' do
      expect(json["searches"]).to be_a Array
    end

    it 'should have `searches` as a root key' do
      expect(json).to have_key("searches")
    end
  end
end

We have added quite a few things, so let’s take a moment to review.

First, we added a route test in the controller test. Second, we’re stubbing the StoreIndex to return an empty array.

We stub our the index because this test should not be responsible for testing the indexes. Instead, it should be responsible for testing what happens with the return value. We’ll refactor this test later to add models in the return value.

Finally, we added in tests to ensure the root key is searches and the value of it is an array. Adjusting the controller to include a root key option with the value of “searches” makes this test pass.

class Api::V1::SearchesController < ApplicationController

  # v1/searches
  def index
    render json: StoreIndex.query(query_string: {fields: [:name, :description, :body], query: params[:q], default_operator: 'or'}).load.to_a, each_serializer: SearchesSerializer, root: "searches"
  end

end

In order for the search API to interface with Ember, we need to normalize the results we get back. This is easier explained by examining sample data responses.

If we have four different models — Vendor, ReviewAuthor, Review, Product — we want the results to show the same data for each one like so:

{
   "searches":[
      {
         "id":1,
         "type":"Product",
         "title":"Some product title",
         "description":"Some product description"
      },
      {
         "id":1,
         "type":"Vendor",
         "title":"Some product title",
         "description":"Some vendor description"
      },
      {
         "id":1,
         "type":"Review",
         "title":"Some review title",
         "description":"Some review description"
      },
      {
         "id":1,
         "type":"ReviewAuthor",
         "title":"Some author name",
         "description":"Some author bio"
      }
   ]
}

Each of the models have different attributes, but we map them to display fields for UI purposes. In order to render JSON, we will need a serializer. We’ll call it SearchesSerializer.

$ rails g serializer searches
create app/serializers/searches_serializer.rb

Now that we have a serializer, let’s add in more tests to our controller spec:

it 'should respond with one product' do
expect(json["searches"].map(&["title"])).to include(product.name)
end

it 'should respond with one vendor' do
expect(json["searches"].map(&:["title"])).to include(product.name)
end

We want to ensure that the results have title attributes which include our sample data. In order to do this, we’ll create aliases on each model mapping title and description. For example, the vendor model now looks like this:

class Vendor < ActiveRecord::Base

  # es indexes
  update_index('store#vendor') { self } # specifying index, type and back-reference
  # for updating after user save or destroy
  # associations
  has_many :products

  alias_attribute :title, :name

end

After we have aliased the appropriate methods on each model, we can use their respective attributes in the serializer like so:

class SearchesSerializer < ActiveModel::Serializer
  attributes :id, :title, :description, :type

  def id
    object.class.to_s.downcase + '|' + object.id.to_s
  end

  def type
    object.class.to_s.camelize
  end

end

I want you to notice two things. First, we have added in the type string. This will be used for display purposes on the front end.

Second, we’ve overwritten the ID attribute to increase the uniqueness of the value. We do this because in ember-data, two records with the same ID will cause issues with the local caching. A simple CURL command will test out new searches endpoint for our Tember store.

$ curl -X GET 'http://localhost:3000/v1/searches?q=oak'

{"searches":[{"id":"vendor|1","title":"Oakmont Oaks","description":"Oakmont provides choice oak from Orlando, Louisville, and Seattle","type":"Vendor"},{"id":"vendor|3","title":"Big Hill","description":"Big Hill provides oak, northern ash, and pine from various regions of Canada.","type":"Vendor"}]}

Our data is looking great! Now, it’s time to hop into the front-end of this ecommerce store and start coding up some Ember.js goodness.

Sign up for a free Codeship Account

Building an Autocomplete Search in Ember

Ember is a front-end framework for building ambitious web applications. I started with Ember almost exactly one year ago today and have continually been impressed by the community, the framework’s maturity, and the tooling Ember is shipping with (ember-cli).

Without further delay, let’s dive in! We’ll start by creating a new Ember app with the cli; if you have not used ember-cli before, check it out!

ember new tember-ember

If we cd into tember-ember, we can start setting up our Ember project. I’m going to start by editing the bower.json to ensure we’re on the correct Ember and ember-data versions.

At the time of this article, Ember is on v2.2.0 and ember-data is on v2.2.1. Let’s boot up the app and see what we get:

WelcomeToEmber

A bit boring if you ask me. Let’s clean it up a bit.

Since we’ve been riding the edge with Rails 5, we shouldn’t stop there! Let’s install Bootstrap 4 and give ourselves a fancier UI:

bower install bootstrap#v4.0.0-alpha.2

With some standard and some new bootstrap commands dropped into templates/application.hbs, we’re beginning to look a bit better:

<nav class="navbar navbar-full navbar-dark bg-danger">
  <a class="navbar-brand" href="#">
    Tember,
    <small>a store for high quality wood workers.</small>
  </a>
</nav>
{{outlet}}

TemberApp

Creating a service-fed component in Ember 2.x style

At this point, we’re ready to start searching our store and displaying our results. To do this, we’ll create a store-search component.

Ember has moved away from the traditional MVC in favor of a component-driven model. Components make it much easier to isolate pieces of UI, making them easier to understand, test, and manage.

If you’re familiar with the 1.x methodology, you might be asking yourself, “How do components get their data without a controller?” Valid question. In the past, Ember has used the Route to load data which could then be sent into a component through the controller. Routes are still a thing in 2.x, except the new norm will be routable components and components fed via services. We’ll touch on this shortly.

In the meantime, let’s generate our component:

$ ember g component store-search

installing component
  create app/components/store-search.js
  create app/templates/components/store-search.hbs
installing component-test
  create tests/integration/components/store-search-test.js

Now, make the component look like this:

templates/components/store-search.hbs

{{input name="query" value=query class="form-control" placeholder="Search for anything..."}}

{{#if search.noResults}}
  <li class='no-results list-unstyled'>
    No results for "<span>{{query}}</span>".
  </li>
{{/if}}

<div class='row'>
  {{#each search.results as |result|}}
    <div class='col-xs-6 col-sm-4'>
      <a href="{{result.link}}">
      <div class="card">
        <div class="card-header">
          {{result.type}}
        </div>
        <div class="card-block">
          <h4 class="card-title">{{result.title}}</h4>
          <p class="card-text">{{result.description}}</p>
          <p class="card-text"><small class="text-muted">{{result.lastUpdated}}</small></p>
        </div>
      </div>
      </a>
    </div>
  {{/each}}
</div>

You might be noticing the familiar attributes we created in our Rails serializers; they will be injected into this search component.

The component will be responsible for telling the user if there are no results, and if there are results, the user will begin to see them as Bootstrap 4 cards. The card component of Bootstrap 4 replaces the panel/well of previous Bootstrap versions.

The data being pushed into the component will be from the search model. It will also have a link property which will be used to link to the resource. Let’s create that now:

// app/models/search.js
import DS from 'ember-data';

export
default DS.Model.extend({
  type: DS.attr('string'),
  description: DS.attr('string'),
  title: DS.attr('string'),

  link: Ember.computed('type', function() {
    switch (this.get('type')) {
      case 'Product':
        return `product/${this.get('id')}`;
      case 'Vendor':
        return `vendor/${this.get('id')}`;
      case 'ReviewAuthor':
        return `review-author/${this.get('id')}`;
      case 'Review':
        return `review/${this.get('id')}`;
    }
  })
});

As I mentioned above, there are more ways than routes to load data. Another strategy to bring data into Ember is through services. Services are essentially a singleton object in Ember with a slick API to inject them anywhere you need — components, in our case.

Let’s create an Ember service to bring data into this search component.

ember g service search

installing service
  create app/services/search.js
installing service-test
  create tests/unit/services/search-test.js

As mentioned previously, we want the search component to let us know if there are zero results for a given query. In order to do this, we need to cache the current search query and results.

We’ll handle all of this functionality inside the service by simply updating the searchValue attribute. Inside the service, we’ll add a few methods:

// app/services/search.js
import Ember from 'ember';

export
default Ember.Service.extend({
  store: Ember.inject.service(),

  searchValue: null,
  searchResults: [],
  resultsEmpty: false,

  noResults: Ember.computed('resultsEmpty', 'searchValue', function() {
    // ensure there is a search query and the results
    // are empty to prevent "No results for ''".
    if (this.get('resultsEmpty') && Ember.isPresent(this.get('searchValue')) && this.get('searchValue').length > 0) {
      return true;
    } else {
      return false;
    }
  }),

  results: Ember.computed('searchResults.[]', function() {
    return this.get('searchResults');
  }),

  _fetchSearchResults: Ember.observer('searchValue', function() {
    // exit without making a reqeust if value cast is empty
    if (Ember.isBlank(this.get('searchValue'))) {
      return [];
    }

    this.get('store').query('search', {
      q: this.get('searchValue')
    }).then((results) => {
      // check if the query results are empty
      if (results.get('length') > 0) {
        this.set('resultsEmpty', false);
      } else {
        this.set('resultsEmpty', true);
      }
      this.set('searchResults', results);
    });
  })
});

The noResults method is responsible for letting the component know a search has occurred, but it returned no results. The other methods are self explanatory: One fetches results from Rails, and the other is a computed property returning the results.

As you can see, we rely on the service to do almost all the work. The component just needs to be aware of how to interact with the service:

// app/components/store-search.js
import Ember from 'ember';

export
default Ember.Component.extend({

  search: Ember.inject.service(),

  query: null,

  _valueChanged: Ember.observer('query', function() {
    // use the run loop to add a debounce
    Ember.run.debounce(this, function() {
      // check if the query is at least 2 chars
      if (this.get('query').length > 2) {
        this.set('search.searchValue', this.get('query'));
      } else if (this.get('query').length === 0) {
        this.set('search.searchResults', []);
      }
    }, 200);
  }),

});

This component starts off by injecting the search-service and setting the query attribute to null.

Lastly, we set an observer to monitor the query value. It interfaces with the search-service every time it’s updated by changing the service’s current searchValue updating the results.

Now that we have the service and component wired up, let’s check it out:

Tember

And there you have it! An autocompleting cross model search component built with Ember.js, Rails 5, and Elasticsearch.

Conclusion

I hope this post has helped you become more confident in your ability to integrate Rails and Elasticsearch to create a scalable search solution for any use case. There’s certainly much to be desired when it comes to the accuracy of the search, but every use case needs to be dialed in with the appropriate Elasticsearch techniques.

Don’t forget to check out the Ember and Rails repos:

Happy coding!

Subscribe via Email

Over 60,000 people from companies like Netflix, Apple, Spotify and O'Reilly are reading our articles.
Subscribe to receive a weekly newsletter with articles around Continuous Integration, Docker, and software development best practices.



We promise that we won't spam you. You can unsubscribe any time.

Join the Discussion

Leave us some comments on what you think about this topic or if you like to add something.

  • The chewy gem has an issue in its latest version (0.8.3) with Rails 5 applications in API-only mode, which breaks the middleware chain because assets configuration isn’t available for those apps. There’s a PR out which fixes the issue and hopefully gets merged in one of the next releases: https://github.com/toptal/chewy/pull/303