An Introduction to APIs with Phoenix

Development

Reading Time: 10 minutes

Phoenix is taking the internet by storm, with good reason. It’s productive, fault-tolerant, concurrent, safe as a compiled language, and blazing fast. It shares many of Rails’ core values, such as convention over configuration, restful resources, and a focus on developer happiness.

The cherry on the top: Phoenix was designed from the ground up for WebSockets connections — so you know it’s ready for today’s web.

Today, we will build a restful API with Phoenix and compare it to a Ruby on Rails API. Before we get our hands dirty, however, let’s make a case for learning Phoenix. Why should you switch?

Performance

If you’re looking for performance, prepare to be delighted. Phoenix is built on top of Elixir, which uses BEAM (the Erlang virtual machine). Elixir compiles directly to BEAM byte code. Because of this, it can take advantage of the 30 years of stability and performance optimizations that Erlang offers.

Elixir is by no means a fast language. If you want to crunch numbers or manipulate images, then you should probably pick Rust, Go, or C over Elixir. However, Elixir is extremely performant when handling thousands or even millions of connected devices. This is why Phoenix responds to calls in a few milliseconds, if not microseconds.

You might say that performance is not a good reason to switch. After all, if you use Rails, that means that fast time to market is more important to you than performance. That’s what Rails is best at: getting out of your way as you focus on building software.

The good news is that Phoenix is great at that as well. So you don’t have to pick either performance or productivity: Phoenix delivers on both fronts.

Fault Tolerance and Concurrency

In traditional languages (imperative, procedural, object-oriented), the construct for parallel or concurrent execution of code is the thread.

Web servers or frameworks typically have some type of thread pool, and when a request comes in, the server/framework decides which thread should handle the request. This can be problematic for multiple reasons, memory management and error recovery being the two main problems.

Heroku restarts its dynos daily to prevent memory from skyrocketing. Recently, memory leaks have been found in mainstream open-source projects like Sidekiq. This is because thread management is difficult. And memory management doesn’t even deal with problems associated with a thread that has a segfault and brings down the whole server.

These problems simply don’t exist in Elixir. The construct used for concurrent execution is the process.

Processes are independent applications that have their own garbage collector. They are completely isolated. If a process has a catastrophic failure, only that process is affected, and the application continues to work normally.

This is exactly how Phoenix servers operate. When a request comes in, it isn’t being shared with a thread pool. Instead, each request uses its own independent web server. If that process (web server) dies, the application is unaffected. And when that request is finished, that process garbage-collects itself.

Productivity

Everything so far makes for a rock-solid web framework, but it still says nothing about a developer’s ability to ship products.

Rails has been the framework that stands apart in this aspect. Phoenix was written by rock-solid Rails developers. It takes all the good lessons that 10 years of Rails brought but leaves behind the things that Rails can’t/won’t let go of.

The easiest way to explain this is to look at some code and talk about the differences. The following assumes the reader has Elixir (1.2.0), Phoenix (1.1.2), and PostgreSQL installed. The Phoenix installation guides can be found here.

Create an App

One of the first problems a young Padawan Rails developer faces is knowing when to type rails and when to type rake. And when teaching young developers, that’s surprisingly difficult to explain.

The rails command is used to run generators, start the server, create new apps, and so on, but the rake command is used to migrate the database, run the tests, etc.

There is no rhyme or reason to it. A developer will type rails generate migration to create the migration but rake db:migrate to make the migration happen. It’s confusing. Thankfully much of this will be fixed in Rails 5; most commands will use rails, so look for that in a few months.

The Elixir language has a tool called mix. The mix command is always used. If you ever want to know what commands are available, you just type mix help.

In a command prompt, grep for “phoenix”:

$ mix help | grep -i phoenix
mix phoenix.new       # Create a new Phoenix v1.1.2 application

Now use mix to create a new Phoenix API:

$ mix phoenix.new todo_api
...
Fetch and install dependencies? [Yn] y
* running mix deps.get

We are all set! Run your Phoenix application:

    $ cd todo_api
    $ mix phoenix.server

You can also run your app inside IEx (Interactive Elixir) as:

    $ iex -S mix phoenix.server

Before moving on, configure your database in config/dev.exs and run:

    $ mix eco.create

The option --no-brunch does not install brunch.io, which is the build tool used to build assets; it requires Node.js. And the --no-html removes code involved in creating traditional websites.

If you don’t use those options, a hybrid app is generated: an app that can have API routes and traditional HTML routes. There is no performance penalty for this. The only reason I skipped those options is because they are outside the scope of this blog post.

The much anticipated Rails 5 --api option is an all-or-nothing approach. If you need HTML pages later, you cannot add them. This makes it almost unusable. Most APIs eventually have some type of traditional HTML pages for administrative use.

The instructions from the last command are nice. They tell the user how to run the server and create the database using Ecto. Ecto is an ORM much like Active Record or LINQ.

Following the instruction given by the phoenix.new command:

$ cd todo_list

Modify the config/dev.exs and config/test.exs files to have the proper credentials. On a Mac where PostgreSQL was installed with Homebrew, just remove the username and password.

config :todo_api, TodoApi.Repo,
  adapter: Ecto.Adapters.Postgres,
  # username: "postgres", # remove
  # password: "postgres", # remove
  database: "todo_api_dev",
  hostname: "localhost",
  pool_size: 10

Create the database:

$ mix ecto.create

Run the tests:

$ mix test
...

Finished in 0.1 seconds (0.1s on load, 0.00s on tests)
3 tests, 0 failures

Mix compiled the app for the first time. Don’t worry, mix only recompiles when files change and then only the files that changed. Type mix test again. It’s instantaneous!

Three tests for an app that does nothing. What are they?

# /test/views/error_view_test.exs
defmodule TodoApi.ErrorViewTest do
  use TodoApi.ConnCase, async: true

  # Bring render/3 and render_to_string/3 for testing custom views
  import Phoenix.View

  test "renders 404.json" do
    assert render(TodoApi.ErrorView, "404.json", []) ==
           %{errors: %{detail: "Page not found"}}
  end

  test "render 500.json" do
    assert render(TodoApi.ErrorView, "500.json", []) ==
           %{errors: %{detail: "Server internal error"}}
  end

  test "render any other" do
    assert render(TodoApi.ErrorView, "505.json", []) ==
           %{errors: %{detail: "Server internal error"}}
  end
end

These tests assert that out-of-the-box errors return proper JSON. But more importantly they test views, one of the core differences in Phoenix. A view is a group of functions used to render JSON, or if this was an HTML view, these functions would be available in the template.

Open the view and notice that it’s just a module with functions:

# /web/views/error_view.ex
defmodule TodoApi.ErrorView do
  use TodoApi.Web, :view

  def render("404.json", _assigns) do
    %{errors: %{detail: "Page not found"}}
  end

  def render("500.json", _assigns) do
    %{errors: %{detail: "Server internal error"}}
  end

  # In case no render clause matches or no
  # template is found, let's render it as 500
  def template_not_found(_template, assigns) do
    render "500.json", assigns
  end
end

The important takeaway for a JSON API is this: Views create dictionaries that represent the JSON for an action. And views are easy to test. This comes in handy if views have complex logic.

However, most of the time, a controller test that asserts the proper response is good enough. Those types of tests are considered integration tests.

Sign up for a free Codeship Account

First Resource

Frameworks aren’t really mature until they have generators. Luckily Phoenix has plenty. Listing them is easy:

$ mix help | grep -i phoenix
mix phoenix.digest      # Digests and compress static files
mix phoenix.gen.channel # Generates a Phoenix channel
mix phoenix.gen.html    # Generates controller, model and views for an HTML based resource
mix phoenix.gen.json    # Generates a controller and model for a JSON based resource
mix phoenix.gen.model   # Generates an Ecto model
mix phoenix.gen.secret  # Generates a secret
mix phoenix.new         # Create a new Phoenix v1.1.2 application
mix phoenix.routes      # Prints all routes
mix phoenix.server      # Starts applications and their servers

Use the phoenix.new.json generator to scaffold out a JSON resource:

$ mix phoenix.gen.json Todo todos description:string 
complete:boolean
* creating web/controllers/todo_controller.ex
* creating web/views/todo_view.ex
* creating test/controllers/todo_controller_test.exs
* creating web/views/changeset_view.ex
* creating priv/repo/migrations/20160120010759_create_todo.exs
* creating web/models/todo.ex
* creating test/models/todo_test.exs

Add the resource to your api scope in web/router.ex:

    resources "/todos", TodoController, except: [:new, :edit]

Remember to update your repository by running migrations:

    $ mix ecto.migrate

The generator says to add the a line to the router. Add it to the /api scope:

defmodule TodoApi.Router do
  use TodoApi.Web, :router

  pipeline :api do
    plug :accepts, ["json"]
  end

  scope "/api", TodoApi do
    pipe_through :api

    resources "/todos", TodoController, except: [:new, :edit]
  end
end

Routes initially feel very familiar to a seasoned Rails developer:

# Rails
resources :todos, except: [:new, :edit]
# Phoenix
resources "/todos", TodoController, except: [:new, :edit]

They can be written very similar to Rails, with a few exceptions. The controller is specified in the route. There are no plurals. Anyone who has ever lost several hours tracking down a bug, only find out they didn’t pluralize correctly, should be ecstatic about this.

Lastly, routes have pipelines, which are composable middleware stacks. They are composed of plugs which are middleware similar to Rack but with several advantages.

Run the migration and then run the tests:

$ mix eco.migrate
// stuff happens ...
$ mix test
.............

Finished in 0.2 seconds (0.1s on load, 0.07s on tests)
13 tests, 0 failures

Awesome, another passing test suite for free. For now, ignore the tests. Instead focus on the similarities and differences in the Phoenix framework. Open the model first:

# web/models/todo.ex
defmodule TodoApi.Todo do
  use TodoApi.Web, :model

  schema "todos" do
    field :description, :string
    field :complete, :boolean, default: false

    timestamps
  end

  @required_fields ~w(description complete)
  @optional_fields ~w()

  @doc """
  Creates a changeset based on the `model` and `params`.

  If no params are provided, an invalid changeset is returned
  with no validation performed.
  """
  def changeset(model, params \\ :empty) do
    model
    |> cast(params, @required_fields, @optional_fields)
  end
end

This is another major difference from Rails. In Rails, the mantra is “fat model, skinny controller.” This is not the case with Phoenix.

Models are a data type and are treated as such. They are just vanilla structs that use the special schema macro to map to the database. There is one exception to this. Functions called “changesets” are used to scrub params and validate data. In the above example, the cast\3 method is being used to scrub the params.

Now open the view:

# web/views/todo_view.ex
defmodule TodoApi.TodoView do
  use TodoApi.Web, :view

  def render("index.json", %{todos: todos}) do
    %{data: render_many(todos, TodoApi.TodoView, "todo.json")}
  end

  def render("show.json", %{todo: todo}) do
    %{data: render_one(todo, TodoApi.TodoView, "todo.json")}
  end

  def render("todo.json", %{todo: todo}) do
    %{id: todo.id,
      description: todo.description,
      complete: todo.complete}
  end
end

Exactly what was expected. A set of functions that’s used to generate JSON. This is great — it will be easy to modify this code later.

Now open the controller:

# web/controllers/todo_controller.ex
defmodule TodoApi.TodoController do
  use TodoApi.Web, :controller

  alias TodoApi.Todo

  plug :scrub_params, "todo" when action in [:create, :update]

  def index(conn, _params) do
    todos = Repo.all(Todo)
    render(conn, "index.json", todos: todos)
  end

  def create(conn, %{"todo" => todo_params}) do
    changeset = Todo.changeset(%Todo{}, todo_params)

    case Repo.insert(changeset) do
      {:ok, todo} ->
        conn
        |> put_status(:created)
        |> put_resp_header("location", todo_path(conn, :show, todo))
        |> render("show.json", todo: todo)
      {:error, changeset} ->
        conn
        |> put_status(:unprocessable_entity)
        |> render(TodoApi.ChangesetView, "error.json", changeset: changeset)
    end
  end

  def show(conn, %{"id" => id}) do
    todo = Repo.get!(Todo, id)
    render(conn, "show.json", todo: todo)
  end

  def update(conn, %{"id" => id, "todo" => todo_params}) do
    todo = Repo.get!(Todo, id)
    changeset = Todo.changeset(todo, todo_params)

    case Repo.update(changeset) do
      {:ok, todo} ->
        render(conn, "show.json", todo: todo)
      {:error, changeset} ->
        conn
        |> put_status(:unprocessable_entity)
        |> render(TodoApi.ChangesetView, "error.json", changeset: changeset)
    end
  end

  def delete(conn, %{"id" => id}) do
    todo = Repo.get!(Todo, id)

    # Here we use delete! (with a bang) because we expect
    # it to always work (and if it does not, it will raise).
    Repo.delete!(todo)

    send_resp(conn, :no_content, "")
  end
end

The syntax is different from Ruby, but this is the exact same logic one would expect to find in a Rails controller.

One difference is that the create and update actions return the changeset when the request is invalid, instead of the model. Given that models are just data objects that map to the database, this makes sense. A different data type, the changeset, is used to carry information about valid and invalid data.

Conclusion

If you’ve been dreaming of a framework that is blazing fast, fault-tolerant, compiled, and concurrent, but you’ve always come back to Rails because of its productivity… dream no more. The next blog post in this series will tackle CORS, authentication from scratch, and dependency management. Until then, enjoy your new-found productivity and performance!

Subscribe via Email

Over 60,000 people from companies like Netflix, Apple, Spotify and O'Reilly are reading our articles.
Subscribe to receive a weekly newsletter with articles around Continuous Integration, Docker, and software development best practices.



We promise that we won't spam you. You can unsubscribe any time.

Join the Discussion

Leave us some comments on what you think about this topic or if you like to add something.

  • Pingback: 19-02-2016 - Freaking friday link pack! - Magnus Udbjørg()

  • Roman Trofimov

    Very interesting. Waiting for the next blog post.

  • Pingback: Links for Week 7 of 2016 |()

  • Mauricio Serna

    Excellent. Thanks

  • collegeimprovements me

    Same here. This is awesome. What are some good learning resources about phoenix ? When will part2 come ?

  • arpieb

    “If a process has a catastrophic failure, only that process is affected, and the application continues to work normally.”

    This isn’t *entirely* correct… If a child process is linked to its parent, and no exception handling has been added to the parent, it *can* bring down the house:

    http://elixir-lang.org/getting-started/processes.html#links

    That being said, the proper use of linked processes is within an OTP supervisor where the crash is handled and you don’t sweat it. Phoenix utilizes this mechanism so it’s baked in.

  • hubertlepicki

    Codeship, this is great. You know what would be even better? A default pipeline to build Elixir Phoenix apps on your CI :D

  • samof76

    Awesome. But I presume you mix command here…

    mix phoenix.new todo_api

    should read…

    mix phoenix.new todo_api –no-html –no-brunch

    • Micah Woods

      Good catch, thanks!

  • PhoenixFF

    Ecto is not an ORM.

  • Thanks for the article.

    Minor typo:


    cd todo_list

    istead of


    cd todo_api

  • Vladimir Kravchenko

    Cool set of articles, thanks!
    One more typo:
    > mix eco.migrate

  • Pingback: What makes Phoenix a good catch? - Software consultancy firm()

  • Zack

    Yeah there’s a lot of typos in here. Don’t take commands he wrote as what you should be typing, look at the command line outputs and also use some common sense.

  • how do i create custom fucntion in controller and club those with api
    def filterbackend(conn, %{“user” => user_params}) do

  • I am currently trying to grasp what would be the best way to write API-only services in Elixir. We use Ruby+Grape for quite few of our current services. So https://github.com/elixir-maru/maru seems to be perfect match. But I was not really able to find enough information what the best practice is.

    Do you have any experience with Maru, would you stick to Phoenix rather?

    BTW you hint that in Phoenix, controllers are fat (opposite to fat models in Rails). It seems that Chris McCord is saying, that you should put logic to separate modules – https://youtu.be/tMO28ar0lW8?t=5m34s

  • Watwatwat

    How is using “mix phoenix.x” and “mix ecto.x” any different from a confusion stand point as using rails and rake? Adding a word to the front doesn’t make it less confusing.

  • You have a typo near the beginning when installing the database. Should be ecto.create. You left off the t in ecto.

  • There is an error: mix eco.create should be mix ecto.create

  • Typo fix: `mix eco.create` -> `mix ecto.create`