Heroku and Puma vs. Heroku and Unicorn

Most PopularTech Room

UPDATE: We’ve added performance comparisons done by Jonathan Rochkind to the additional links. He had Puma come out on top as well.

UPDATE 2: Clint looked into using preload for the Puma workers and reported his findings in the comments. Take a look, but the gist is preload is not necessary it seems

18 months ago I wrote a blog post about how to use Unicorn to optimize our Heroku performance. Since then we’ve been using Unicorn on Heroku.

Over the last couple of months our business grew a lot and Unicorn seemed to take more resources than necessary. We switched to 2X instances and still needed quite a lot of workers.

Although larger Heroku bills were part of our decision to optimize, we mostly felt the quality of our service was diminishing.

We started with Puma as it seemed to be one of the more widely used options. For the last month we have used Puma in production on the Codeship and are happy with it. There are a couple of potential pitfalls, but overall it made our performance better and Heroku bills smaller.

We are still using MRI 2.0.0-p195 on Heroku and haven’t switched to either JRuby or Rubinius. Although there is a lot performance we could gain from this switch we are happy with the performance we have and it would make our development more complicated.

We optimized different parts of our application at the same time, so we don’t have scientifically valid data for our change from Unicorn to Puma. For example we switched to a Database with more memory on Heroku and changed our auto-reload functionality. All of this combined resulted in a much better performance, but it can’t be attributed to Puma alone.

Setting Up Puma on Heroku

Getting Puma to run on Heroku is very easy. You should be able to get everything up and running in minutes.

Codeship – A hosted Continuous Deployment platform for web applications

Add Puma to Gemfile

First add Puma to your Gemfile and remove any other servers.

Add Puma To Procfile

Add the following line to your Procfile:

This allows you to set the number of threads and workers Puma uses through the environment.

It also uses bash syntax to define defaults if no environment values are set. Starting several Puma workers allows you to get even more out of your Heroku Dyno.

Setting the values through the Environment helped make it easy to figure out the numbers of workers and threads that work fine. This will be different for every application and you should experiment with it. Currently we use 10 Threads and 2 Workers.

Set DB connections through Environment

As described in the Heroku documentation set up a database_config initializer which lets you define your pool size and reaper frequency

Update Thanks to Zachariah Clay for bringing up a bug in the comments and sending an updated Gist for setting the database connection. The following Gist is still the old one, so please take a look at his as well.

The ActiveRecord reaper will regularly check your open connections and remove them if they become stale. You can read more about it in the Heroku Docs

Monitor with NewRelic

We use NewRelic for all of our monitoring. Their Request Time and Instance views helped is to see the improvements and make sure our memory consumption isn’t too much.

Lessons Learned

You should set the MIN_THREADS and MAX_THREADS environment variables to the same value. As your dyno doesn’t need to release the resources anyway it’s perfectly fine to set it to a static value.

Make sure your MAX_WORKERS is lower than the DB_POOL_SIZE, otherwise your application might not be able to connect to the database and throw exceptions.

Pitfalls

After a new deployment the memory climbs until it plateaus at currently ~250MB on Average. Other people have mentioned that there might be memory leaks with Puma, but it is probably somewhere in our codebase. We are currently looking into this, but it hasn’t been a problem so far. Just something to keep in mind.

Our assumption is that Unicorn hid this problem before by restarting the workers regularly.

Conclusions

So far Puma is running great and we run way less dynos. This might be due to a number of optimizations, but Puma was definitely an important part in that.

It is very easy to get started and if you feel you could gain better performance definitely give it a try.

As passenger now supports Heroku we might look into it as well sometime in the future. Right now we are happy with Puma though.

Tell us your experience with Puma or other performance optimizations on Heroku in the comments.

Links to get started started

Subscribe via Email

Be sure to join 13,643 subscribers of our newsletter to receive updates on software development best practices, Continuous Delivery and tips and tricks to start shipping your product faster.

Join the Discussion

Leave us some comments on what you think about this topic or if you like to add something.

  • Clint

    If you’re using Clustered mode, you likely want to use `preload!` so your workers are already loaded by the time they get their first request. You should then treat the database config things like Unicorn, and setup the connection in Puma’s `on_worker_boot` block in a config file and not an initializer. Reasons being that the initializer is only ran once (the master process), and then forks workers. in the setup you show you just have the one pool, and ideally you’d want each worker to have it’s own database connection pool to share with all it’s threads.

    • https://www.codeship.io/ Florian Motlik

      Thank you. Great if it’s helpful.

      We are using Herokus preboot, so we never really had the problem that preboot helps with before, but definitely important to include this. Do you have an example or a gist for this? Happy to include it in the blogpost, just to make sure I get it right.

      • Clint

        An example config/puma.rb file I use:

        https://gist.github.com/catsby/7025955

        I did some testing this morning and now doubt how necessary it is. The impetus was ensuring each worker has it’s own connection pool so that when load increases, the threads don’t compete for the limited AR connection pool.

        Originally it seemed that with clustered mode the initializer was only run once, but further experiments this morning showed me that if you don *not* preload, then each worker will run the initializer.

        If you *do* preload the app, then the initializer is ran only once, and this on_worker_boot block is a good idea. At this point through, I’m not convinced preload is necessary :/

        • https://www.codeship.io/ Florian Motlik

          That’s great, thanks for looking into this

        • http://jjb.cc/ John Bachir

          I believe –preload is necessary in order to take advantage of copy-on-write, introduced in ruby 2.0. This might signficantly reduce memory consumption in some cases.

          • Clint

            I saw your GitHub issue regarding this and don’t agree. I’ll follow up there… https://github.com/puma/puma/pull/387

          • https://www.codeship.io/ Florian Motlik

            We might just add this to our system and see if it changes the memory consumption. Seems like a change that can’t really cause any headaches but only make things potentially better. I’ll have to look into it

        • http://audiobox.fm/ Claudio Poli

          Another interesting bit is that the Puma phased-restart feature (SIGUSR1 to the master pid) does not work along with `preload_app!`,in that case a line is printed in the logs and a normal restart happens.

          So I guess the initializer stuff is good also for who wants the phased-restart feature.

          • https://www.codeship.io/ Florian Motlik

            Thanks for this feedback. Won’t work on Heroku, but super important on other Systems.

      • Scott R.W.

        We use preboot as well, but ran into major issues when running without –preload. This was a few months ago, so something in puma or the heroku stack may have changed since then, but at the time –preload was definitely required.

        Heroku cycles each dyno automatically once a day (more or less), and cycling doesn’t follow the preboot behavior. Instead it sends requests to the dyno as soon as the server binds to its port, which with –preload off is almost instantly. The dyno starts queuing requests which backup for 10+ seconds while the workers actually boot.

        A few times a day we’d see massive spikes of request queuing when heroku decided to automatically cycle one of our dynos. Switching to –preload completely fixed this.

        • https://www.codeship.io/ Florian Motlik

          Thanks for the feedback. We haven’t seen those spikes, but will give the preload a try and report back

  • jeremywoertink

    By chance do you know if there’s a reaper setup like that ActiveRecord one, but for Mongoid?

    • https://www.codeship.io/ Florian Motlik

      I don’t, sorry. Quick google didn’t show anything either

      • jeremywoertink

        No worries. Another question about this setup though… Puma has a “–preload” option. I don’t see in you guys using it in the Procfile, and it doesn’t seem to be used with the fake_work_app sample. Do you think using that mode would be beneficial?

  • narangjasdeep

    Shameless plug: I’ve been moving over my own apps to Puma lately and definitely noticed improvements in performance without switching VM (we’ve exclusively used MRI so far). Lately, even did a small blog post on the topic:

    http://jasdeep.ca/2013/07/deploying-rails-apps-with-puma-and-nginx/

    This should get you started on non-Heroku platforms with Puma & Rails, well for development & staging purposes. I still have to do a post on setting up Puma for actual production deployments.

  • Pingback: Learnings from using Puma in production on Heroku | Rocketboom()

  • Pingback: Learnings from using Puma in production on Heroku | Enjoying The Moment()

  • Conor Wade

    Thanks Florian great article.

    One quick question. How are you handling timeouts and locked dynos in your application with Puma it since doesn’t handle them?

    • https://www.codeship.io/ Florian Motlik

      We haven’t really had a problem with either so far, so if a Timeout does occur the connection is cut off by Heroku directly.

      We might look into this in the future if we see problems, but not necessary so far.

      • Conor Wade

        Thanks Florian.

    • Clint
      • Conor Wade

        I just tried rack-timeout yesterday and had a bit of an issue with it throwing some strange errors but I will endeavour again with it.

        • https://www.codeship.io/ Florian Motlik

          Rack-timeout would probably be my goto solution as well

  • http://alexjsharp.com/ Alex Sharp

    Hey, thanks for the helpful article. I’ve got one question about a unicorn detail:

    > Our assumption is that Unicorn hid this problem before by restarting the workers regularly.

    Can you elaborate on that? Did you have some custom logic that was bouncing unicorn workers every so often? Trying to understand it in context of unicorn vs puma memory consumption.

    Thanks!

  • nil2

    Thanks for the cite, appreciate it.

    -jonathan rochkind

    • https://www.codeship.io/ Florian Motlik

      Sure, thanks for doing the comparisons

  • Alex

    Thanks for the post, I’m following the topic unicorn vs puma for quite a while now, as we are using unicorn for delivering our SDK packages and providing platform builds here at V-Play. I recently found a SO thread ( http://stackoverflow.com/questions/18398626/why-did-gitlab-6-switch-back-to-unicorn ) about Gitlab switching back to unicorn because of heavy CPU loads in combination with MRI. Guess you haven’t experienced these issues too?

    • https://www.codeship.io/ Florian Motlik

      So far we haven’t really seen these problems. Puma is running very stable

  • http://www.behrang.org/ Behrang

    > We switched to 2X instances and still needed quite a lot of workers.

    I assume you mean Unicorn workers, not Heroku worker processes, right?

    • https://www.codeship.io/ Florian Motlik

      Exactly

  • Pingback: TI Especialistas Introdução ao Rails Composer()

  • http://zac.im/ Zachariah Clay

    I was running into some issues with your `database_connection.rb` initializer when deploying to heroku.

    The code in your example `if Rails.application.config.database_configuration` returns an empty hash if there are no database configurations in your config. An empty hash is still considered true, so the code continues and then assigns config to `Rails.application.config.database_configuration[Rails.env]` which is nil (because you looked up a key in an empty hash). The code then tries to assign a value to config. So, I kept getting an error when deploying that said `NoMethodError: undefined method `[]=’ for nil:NilClass` .

    On my instance at least, my database config is in `ActiveRecord::Base.configurations` so I can access it with `ActiveRecord::Base.configurations[Rails.env]`. (This is from heorku’s docs about database concurrency https://devcenter.heroku.com/articles/concurrency-and-database-connections#threaded-servers)

    So a different (maybe better way) to assign and check for config would be `if config = ActiveRecord::Base.configurations[Rails.env] || Rails.application.config.database_configuration[Rails.env]`.

    I forked your gist with updated code – https://gist.github.com/mebezac/00881c692e31d62726d3

    • https://www.codeship.io/ Florian Motlik @codeship

      Great, thank you. I’ve updated the text to link to your gist and comment. Very helpful!

      • http://zac.im/ Zachariah Clay

        Just switched over to Puma, thanks for this very helpful post!