Container OS Comparison


Reading Time: 5 minutes

For anyone who’s been following the container community’s rise the past two years (after Solomon Hykes’s famous five-minute presentation at PyCon), you have surely seen many companies and projects spring up, offering really innovative ways of managing your applications.

There are several projects around management, network, storage, logging, monitoring, and more (check out this awesome mind map of the ecosystem). However, I think the most prevalent projects are the ones that make up the base infrastructure of what has or will become your new application environment: the container OSes.

One question I always get when talking to people about containers at meetups and conferences is “Which is the best OS to run containers?” That’s usually followed by “Is it CoreOS? What about RedHat? I’ve also heard about something called RancherOS?”

I love these discussions; it’s very similar to “Which Hypervisor is the best?” Of course, the answer is always “It depends.” Still, I’m going to try to explain some of the key benefits and differences between the most popular container OSes that are available at the time of writing.


This is the poster child of container OSes. CoreOS is focused on large-scale deployments, mostly targeting enterprises, and it’s gained quite the following (hundreds of contributors and usually 500+ IRC users in #coreos on FreeNode). It comes bundled with a few really interesting tools developed by the CoreOS team, such as etcd, fleet, and flannel. These tools help you get started with a cluster of CoreOS very quickly. They’re also great starting points to get to understand the concepts behind service discovery, resource scheduling, and container networking.

In December 2014, the CoreOS team also announced another type of container runtime engine called rkt. This was a response to what the team saw as Docker’s diversion from its original container manifesto, and they wanted to see if the community felt the same. CoreOS still runs both Docker and rkt containers, so no worries about functionality issues yet or in the near future.

The CoreOS team has also teamed up with Google (Google Ventures is also an investor in CoreOS) and has created Tectonic, a really interesting way of getting CoreOS running with Kubernetes easily and efficiently. Tectonic is a commercial Kubernetes platform, which can be important if you’re looking for more than community support when running large-scale containers in production.


Rancher is trying to take what a container OS should be to the next step, where everything in RancherOS is a Docker container. They run a system Docker as PID 1 and then launch a container that runs the user Docker for all the user containers.

This might seem crazy, but for an OS that should do nothing else, it makes sense. Rancher has stripped away everything not needed, which makes the OS very lightweight. The installation ISO is just 22MB.

What is even more interesting are all the services added on top of the OS, using the Rancher system. When you consider what’s needed in a container production system, you usually need functions such as secure and easy networking, service discovery, load balancing, monitoring, and scheduling. Rancher adds all this on top of RancherOS, plus some. It’s a very comprehensive system, and I highly recommend you give it a closer look.

Snappy Ubuntu Core

This is an interesting project that Mark Shuttleworth announced last year while also calling some of the then-available container OSes “bloated.”

The Snappy Ubuntu Core OS comes with a new type of application manager (“snappy”) and focuses on running apps and containers. Some might argue that this goes against what a container OS should do, but it might also be a good transitional OS. It presents a great learning opportunity for those who just want to test things out and don’t have the time to learn the intricacies of etcd, Consul, fleet, Kubernetes, and all the other tools.

The base of the system is the “Ubuntu Core.” On top of that, your apps lives in read-only images (similar to containers), and the apps can be transactionally updated. This is huge — you don’t have to download an entire application again to deploy a new version, just download the changes that have been made.

The Snappy Ubuntu Core OS isn’t a pure container OS but has some interesting aspects to it. Anyone who’s been running Ubuntu in production or is interested in running both apps and containers side by side should definitely check it out.

RedHat Project Atomic

This distribution is built using upstream RPMs from CentOS, Fedora, and RHEL, and enables what RedHat calls “atomic” upgrades and rollback. It’s up to you, dear reader, to choose which distribution you want as a base so you feel comfortable with managing your hosts.

The OS comes with built-in functions for Docker, flannel (from the CoreOS team), Kubernetes, a transactional operating system update tool called rpm-ostree that always keeps an older version of the OS available (similar to CoreOS), and of course systemd.

Project Atomic also uses SELinux to try to secure your containers as well as manage access to and from them. I think it’s great that they’re using already established and trusted tech for this. I’m guessing we will see more from RedHat around this project, but for now it’s pretty barebones — which can also be a good thing.

Mesosphere DCOS

Mesosphere DCOS is a project that is unfortunately commonly mistaken for Apache Mesos (naming issues?). But ignoring that, it has a very robust and innovative way of looking at how to manage containers.

It takes the open source projects Apache Mesos, Marathon, Zookeeper, and a few other services, bundles them together in a clever way and also adds enterprise features on top. This DCOS product just went GA and comes in two flavors:, a community edition (free of charge) for AWS workloads, and an enterprise version for everything else.

The most interesting thing about Mesosphere DCOS is that it’s not just limited to container management. It’s building on Mesos, after all, which can do a lot more. How about deploying a Hadoop cluster? Maybe a large-scale Cassandra cluster? Mesosphere has that built-in, and I believe this is one of the key differentiators from the other container OSes that can make Mesosphere DCOS very successful.

VMware Photon

Announced and released in April, VMware Photon is a very new container OS and the first part of an ongoing open-source effort by VMware. VMware is definitely focusing on large-scale deployments of applications, as can be seen by their other project called Lightwave, providing identity services including authentication and authorization for large-scale distributed infrastructure, applications and containers. More to come here, I’m sure.


Obviously, there’s a lot of work going into all of the above projects, and they each bring something of their own to the field. I hope this comparison will help you pick a project you find interesting and explore the possibilities of the different container OSes and their management systems. Have fun!

PS: If you liked this article you might also be interested in one of our free eBooks from our Codeship Resources Library. Download it here: Turning Your App into Separate Containers for Better CI/CD

Posts you may also find interesting:

Sign up for a free Codeship Account

Subscribe via Email

Over 60,000 people from companies like Netflix, Apple, Spotify and O'Reilly are reading our articles.
Subscribe to receive a weekly newsletter with articles around Continuous Integration, Docker, and software development best practices.

We promise that we won't spam you. You can unsubscribe any time.

Join the Discussion

Leave us some comments on what you think about this topic or if you like to add something.

  • What about SmartOS? It’s been container-native for, like, ten years and Joyent just recently added support for running Docker containers on bare metal.

  • Jonas Rosland

    Hi Carlos, thanks for your comment! Yes, SmartOS is indeed very interesting and has been around for many years already. They’ve done some incredible work to get Docker containers to work on their platform. The purpose of the comparison here is to show some of the Linux OSes that are focused on containers (mainly Docker but also others), and it’s never gonna be an exhaustive list. SmartOS could’ve been on the list as another alternative.

    • To you point about this not being exhaustive and Carlos invoking the 10 year mark… I’d be curious to get your thoughts on NixOS as Nix approaches a 2.0 release level (circa 2004!). While I’ve only dabbled with NixOS, it is an interesting approach.

      • Jonas Rosland

        Hi Jay!

        NixOS is a really cool Linux distribution, and you can absolutely run containers on it, but it’s not a container-focused distribution so it didn’t fit the bill of this comparison.

    • Jim Pick

      I was wondering why SmartOS wasn’t on the list too. Pretty big omission.

  • “Which is the best OS to run containers?” Failing to mention SmartOS is like writing an article about graphical user interfaces and only mentioning Windows derivatives without mentioning the Macintosh.

  • Matrix AI

    We created a list of “data center operating systems” here Feel free to make a pull-request.

  • martintajur

    What about Amazon ECS and Google’s Container Engine? How would using such services compare against hosting a bunch of hosts yourself, running one of the OSes mentioned above?

    • Jonas Rosland

      Hi Martin!
      They are both great services, you have more such as TutumCloud and then you have PaaS solutions such as Pivotal’s Cloud Foundry and EngineYard’s Deis. There are so many options out there, in the article I focused on the container OS that you would run in your own DC, not hosted environments. I highly recommend trying them out as well, as they can be a great alternative or addition to your infrastructure.

  • Pingback: Technology Short Take #52 · Scott's Weblog · The weblog of an IT pro specializing in virtualization, networking, open source, and cloud computing()

  • Pingback: Container OS comparison | via @codeship | 机械鼠的博客()

  • Larry Weya

    I definitely agree that Joyent’s SmartOS deserves a mention on this list. Its been used in production for years and I would argue that its the most mature of the bunch plus now runs docker.

  • Pingback: Docker: A Comparison of Minimalistic Operating Systems | inovex-Blog()

  • Joe Drumtra

    Thanks for the overview. Could you provide an update on any recent OS gains/losses? @jonasrosland:disqus

    • Jonas Rosland

      Hi Joe,
      That might make for a great update blog, I’ll think about it. Thanks for the feedback!

  • Pingback: Контейнерные ОС следующего поколения. CentOS Atomic Host - Имею скафандр, готов к приключениям()

  • What about ResinOS? – that one seems very interesting!

  • Pingback: Oracle发布开源的容器工具集,包含使用Rust编写的新容器运行时环境 - 莹莹之色()

  • Mustafizur Rahman

    Hi Dev,
    Here I have share few of my experience about the CoreOS and SmarOS on production.
    SmartOS is fork of Sun 5.1 and CroeOS is fork o Gentoo Linux . SmartOS is quite good features which we can say Smart Hypervisor for LXC, KVM and native OS. SmartOS is growing more pkg and docker capability. Most reliable and advantage is ZFS integrated which is very less data loss and performance penalty.
    But SmartOS doesn’t support the HA itself same on CoreOS. Actually for OS container we need real HA 1:N means at least 1+2 for backup where real needs ,
    1. Virtual network HA
    2. Drive HA or
    I have been pocking around how I can keep my all container 100% up running still I couldn’t get it. Where each container runs : Nginx, Mysql, php for each customer application. I have been using last 1 year in production SmartOS but it has few difficulties for container. I have run 500+ centos 6.8 LX machine but after few days when load is high server tools are not working like vmadm , ipadm, zoneadm.
    Last week I have faced zone drive and swap allocation issue on that time its halt all container application but it doesn’t throw any error and bug. Its looks like SmartOS and all other cloud OS need to more efficient to manage its server resources. But SmartOS and CoreOS is very amazing for CPU and Memory speed but issue is drive management.

    I need help how I can make the HA 1:2 for each customer container where all container data will real time replica of each physical servers.

    if anyone know how to manage SmartOS / CoreOS container level HA ? Does it handle own or its depends on other OS like DC/OS or Kubernet.. Also it doesn’t have control over OS usages own memory and zone zfs allocation.

  • Pingback: A Comparison of Minimalistic Docker Operating Systems()

  • Pingback: Kubernetes 운영 17개월의 성과와 교훈 » 안드로메다 토끼()

  • Andreas Sylvan

    What about resinOS and hypriotOS?