Running Services in Docker 1.12

DevelopmentIndustry

Reading Time: 3 minutes

Great news! Docker for Windows and Mac is now in public beta, which means that Docker is that much easier to use for local development regardless of your preferred environment. You can download your preferred flavor of Docker at docker.com/getdocker.

Starting today, Docker for Mac and Windows also ships with Docker v1.12-rc2. With the new release, Docker becomes a really powerful tool for orchestrating your application.

What’s New in Docker 1.12?

Docker 1.12 comes with out-of-the-box capabilities for multi-container and multi-host app deployments, with networking and security built in. This means that with Docker 1.12 alone, you can deploy a highly available application to a Swarm cluster without needing to roll your own orchestration tools or spend a ton of hours trying to configure external orchestrators and network managers.

All of the orchestration features in 1.12 are backwards compatible. If you’re already an orchestration expert and have implemented your own solution, nothing will change. All of the extra orchestration features are opt-in.

To support the new orchestration capabilities, this release adds a couple new commands to the Docker API. Instead of only controlling things at a container level via docker run my-image, you can now declare a desired state for a service using the command docker service — provided you’ve set up your hosts in a Swarm cluster.

For example, I might want to run Elasticsearch as part of my application. With Docker 1.12 and using a Swarm cluster, I can scale out my Elasticsearch services quickly. Let’s start with just one replica of the Elasticsearch service:

~ $ docker service create --name elasticsearch elasticsearch
7u8dujdgrq8yk6tkqj6z8dcz8
~ $ docker service ls
ID            NAME           REPLICAS  IMAGE          COMMAND
7u8dujdgrq8y  elasticsearch  0/1       elasticsearch

Take a peek at the service tasks to wait for the replica to fire up.

~ $ docker service tasks elasticsearch
ID                         NAME             SERVICE        IMAGE          LAST STATE              DESIRED STATE  NODE
4f8xsmbcfzukakyn9nfl1ojc2  elasticsearch.1  elasticsearch  elasticsearch  Running About a minute  Running        moby

We can see that one instance of the service is running now, and executing docker service ls again will show that 1/1 replicas are up.

We can now take this service from 1 to 10 with just one command.

~ $ docker service update elasticsearch --replicas=10
elasticsearch
~ $ docker service ls
ID            NAME           REPLICAS  IMAGE          COMMAND
7u8dujdgrq8y  elasticsearch  10/10     elasticsearch

This feels a lot like docker-compose scale, but it’s fully integrated as part of Docker Engine.

Monitor the individual tasks running against the service by executing docker service tasks elasticsearch, and you’ll be able to see all 10 containers running. Note that docker ps is specific to the host you’re on, so you may only see a small fraction of the total containers running. Don’t panic — that’s because they’re running on different hosts. Containers are implementation details with this new pattern. Rely on docker service subcommands to check the state of your services.

All commands related to running services with the Docker API in 1.12 are declarative. This means that you tell Docker how you want your services to be running, and Docker takes over the work of making sure that the actual state of your cluster matches the declared state of your service. Instead of saying docker run elasticsearch, which is an imperative statement just telling Docker to execute a command, the declarative docker service create --name elasticsearch --replicas 10 elasticsearch is instructing Docker to make sure 10 instances of Elasticsearch are available at all times.

If a node in your cluster goes down or if a container fails, Docker takes care of restarting services to satisfy the declaration you made when you started the service. We can see this in action by simply nuking one of the Elasticsearch containers from orbit.

~ $ docker rm -f elasticsearch.6.54tytejr229scqfnwzrkbm7nt

If we’re super fast, we can catch our service in a state that doesn’t match our declared state of 10 replicas.

~ $ docker service ls
ID            NAME           REPLICAS  IMAGE          COMMAND
7u8dujdgrq8y  elasticsearch  9/10      elasticsearch

But give it a minute, and you’ll see that a new replica was created.

~ $ docker service ls
ID            NAME           REPLICAS  IMAGE          COMMAND
7u8dujdgrq8y  elasticsearch  10/10      elasticsearch

With v1.12, you have access to a powerful yet simple way create self-healing services running on a cluster of Swarm nodes, and this release helps make orchestration accessible for dev and ops alike.

There are tons of other cool features rolled into 1.12 related to networking and security. Check out the Docker blog post on 1.12. This new release is secure by default with end to end encryption, and also provides container-aware dynamic load balancing.

Subscribe via Email

Over 60,000 people from companies like Netflix, Apple, Spotify and O'Reilly are reading our articles.
Subscribe to receive a weekly newsletter with articles around Continuous Integration, Docker, and software development best practices.



We promise that we won't spam you. You can unsubscribe any time.

Join the Discussion

Leave us some comments on what you think about this topic or if you like to add something.

  • Pingback: "On the other side of the screen, it all looks so easy." - Kevin Flynn - Magnus Udbjørg()

  • Pingback: DockerCon 2016: Just Wow! | Docker Blog()

  • DanyC

    Nice write-up ! Do you still need docker-machine in case i use docker for mac to be able to spun up swarm cluster?

    I saw another blog saying i do need hence my question

    • Laura Frank

      Docker for Mac and Docker Machine are both fine — as long as you are using Docker version 1.12.

  • git_it

    Have you had any luck getting elasticsearch containers launched in this manner to cluster together? With the above commands, it launches 10 independent containers that are inaccessible.
    I tried to get them to cluster up by first creating an overlay network with
    docker network create –driver overlay mynet
    and then creating my service like this:
    docker service create –name es -p 9200:9200 -p 9300:9300 –network mynet elasticsearch -Des.network.host=0.0.0.0 -Des.discovery.zen.ping.unicast.hosts=es
    but the mesh routing to host “es” confused the containers, I think because they were all answering to the same IP address.
    I think it would work if I could specify -Des.discovery.zen.ping.unicast.hosts=es.1., but I’ve found myself in a chicken-and-egg scenario. I can’t get the task id of the first container until I’ve created the service, and I can’t create the service until I know the task id.
    I also tried creating separate services for es-master and es-node, so I could specify unicast hosts for es-node service, but it wouldn’t accept multiple services on the same port.
    So, I tried creating the master with
    docker service create –name es-master -p 9200:9200 -p 9300:9300 –network mynet elasticsearch -Des.network.host=0.0.0.0
    and the node service with
    docker service create –name es-node -p 9201:9201 -p 9301:9301 –network mynet elasticsearch -Des.network.host=0.0.0.0 -Des.discovery.zen.ping.unicast.hosts=es-master:9200 -Des.transport.tcp.port=9301 -Des.http.port=9201
    Both services start, but they still don’t see eachother. I can reach each one from the other at es-master:9200 and es-node:9201, but each elects itself as master.
    The next thing I’d like to try that I think could work is using calico so maybe they can share the same ports, but as far as I can tell, calico doesn’t work with 1.12 in swarm mode.

    • Joost van der Griend

      Running into the same issue. Have you made any progress?