A while ago, I wrote about how to bind-mount the Docker control socket instead of running Docker-in-Docker. This is a huge win for CI use-cases, and many others. Here I want to talk about a more generic scenario: controlling any Docker setup (local or remote Engine, but also Swarm clusters) from a container, and the benefits that it brings us.
Bind-mounting the control socket
If you have never done this before, I invite you to try the following command (on a Linux machine running a local Docker Engine, or if you are one of the lucky fews who have access to Docker Mac when this article is published):
docker run -v /var/run/docker.sock:/var/run/docker.sock \ -ti docker docker ps
This will execute
docker ps in a container (using the
official image), and it will display the containers running on your
local Docker Engine. There will be at least the container running
docker ps itself, and possibly other containers that are running
at that moment.
This gives us a way to control the Docker Engine from within a container. This is particularly convenient when you want to create containers from within a container, without running Docker-in-Docker.
However, this only works if you are connecting to Docker using a local UNIX socket. In other words, it doesn’t work if you are using:
- a remote Docker Engine (with or without TLS authentication),
- a local boot2docker VM,
- a Swarm cluster.
If you are using a remote Engine without TLS authentication,
the only thing you need to do is to set the
environment variable. Then all standard tools (like the Docker
CLI and Docker Compose) will automatically detect this variable
and use it to contact the Engine. But if you are using
TLS authentication (and you definitely should!) things
are a bit more complex.
I want to give you a generic method to connect to any Docker API endpoint from within a container, regardless of its location (local or remote), even if it’s using TLS, even if it’s actually a Swarm cluster instead of a single Engine.
Let’s look at our environment
To connect to remote Docker API endpoints using TLS, we need a bit more
DOCKER_HOST environment variable. First of all, we need
to tell our local Docker client to use TLS, by setting the
environment variable. We also need to provide the client with:
- a private key,
- a certificate (used to prove our identity to the remote server),
- a root certificate (used to check the identity of the remote server).
Those elements will be stored in three files in PEM format:
By default, the client looks for those files in the
but this can be changed by setting the
variable to another directory.
If you are using Docker Machine, you can easily check what those variables
look like with the
env command; e.g. if you have a Docker Machine named
$ docker-machine env node1 export DOCKER_TLS_VERIFY="1" export DOCKER_HOST="tcp://18.104.22.168:2376" export DOCKER_CERT_PATH="/home/docker/.docker/machine/machines/node1" export DOCKER_MACHINE_NAME="node1" # Run this command to configure your shell: # eval $(docker-machine env node1)
If we want to connect to this Docker Engine from within a container, we just need to set those environment variables, and make the three PEM files available within the container.
Transporting TLS material and settings
Of course, we could manually copy-paste the environment variables and
the three PEM files from our host to our container. But we are going
to automate the process, and store everything we need in a data container
dockercontrol. Assuming that our environment is currently
setup to talk to some remote Docker API endpoint secured with TLS,
we can run the following command:
$ tar -C $DOCKER_CERT_PATH -cf- ca.pem cert.pem key.pem | docker run --name dockercontrol -i -v /docker busybox \ sh -c "tar -C /docker -xf- echo export DOCKER_HOST=$DOCKER_HOST >>/docker/env echo export DOCKER_TLS_VERIFY=1 >>/docker/env echo export DOCKER_CERT_PATH=/docker >>/docker/env"
This will create a tar archive locally, containing the three PEM files;
then it will stream this archive to a container which will unpack it
on the fly; and finally we create an
env file that can be sourced
later to restore the environment variables.
Now, when we need to talk to our Docker API endpoint from a container,
all we have to do is to start that container with
--volumes-from dockercontrol, and
$ docker run --rm --volumes-from dockercontrol docker \ sh -c "source /docker/env; docker ps"
If you want a totally transparent operation (i.e. you don’t want
to change the container so that it sources
/docker/env) you can
also read that file on your host, and pass down the environment
variables to your containers.
Data containers vs. named volumes
You might be wondering why I’m using an old-fashioned data container
instead of creating a proper [named volume[ (with
docker volume create).
This would indeed be more
“Dockerish” but wouldn’t work (yet) with Swarm; e.g. when
docker run -v dockercontrol:/docker … we would have
to add an affinity constraint to make sure that the container
is created on the host that has the
I think it’s simpler to use a data container for now.
Putting everything together
I wrote a little shell script to automate the whole process; it’s available on jpetazzo/dctrl on GitHub.
It lets you run:
$ dctrl purple
This will create a data container named
the information necessary to connect to the current Docker API
Then, if you need to run a container that has access to this Docker API endpoint, you can do:
$ eval \$(docker run --rm --volumes-from $CONTROL alpine sed 's/DOCKER_/DOCKERCONTROL_/' /docker/env)" $ docker run --volumes-from $CONTROL \ -e DOCKER_HOST=\$DOCKERCONTROL_HOST \ -e DOCKER_TLS_VERIFY=\$DOCKERCONTROL_TLS_VERIFY \ -e DOCKER_CERT_PATH=\$DOCKERCONTROL_CERT_PATH \ ...
What can we use this for?
This allows us to create containers from a container, even when running on e.g. a Swarm cluster. (I initially thought about writing this blog post when 3 different persons, the same week, asked me “how can I bind-mount the Docker socket when the Docker Engine is not local, but on a remote host?”)
But this is also very useful in the general case when you need a container to be able to interact with your overall Docker setup; e.g. to setup or reconfigure a load balancer. See for instance dockercloud-haproxy, which accesses the Docker Events API to notice when backends are added to a service, and dynamically update load balancer configuration accordingly.
Another example would be the implementation of a replication controller in a container. This container would be given e.g. a Compose file and a set of scaling parameters (the number of desired intances for each service). It would bring up the application described by the Compose file, scale it according to the scaling parameters, and watch the Docker Events API to adjust the number of containers should any node go down during the lifecycle of the application. (That container would be started with a rescheduling policy, to be automatically redeployed by Swarm if its own node goes down.)
Generally speaking, any kind of application that needs access to the Docker API would benefit from this as soon as you want to be able to run it seamlessly in a container.
Power to the people
This takes me to one of my favorite features of Docker Swarm: the fact that it uses the same API as the Docker Engine.
This means that as a developer, when I build my application on my local machine with a single Docker Engine, I leverage the full API that will be available on a Swarm cluster:
- if I need to partition my app across multiple networks,
I can do it on a single node with the default
bridgedriver, and when I deploy on a Swarm cluster, everything will work exactly the same way, thanks to the
overlaydriver (or whatever network plugin has been deployed by my ops team);
- if I need to use persistence and volumes, same story: I will use the default local driver in my environment, and if I’m running on a cluster with a volume plugin like Flocker or PortWorx, it will automatically achieve reliable persistence without changing anything on my side;
- if I want to automatically scale up and down a background worker depending on the backlog size of a message queue, I can develop and test this locally, because the API used to scale (and gather metrics) will be the same in my local environment and the production one.
This last example is particularly powerful. If you are developing an app intended to run in the public cloud, and want to use auto-scaling, you won’t be able to test the auto-scaling behavior locally - unless your cloud provider gives you the option of installing a fully functional cloud instance locally, on your development laptop. With Docker, the fully functional cloud instance is the Docker Engine that you’re already using to power your containers.
If you have some other creative scenario involving controlling the Docker API from within a container, let me know!
Want to learn more about Docker? I will deliver two Docker workshops next month (May 2016) in Austin, Texas: an intro-level workshop and an advanced orchestration workshop using Compose, Swarm, and Machine to build, ship, and run distributed applications. If you want to attend, you can get 20% off the conference and workshop prices using the code PETAZZONI20. I will also deliver those workshops at other conferences in Europe and the US, so if you’re interested, let me know!