~jpetazzo/My notes on Amazon's ECS (EC2 Container Service), aka Docker on AWS

Discuss on Hacker News

This morning, I watched AWS’ webinar presenting their container service. Here are some quick notes, for those of you who are as curious as I was about it!

This is not meant to be an intro to Docker. This is not meant to be an intro to EC2 or to AWS. This is for people who are already familiar with AWS, specifically with EC2, and who are already familiar with Docker, and wonder what’s behind the ECS (EC2 Container Service) announcements made at AWS re:invent last November.

AWS has made the video available if you want to watch the webinar yourself.

Bullet points

TL,DR:

I'm out of bullet points

Glossary of terms

Here is some vocabulary to help you to mash through the ECS docs.

Container instance

A “container instance” can be any EC2 instance, running any distro (Amazon Linux, Ubuntu, CoreOS…)

It just needs two extra software components:

The ECS agent is open source (Apache license). You can check the ECS agent repo on github.

Cluster

That’s a pool of resources (i.e. of container instances).

A cluster starts being empty, and you can dynamically scale it up and down by adding and removing instances.

You can have mixed types of instances in here.

It’s a regional object, that can span multiple AZs.

Task definition

It’s an app definition in JSON. The format is conceptually similar to Fig, but not exactly quite like it.

I don’t know why they didn’t pick something more like Fig, or more like the Docker Compose project. It might be because almost everything else on AWS is in JSON, and they wanted to stick to that.

Note: Micah Hausler wrote container-transform, a tool to convert Fig/Compose YAML files to the ECS task format:

Task

A task is an instanciation of a task definition. In other words, that will be a group of related, running containers.

The workflow

So, how does one use that? The workflow looks like this:

  1. Build image using whatever you want.
  2. Push image to registry.
  3. Create JSON file describing your task definition.
  4. Register this task definition with ECS.
  5. Make sure that your cluster has enough resources.
  6. Start a new task from the task definition.

Now, diving into the details; there are 3 ways to start a task:

  1. Use the CLI command start-task. You must then specify the cluster to use, the task definition, and the exact container instance on which to start it. It’s a bit like doing manual scheduling.
  2. Use the CLI command run-task. You must then specify the cluster, task definition, and an instance count. It will run ECS default resource scheduler (which is a random scheduler).
  3. Bring your own scheduler!

The webinar had a demo involving Mesos; they started container from Marathon, from Chronos, and using the CLI as well, and the containers were visible everywhere. That looked cool. Initially, I didn’t understand how it worked; but the people who built it were kind enough to chime in and explain:

Networking

Not much on that side.

Containers can be linked (the task definition allows to name containers, and then to indicate that a container is linked to another one) but I don’t know how that works.

My personal take

My understanding is, that ECS as it is today is a technological preview.

There are a some items that are still to be clarified, like the use of private registries (but on that front, Docker Hub Enterprise might eventually come on AWS; and it will likely integrate nicely with ECS).

Docker-centric point of view

I would love if:

Those interoperability points would let anyone move their container workloads seamlessly form/to ECS. More importantly, they will let anyone use the elasticity and scale of EC2, without having to learn APIs and concepts specific to ECS.

AWS-centric point of view

I would love if:

Last words

Full disclaimer: I haven’t tested ECS yet (and unfortunately, I don’t know if I’ll be able to). So if you have any feedback or useful tip that would be useful for others, don’t hesitate to let me know!

This work by Jérôme Petazzoni is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.