Lead Image © jager, 123RF.com

Lead Image © jager, 123RF.com

Deis combines Docker and CoreOS

Container Ship

Article from ADMIN 25/2015
Deis combines Docker and CoreOS to create a platform-as-a-service tool, and the developers say version 1.0 is ready for production.

Container-based virtualization solutions such as Docker have less hardware overhead and support higher densities than classical virtualization tools such as KVM [1] or Xen [2]. This increased efficiency, along with simpler management, explains the meteoric rise of Docker [3], LXD [4], OpenStack [5], and other container-based tools: Everything called a "container" is currently very popular, and no end is in sight for the hype (Figure 1).

Figure 1: Despite the hype surrounding the two components, Docker and OpenStack are still not a perfect couple, which is why other providers are sensing an opportunity. (Image from: blog.docker.com)

Developers are busily building a whole new fleet of solutions designed to support the container paradigm. Project Atomic [6] and CoreOS [7] [8] are perfect examples of this trend. CoreOS is a naked Linux distribution that only supports one thing really well: the operation of Docker containers (see the "CoreOS" box).


Microdistributions such as CoreOS are in fashion. CoreOS offers many useful tools that make it easy to virtualize systems centrally. The etcd tool [9], for instance, continuously equips all virtual machines (VMs) with current versions of all relevant configuration. Systemd takes care of the resources on individual hosts, and Fleet [10] orchestrates the system fleet.

Useful built-in components make systems like CoreOS popular; therefore, they play an important role in the future plans for many IT companies.

However, not everything is on a positive track in the Docker world. Many Docker detractors believe the root of the problem is Docker's overlay design, because they think its modifications are too wild, but the security also leaves something to be desired.

A Docker container is internally nothing more than a filesystem that receives various deltas via an overlay layer (AuFS), including the software Docker container users need. The problem is, what works well for developers using Docker regularly goes wrong in the operator's daily work, because if the admins install just any container, they may get a Trojan horse in their setup. Moreover, what boots for the developer is not automatically ready for production use. No wonder then that some people regard Docker as a great tool for developers but a disaster for operators and unhelpful in the devops area.

Deis [11] steps up to avoid some of Docker's architectural vulnerabilities and to integrate the product into a production-ready context. According to the Deis project website, "Deis is a lightweight application platform that deploys and scales 12-factor apps as Docker containers across a cluster of CoreOS machines" (see also the box titled "Twelve Factors").

Twelve Factors

Heroku and several other PaaS providers created a system of rules for building portable applications that run easily on PaaS platforms. Deis sticks rather precisely to these rules. These "12 factors" [13] serve as guidelines for building PaaS-ready applications:

  • The PaaS application code is always in a tracked repository such as Git [14].
  • Features used from other software are clearly stipulated, explicitly declared as dependencies, and isolated.
  • Configuration files using login credentials are not put in the application itself but are instead a part of the environment in which a PaaS application is running. The Deis etcd tool implements this policy perfectly: It reads all parameters from a database with a central API from which all services feed.
  • Services (e.g., MySQL databases, as well as Twitter and other social networking services) and applications cannot be separated from one another during deployment (e.g., etcd runs in the background but provides important information about the application).
  • The preceding rule does not apply to development and production lines, which the admin separates better. The point is explicitly aimed at typical Docker containers that often go directly from the developer's hands into live operation.
  • Stateless protocols scale better in width than stateful protocols and are given preference if technically possible. A defined standard port guarantees the accessibility of every application.
  • Apps are completely self-contained and export services via port binding.
  • Small processes work independently, so the application scales better.
  • Software that runs robustly but can also be started quickly and ended reliably provides the user with a consistent PaaS feeling.
  • Developing and staging activities are based as closely as possible on the actual production model.
  • An admin understands logs less than text files, but more than event sequences; apps should not save or route output streams but simply send them to stdout. The developer can watch the streams, but production systems should save them externally.
  • Administration and management processes should run independently of processes that are necessary for everyday use.

These 12 concepts should also help Deis handle applications or locales for users based on CoreOS, Docker, and Git.

The Deis project aims to clear the container jungle around Docker and create standardized containers that then provide platform-as-a-service (PaaS). The Deis developers say their big model was Heroku [12] – a pioneer of PaaS cloud offerings – and the similarities are evident.

The tricky task for any provider is: How do you equip the VMs to satisfy what the customers want as a platform for operating software? Also, how can this process be automated so that "as a service" are not just empty words and customers can stock up on virtual systems as required? Deis provides a toolbox with which admins can bring together the platforms for their customers.

Deis in a Nutshell

Deis uses etcd to scale a PaaS application sensibly in width. If Deis starts a PaaS VM, etcd steadily receives a series of entries, which an admin can use to restore the VM later.

The Deis developers themselves describe the workflow of their application in the Deis documentation using a graph (Figure 2). The figure clearly shows the three core components: Deis serves as a kind of tool that takes care of the coordination between the PaaS services, Docker, and CoreOS; that is, it acts as the control panel.

Figure 2: The Deis workflow can be roughly divided into three phases. (Image from: http://docs.deis.io)

On the basis of an initial image, Deis creates Docker containers on demand with specific capabilities. CoreOS then runs the constructed Docker containers.

The Deis workflow occurs in three phases. In the first phase, Deis generates the container. Relying on Git, Deis receives information about container properties via finished instruction packages – called applications . The applications contain the code that is later supposed to run within a container.

So, if an operator wants a Jetty server, the corresponding Deis application will supply Java code. In every application, a special file called procfile describes the application and language to Deis.

A whole range of completed applications [15] on Deis is already waiting in the network. The user can install the applications via the Deis client, which is a separate command-line tool. A deis create command running in the application directory will suffice to create the application. Then, git push deis Master is used to trigger the build process (Figure 3).

Figure 3: From the application to the finished container in a few seconds: git push makes it possible. However, Deis is still missing a GUI today.

In the background, Deis creates a Docker container based on blank templates that obeys the application commands. Ubuntu 14.04 is standard for these images, but you can specify the image through a specific file in the application: All options for the user are exposed by Dockerfile [16], which Docker also offers.

Because Deis always builds a container from plain images, an application that follows the 12 factors can be reproduced easily later. Thus, in the second phase, a naked container without its own identity lands in the Docker registry.

A classic Docker container from Deis does not contain support services such as databases. The PaaS provider is given the task of operating a central database or central instances of other services that containers refer to later. This design ensures scalability because PaaS customers no longer have to think about the infrastructure.

The third phase deals with VM scheduling. A separate Deis component – the scheduler – goes through the list of available CoreOS hosts in the background and decides on which host it will start the container. If a suitable host is found, Deis gets the container off the ground and gives it a range of configuration parameters that are not part of the image. A Deis container is, in fact, a generic Docker image that Deis combines with specific settings from etcd at run time.

The Network et al.

The impressive Deis network component, which is referred to as a router , demonstrates that Deis is deeply involved in the Docker processes. To enable PaaS for different protocols, Deis must centrally control how VMs access their network. The router component is basically the jack of all trades (Figure 4): Based on Nginx load-balancing capabilities, the Deis router ensures that VMs are accessible externally. Routers in Deis are horizontally scalable and work like a mesh network. They also only expose VMs if the container underlying a VM is started successfully.

Figure 4: The basic Deis components are control and data planes. The routers, which also work as a mesh network, are particularly important. The black squares are the attached resources. (Image from: http://docs.deis.io)

In addition to central components such as builders, registry (the Deis API), and routers, several other components contribute to Deis. PostgreSQL stores state information about the VMs as a database, and Ceph provides persistent storage to containers that need it. A centralized logging system uses Logspout and Logger. Via an API, the admin can find out the current state of each VM. A cache based on Redis speeds up communication.

Steep Learning Curve

Deis offers many exciting and useful features, but the complex configuration means a lot of background work for the administrator to get the required Deis infrastructure up and running.

The maxim, leaving most of the work to git push as the only necessary command, is in the Heroku style. Deis does not grasp it completely, but the deis create preceding the container start is pretty close to the target, which may be enough for developers. Anyone who is used to working with Git and is familiar with how it works will like the simple approach.

Not all PaaS platform customers are trained programmers, however. PaaS is supposed to create low initial hurdles, so a user interface that customers could use to click together their containers delivered by Deis, would be important.

Deis is missing a working GUI, though, and there is nothing in the network regarding this issue – except for a bug report [17]. According to a statement by Deis developers, a user interface is not on the to-do list for the near future. Anyone who would like to familiarize less savvy customers with a company's PaaS services will find the process difficult with Deis.

Buy this article as PDF

Express-Checkout as PDF
Price $2.95
(incl. VAT)

Buy ADMIN Magazine

Get it on Google Play

US / Canada

Get it on Google Play

UK / Australia

Related content

comments powered by Disqus

SysAdmin Day 2017!

  • Happy SysAdmin Day 2017!

    Download a free gift to celebrate SysAdmin Day, a special day dedicated to system administrators around the world. The Linux Professional Institute (LPI) and Linux New Media are partnering to provide a free digital special edition for the tireless and dedicated professionals who keep the networks running: “10 Terrific Tools."

Special Edition


Subscribe to ADMIN Update for IT news and technical tips.

ADMIN Magazine on Twitter

Follow us on twitter