Correctly integrating containers

Let's Talk

Pitfalls

Some of the pitfalls on the way to creating useful Kubernetes clusters also need to be mentioned here. Docker, for example, needs to launch a pause container to create a network namespace. In this case, a process that does nothing else runs in this container. Bootstrapping with Docker does not always work without trouble. On Red Hat systems with Calico, for example, inconsistencies with the firewalld daemon occurred and caused a number of solvable yet irritating problems.

Conclusions

Kubernetes is a rapidly evolving ecosystem that pursues many very good strategies the project has not yet fully implemented. If you are aware of the three-month release cycle and are ready to join the development work on Kubernetes, you can look forward to a system that makes developing, updating, and customizing distributed applications easier than any other distributed system.

If you need multitenancy (see the "Multitenancy in Kubernetes" box) or do not work with highly security-sensitive data, Kubernetes can be used now for large, distributed production applications. However, if you have other applications in mind, you should wait one or two versions.

Multitenancy in Kubernetes

Apart from safeguarding network policy, it is also important to secure a cluster at the container level, which means the user is not allowed to use privileged containers unless explicitly authorized to do so.

Users prove their identity to Kubernetes with client certificates or – less recommended – passwords. External services can also be connected. Kubernetes automatically creates client certificates as secrets when creating service accounts. Related concepts will be delivered during the alpha to beta stage of production, although it works quite well already.

Privileged containers are the new root privileges. They are very powerful and dangerous if misused. To build Docker images, you would, for example, embed the Docker socket of the surrounding host in these containers. However, in the worst case, the surrounding node can be hijacked.

The path to a secure system is not easy and can only be outlined here. First, kube-admin creates service accounts, then generates roles via role-based access control (RBAC) [20], and binds them to users or groups. The right to administer a security policy, for example, must be severely restricted.

Roles give rights to resources in the form of verbs. For concepts with pods, verbs are actions: REST API calls here. Put simply, they give rights to perform read (get and list) or write (create or update) operations on pods, services, ingresses, or secrets. Doing so often goes deep into resources and can, for example, prevent users from creating pods without a security context – or with the wrong one.

If you withdraw the right to run root processes in containers with spec.securityContext.runAsNonRoot:true, most of the examples from the Internet will no longer launch. You should definitely switch on the feature in secure environments (Listing 9). It is well implemented with different degrees of success in the various implementations of Kubernetes; you are on the safe side with OpenShift and the Kubernetes distribution by Red Hat.

Listing 9

Pod with Security Context

01 [...]
02 apiVersion: v1
03 kind: Pod
04 metadata:
05   name: hello-world
06 spec:
07   containers:
08   # Specification of the Pod's Containers
09   securityContext:
10     readOnlyRootFilesystem: true
11     runAsNonRoot: true
12 [...]

Infos

  1. Kubernetes: https://kubernetes.io
  2. Cluster networking: https://kubernetes.io/docs/concepts/cluster-administration/networking/
  3. Project Calico on GitHub: https://github.com/projectcalico/calico/
  4. According to the Kubernetes documentation: "a pod is a group of one or more containers (such as Docker containers), with shared storage/network, and a specification for how to run the containers. A pod's contents are always co-located and co-scheduled, and run in a shared context": https://kubernetes.io/docs/concepts/workloads/pods/pod/
  5. Kubernetes v1.7: http://blog.kubernetes.io/2017/06/kubernetes-1.7-security-hardening-stateful-application-extensibility-updates.html
  6. Network policies in Kubernetes: https://kubernetes.io/docs/concepts/services-networking/networkpolicies/
  7. Flannel: https://github.com/coreos/flannel
  8. Packet path: https://github.com/coreos/flannel/blob/master/packet-01.png
  9. Plugin specs for the Container Network Interface: https://github.com/containernetworking/cni/blob/master/SPEC.md
  10. Calico networking using Vagrant and the Calico CNI plugin: https://github.com/projectcalico/calico/tree/master/v2.2/getting-started/kubernetes/installation/vagrant
  11. CoreOS cloud config: https://coreos.com/os/docs/latest/cloud-config.html
  12. ConfigMaps in Kubernetes: https://kubernetes.io/docs/tasks/configure-pod-container/configmap/
  13. DaemonSet: https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/
  14. Felix on GitHub: https://github.com/projectcalico/felix
  15. Calico network policy: https://github.com/projectcalico/k8s-policy
  16. HAProxy: https://haproxy.org
  17. Nginx reverse proxy: https://www.nginx.com/resources/admin-guide/reverse-proxy/
  18. F5 hardware: https://f5.com/products/deployment-methods/hardware
  19. "Kubernetes and CoreOS" by Thomas Fricke, ADMIN , issue 36, 2017, pg. 36, http://www.admin-magazine.com/Archive/2016/36/Safeguard-and-scale-containers
  20. RBAC: https://kubernetes.io/docs/admin/authorization/rbac/

The Author

Thomas Fricke is CTO of Endocode AG in Berlin, Germany. He is a cloud, database, and software architect involved in system automation and DevOps. Contrary to all preconceptions about Germany's capital, he travels through the city of Berlin and its forests by bike.

Buy this article as PDF

Express-Checkout as PDF
Price $2.95
(incl. VAT)

Buy ADMIN Magazine

SINGLE ISSUES
 
SUBSCRIPTIONS
 
TABLET & SMARTPHONE APPS
Get it on Google Play

US / Canada

Get it on Google Play

UK / Australia

Related content

  • Safeguard and scale containers
    Security, deployment, and updates for thousands of nodes prove challenging in practice, but with CoreOS and Kubernetes, you can orchestrate container-based web applications in large landscapes.
  • Monitoring container clusters with Prometheus
    In native cloud environments, classic monitoring tools reach their limits when monitoring transient objects such as containers. Prometheus closes this gap, which Kubernetes complements, thanks to its conceptual similarity, simple structure, and far-reaching automation.
  • A Hands-on Look at Kubernetes with OpenAI
    For research into deep learning algorithms that automatically acquire new skills, OpenAI operates some of the largest Kubernetes clusters worldwide, with up to 36,000 CPU cores. We look at some practical experience with the container management system.
  • Automated orchestration of a horizontally scalable build pipeline
    A Jenkins build pipeline frees developers from repetitive manual build tasks by facilitating automation and provides broad traceability with continuous logging and feedback.
  • OpenShift 3: Platform as a Service
    Red Hat's OpenShift is aimed at companies seeking low-maintenance PaaS environments in which they can develop and test applications.
comments powered by Disqus