Photo by Austin Neill on Unsplash

Photo by Austin Neill on Unsplash

Configuring complex environments

At the Controls

Article from ADMIN 68/2022
By
YAML is often the language of choice when configuring complex environments. We help you get started with YAML and the YAML parser yq.

The DevOps world without YAML [1] is difficult to imagine. In fact, YAML is a superset of JavaScript Object Notation (JSON) [2]. However, the focus of JSON is more on data serialization (e.g., to make data available to an API).

In contrast, YAML plays to its strengths when used as a configuration language because the format is more easily readable than JSON. Python programmers love YAML because, unlike JSON, it uses indentations instead of parentheses to define objects.

Basic YAML Syntax

Listing 1 shows a simple YAML document. The --- string in the first line means a file can contain several such documents; it is then followed by typical key-value pairs, which are familiar if you have used JSON. The first key pair is a simple scalar with a string value, although numbers and booleans are also allowed. The list that follows is a collection of objects. In this case, only numeric values are used, each of which is indented with spaces.

Listing 1

YAML Objects

---
name: starwars collection
year of publication:
    - 1977
    - 1980
    - 1983
movies:
# Only movies from the original trilogy (OT) are listed here.
    ot:
       - Episode IV - A New Hope
       - Episode V - The Empire Strikes Back
       - "Episode VI - Return of the Jedi Knights."

You should avoid using tabs if possible because they can cause issues when processing the data. By the way, you do not have to write strings in parentheses, as shown in the final line of Listing 1. This collection of key-value pairs is a dictionary. Unlike JSON, you can also work with comments in YAML without problems. Comments are introduced at the beginning or end of a line with the hash mark (#).

To process the data stored in this way with Python, you could use the PyYAML module, which converts YAML objects into Python dictionary (dict) objects, which you can then process further according to your own requirements. Listing 2 shows a simple example of the Python script reading data from the starwars.yaml file and forming it before output.

Listing 2

starwars.py

#!/usr/bin/env python3
 **
import yaml
from yaml.loader import SafeLoader
 **
with open('starwars.yaml') as f: sw = yaml.load(f, Loader=SafeLoader) print(yaml.dump(sw, indent=4, default_flow_style=False))

Command-Line YAML Parser

The yq [3], [4] parser is a very good choice for processing configuration files written in YAML. Because it is based on the well-known JSON jq parser [5], it uses very similar syntax. A nice side effect is that you can process JSON data with yq, as well. If your Linux distribution does not offer a preconfigured yq package, simply install the software directly from the GitHub page:

wget https://github.com/mikefarah/yq/releases/download/v4.14.1/yq_linux_amd64 -O ~/bin/yq
chmod u+x ~/bin/yq

On macOS, you can also import the software with the Homebrew package manager.

Searching YAML Documents

A typical task when processing YAML files is to search for a specific key and the value assigned to that key. For example, if you want to filter out all years of publication from the starwars.yaml file, use:

yq eval ".publication-year[]" starwars.yaml

If you only want to know when the first movie was released, put an index on the first element of the list:

yq eval ".publication-year[0]" starwars.yaml

Listing 3 contains a slightly more extensive YAML document that describes a Kubernetes pod. For an initial overview of what keys this file contains, run the command:

yq eval keys pod.yaml

Listing 3

Kubernetes Pod in YAML

apiVersion: v1
kind: Pod
metadata:
    name: my-pod
spec:
    containers:
    - name: db1-container
       image: k8s.gcr.io/busybox
       env:
       - name: DB_URL
          value: postgres://db_url:5431
    - name: db2-container
       image: k8s.gcr.io/busybox
       env:
       - name: DB_URL
          value: postgres://db_url:5432

You can view a list of all container names with:

yq eval ".spec.containers[].name" pod.yaml

The command

yq eval '.spec.containers[].env[].value | select(. == "*32")' pod.yaml
 postgres://db_url:5432

filters with the select function.

Buy this article as PDF

Express-Checkout as PDF
Price $2.95
(incl. VAT)

Buy ADMIN Magazine

SINGLE ISSUES
 
SUBSCRIPTIONS
 
TABLET & SMARTPHONE APPS
Get it on Google Play

US / Canada

Get it on Google Play

UK / Australia

Related content

  • Jenkins Configuration as Code
    The move from Groovy scripts to Jenkins Configuration as Code simplifies the initialization of Jenkins and Jenkins plugins.
  • Verifying your configuration
    Automated acceptance testing is a powerful tool for catching problems related to misconfiguration. We'll show you how to implement your own acceptance testing environment with a free tool called goss.
  • Validating Docker containers
    A new test tool by Google lets you peek inside Docker containers, so you can make sure they hold exactly what you expect.
  • Policy rulesets in cloud-native environments
    What a user is allowed to do in a program is usually defined by a role model, which often poses numerous challenges, especially in the cloud or for infrastructure as code. The free Open Policy Agent offers a flexible way to manage user rights.
  • Incident response with Velociraptor
    The software incarnation of the feared predator in the Jurassic Park movies has been on the hunt for clues to cyberattacks and indicators of compromise. We show you how to tame the beast and use it for your own purposes.
comments powered by Disqus
Subscribe to our ADMIN Newsletters
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs



Support Our Work

ADMIN content is made possible with support from readers like you. Please consider contributing when you've found an article to be beneficial.