Hybrid public/private cloud


Protecting Data

The second major security concern relates to storing data. Especially when processing personal information for members of the European Union (EU), you have to be careful for legal reasons about which of the cloud provider's regions is used to store the data. Relocating the customer database to Japan might turn out to be a less than brilliant idea. Even if the data is stored on servers within the EU, the question of who gets access still needs to be clarified.

Encrypting data in AWS is possible [5]. If you do not have confidence in your abilities, you could equip a Linux VM with a self-encrypted volume (e.g., LUKS [6]) and not store the password on the server. With AWS, this does not work for system disks, but it does at least for data volumes. After starting the VM, you have to send the password. This process can be automated from your own data center. The only possible route of access for the provider is to read the machine RAM; this risk exists where modern hardware enables live encryption, as well.

As a last resort, you can ensure that the computing resources in the cloud only access data managed by the local data center. However, you will need a powerful Internet connection.

Solving a Problem

Assume you have a local Elasticsearch cluster of three nodes: a master node, which also houses Logstash and Kibana, and two data nodes with data on board (Figure 1).

Figure 1: A secure network architecture (here the rough structure) should connect the nodes in the local data center with those on AWS.

You now want to provide this cluster temporarily two more data nodes in the public cloud. You could have several reasons for this; for example, you might want to replace the physical data nodes because of hardware problems, or you might temporarily need higher performance for data analysis. Because it is not typically worthwhile to procure new hardware on a temporary basis, the public cloud is a solution. The logic is shown in Figure 1; the machines started there must become part of the Elastic cluster.

The following explanations assume you have already written Ansible roles for installing the Elasticsearch-Logstash-Kibana (ELK) cluster. You will find a listing for a Playbook on the ADMIN FTP site [7]. Thanks to the structure of these roles, you can add more nodes by appending parameters to the Hosts file, and it includes installing the software on the node.

The roles that Ansible calls are determined by the Hosts file (usually in /etc/ansible/hosts) and the variables set in it for each host. Listing 1 shows the original file.

Listing 1

ELK Stack Hosts File ansible_ssh_user=root logstash=1 kibana=1 masternode=1 grafana=1 do_ela=1 ansible_ssh_user=root masternode=0 do_ela=1 ansible_ssh_user=root masternode=0 do_ela=1

Host is the master node on which all software runs. The other two hosts are the data nodes of the cluster. The variable do_ela controls whether the Elasticsearch role can perform installations. When expanding the cluster, this ensures that Ansible does not reconfigure the existing nodes – but more about the details later.

Extending the Cluster in AWS

The virtual infrastructure in AWS comprises a VPC with two subnets. One subnet can be reached from the Internet; the other represents the internal area, which also contains the two servers on which data nodes 3 and 4 are to run. In between is a virtual firewall, by Fortinet in this case, that terminates the VPN tunnel and controls access with firewall rules.

This setup requires several configuration steps in AWS: You need to create the VPC with a main network. On this, you then assign all the subnets: one internal (inside) and one accessible from the Internet (outside). Then, you create an Internet gateway in the outside subnet. Through this, the data traffic migrating toward the Internet finds an exit from the cloud. For this purpose, you define a routing table for the outside subnet that specifies this Internet gateway as the standard route (Figure 2).

Figure 2: A firewall separates an external subnet and an internal subnet (right). In detail, the AWS connection looks slightly different.

Buy this article as PDF

Express-Checkout as PDF
Price $2.95
(incl. VAT)

Buy ADMIN Magazine

Get it on Google Play

US / Canada

Get it on Google Play

UK / Australia

Related content

  • Roll out hybrid clouds with Ansible  automation
    Designing your own hybrid IT structure as a digital mix of your servers and public or private clouds might be technically elegant and cost effective, but setup is time consuming. Thanks to Ansible, it might take less work than you think.
  • Setting up DevOps Orchestration Platform
    DevOps Orchestration Platform open source framework was developed in Golang and can be used to bootstrap an IT infrastructure dynamically or import details of an existing IT infrastructure locally on VirtualBox or in the Cloud.
  • Jira, Confluence, and GitLab
    Jira, Confluence, and GitLab are very popular DevOps tools and often form the basis for agile work flows. With the right Ansible playbooks, Ubuntu can be turned into an agile work center.
  • What's new in Ansible 2.0
    The new version of the Ansible IT automation platform has a revised and faster engine and 200 new modules.
  • Keeping it simple – the Ansible automator
    The powerful Ansible automator comes without airs and graces, does without complex syntax, self-documents, and has no need to hide its light compared with Puppet and other configuration management tools.
comments powered by Disqus
Subscribe to our ADMIN Newsletters
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs

Support Our Work

ADMIN content is made possible with support from readers like you. Please consider contributing when you've found an article to be beneficial.

Learn More”>


		<div class=