Lead Image © astragal, 123RF.com

Lead Image © astragal, 123RF.com

Automate your VMware configuration with Puppet

Steer the Sphere

Article from ADMIN 32/2016
In ESXi environments, the powerful Puppet automatic configuration tool can perform its services and roll out VMs automatically. A test environment shows the benefits of Puppet in conjunction with vSphere.

In addition to supporting hypervisors such as Amazon EC2 and Google GCE, Puppet lends itself to the use of VMware vSphere. A private cloud created with Puppet and vSphere is superior to public clouds in two respects: In addition to far lower total cost of ownership, given 24/7 use, you also benefit from increased data security with local hosting. In this article, I target IT managers with Puppet experience who thus far have not looked into the use of the product's cloud service modules.

Test Infrastructure

Establishing a complex VMware infrastructure requires huge hardware investments, so instead, I used a VMware Workstation as the basis of my setup. To understand the process, though, you need a workstation with at least eight cores, 16GB of RAM, and 100GB of hard disk storage space. The host operating system is Windows 8 for simplicity's sake – a 64-bit version is mandatory – and VMware Workstation should be installed as usual.

Puppet networks can be controlled by a master, which must be a Linux system. The first step is thus to set up a virtual machine with Ubuntu. Be frugal with the hardware resources; ESX just loves burning up the remaining computing power for virtual machines and the vCenter appliance.

In the next step, log in to your Puppet master system and download the tarball with Puppet Enterprise [1]. After starting the installation wizard, you can accept all the defaults, because you are generating a test installation only.

While the installation process is running, the Ubuntu host needs additional resources. Even though only 3GB of RAM and two processor cores are required, the installation takes two to three hours. After the installation, if you see a warning regarding the MCollective package, you will have to install it manually. Because of a known bug, you also must install the STOMP library before the actual deployment can take place:

sudo apt-get install ruby-stomp
sudo apt-get install mcollective

Installing the master does not automatically enable Puppet Enterprise to interact with the cloud: For some time, the necessary packages have not been enabled by the setup process. To resolve this issue, you need to open a command prompt and install the missing components manually:

cd /puppet-enterprise-3.8.1-ubuntu-14.04-amd64/packages/ubuntu-14.04-amd64
sudo dpkg -i pe-cloud-provisioner-libs_0.3.2-1puppet1_amd64.deb
sudo dpkg -i pe-cloud-provisioner_1.2.0-1puppet1_all.deb

vSphere Setup

After installing Ubuntu, it's time to let ESX out of its cage. Download the VMware vSphere image [2] and configure a new virtual machine. The successful use of ESX requires pass-through virtualization, the existence of at least two processor cores, and at least 2GB of RAM. For the vCenter server, required for communication between Puppet and ESX, you need 10GB of RAM; this can be reduced later if necessary.

The details of the ESX installation are not critical; simply follow the instructions on the screen. The only difficulty relates to the strange command schema. Some dialogs can be cleared off the screen just by pressing the F key.

Once the two basic components are on your system, you need to wire the Puppet master to the ESX host that is responsible for management of the virtual machines. Communication between Puppet and ESX runs through a vCenter server. Download the VMware vCenter Server Appliance [3] and mount the ISO file. Because the installation of a standalone vCenter server requires the presence of a Windows Server operating system, you need to move the administration instance to the ESX server instead. To do this, you need the VMware-VCSA-all ISO image [3], which you can mount on a Windows 8.1 host by double clicking.

In the next step, you simply navigate to the VCSA subfolder and install VMware Client Integration Plugin when you get there. Then, open the vcsa-setup.html file in a browser of your choice. I used Chrome, which proved very cooperative in this scenario. If you get security prompts, select the option Launch Application and press Install on the web page that appears.

After agreeing to the EULA, the installation wizard prompts you, among other things, for the IP address and the password of the ESXi server intended for use as the host. Although P@ssw0rd met all the conditions imposed by the validator, the installation wizard was annoyed but accepted it. The Embedded Platform Services Controller serves as the deployment type, the new SSO domain goes by the name vsphere.local. Use P@ssw0rd for the administrator account, too.

Because the installation wizard still requires 8GB of RAM on the ESX host, even if you choose an appliance of the Tiny type, you should stop all the VMs and expand the ESX instance's RAM to 10GB at this point. In the actual deployment, make sure you select the Enable Thin Disk Mode option, too, because the virtual instance of vSphere otherwise allocates all the space right from the outset. Use the ESXi host for time synchronization, and click on Finish to start the deployment.

After successfully processing the deployment, the VM is available from the URL mentioned on the Installation Complete page; in my case, this was . To log in, use the username administrator@vsphere.local .

Unfortunately, your instance knows nothing about its host; therefore, you need a new data center as the first step. To do this, go to the vCenter Inventory Lists | Datacenters section and create a new entry by clicking the plus icon. In the documentation, the Puppet developers explicitly point out that the data center must reside at the top level of the hierarchy, which means that the vCenter server must be the parent object.

The host can move in during the next step. Click on vCenter Inventory Lists | Hosts to open the dialog box where you can enter a new host by clicking the plus icon. Enter the IP address of the ESXi host and add it to the newly created data center. After a few seconds of computing time, the VM generated by the installation wizard appears in the vSphere web client.

Puppet Configuration

The Puppet Cloud Provisioner module from version 3.8 onward is based on the fog library [4]. Its configuration is located in the root directory of the user responsible for provisioning; to create it, you need to type touch ~/.fog. Fog will accept a variety different parameters. For VMware, it is sufficient to add the following five lines to the .fog file using Gedit:

  :vsphere_username: administrator@vsphere.local
  :vsphere_password: P@ssw0rd
  :vsphere_expected_pubkey_hash: XXX

The statement starting with :default: stipulates that the subsequent configuration is used as the default. If you want to connect your Puppet Master instance to multiple cloud servers, then you can create multiple configurations. To switch between them, in this case, you need to enter:

FOG_CREDENTIAL=default puppet node_VMware <optional commands>

Hashes are used for identification. Unfortunately, it is possible to talk a vCenter server into divulging its hashes. For this reason, I went for a workaround and set XXX as the hash then ran an idempotent command against the server (Listing 1). Puppet responded with an error message, indicating that the hashes did not match. The hash supplied by the server was copied to the FOG file, thus completing the wiring work for Puppet and ESX. For more information on the Warning message in Listing 1, see the "Deprecated" box.


The official documentation of the Cloud Provisioner module marks it as deprecated in the current version of Puppet Enterprise [5]. Fortunately, the developers are not entirely serious. The reason the module is no longer installed by default reflects a change toward managing the individual cloud-provisioning modules in Puppet independently. With interaction via a plugin interface, the developers of plugins and Puppet would no longer need to rely on one another.

At the moment, an AWS plugin is available based on the new architecture. A Puppet Labs employee confirmed on a public mailing list that users of the cloud provisioning module do not need to worry at the moment.

The extension for VMware is not yet available, which is why the Fog-based Cloud Provisioner will still be used for some time. Also, you will receive ample notice before the old cloud provisioning system is discontinued so you have enough time to convert to the new system.

Listing 1

Getting Hash to FOG File

puppet node_VMware list
Warning: Cloud Provisioner is deprecated in PE 3.8.
         For more information and recommendations,
         see the release notes documentation here:
Notice: Connecting ...
Error: The remote system presented a public key with
       hash 31452e1f896 f71542b6b9198188de1b5e59f5af62ffcefdc261df324636c90c7
       but we're expecting a hash of XXX. If you are sure
       the remote system is authentic set
       vsphere_expected_ pubkey_hash: <the hash printed
       in this message> in ~/.fog
Error: Try 'puppet help node_VMware list' for usage

Buy this article as PDF

Express-Checkout as PDF
Price $2.95
(incl. VAT)

Buy ADMIN Magazine

Get it on Google Play

US / Canada

Get it on Google Play

UK / Australia

Related content

  • VMware Tools

    VMware’s recent announcement about vCloud Hybrid Service involves expansion of the vCloud suite toward a public/hybrid cloud. In this article, we attempt to shed some light on the huge number of VMware products.

  • An overview of VMware tools
    VMware's recent announcement about vCloud Hybrid Service involves expansion of the vCloud suite toward a public/hybrid cloud. In this article, we attempt to shed some light on the huge number of VMware products.
  • VMware vRealize Automation 7
    We look at VMware's tool for managing and provisioning cloud infrastructures.
  • New Features in PowerCLI 6.0 R3
    PowerCLI in the current version 6 R3 supports access to nearly all VMware data center and cloud products with around 500 cmdlets, including numerous additional features, APIs, and interfaces. In this article, we show you the most important innovations.
  • Easy configuration management with Puppet
    If you really want your evenings to belong to your job, you don't need to depend on configuration management. But is all your overtime really necessary just to configure a server system?
comments powered by Disqus