Building Big Iron in the Cloud with Google Compute Engine

Iron Ore

Cloud SDK

The Google Cloud SDK is a set of tools and libraries to create and manage your Google Cloud. It supports App Engine, Compute Engine, Cloud Storage, BigQuery, Cloud SQL, and Cloud DNS. Before going further, you must meet the following Cloud SDK requirements:

  • Python 2.7.x
  • Java 1.7+ (for App Engine)
  • A supported OS: Windows (requires Cygwin [6]), Mac OS X, Linux

To set up Gcutil [7], you must download and install the Google Cloud SDK. On the Linux distro of your choice, enter the commands

$ curl | bash
$ unzip
$ ./google-cloud-sdk/
$ gcloud auth login

to transfer the SDK to your machine, unzip the file, run the installation script, and authenticate to the Google Cloud.

Authentication with OAuth2

Google Compute Engine uses the OAuth2 standard for authentication and authorization to access the Google Cloud. OAuth allows users to share data with your website or application while keeping their username and password – and other sensitive information – private.

With a Cloud SDK and authentication, you can now SSH into your new instance. As you see (Listing 1), Google Cloud SDK sets up key-based authentication and takes you right into the instance specified in the gcutil command: gcerocks-instance1.

Listing 1

SSH into an Instance

01 $ gcutil ssh gcerocks-instance-1
03 joe@m0nk3y:~/google-cloud-sdk$ gcutil ssh gcerocks-instance-1
04 INFO: Zone for gcerocks-instance-1 detected as us-central1-b.
05 WARNING: You don't have an ssh key for Google Compute Engine. Creating one now...
06 Enter passphrase (empty for no passphrase):
07 Enter same passphrase again:
08 INFO: Updated project with new ssh key. It can take some time for the instance to pick up the key.
09 INFO: Waiting 10 seconds before attempting to connect.
10 INFO: Running command line: ssh -o UserKnownHostsFile=/dev/null -o CheckHostIP=no -o StrictHostKeyChecking=no -i /home/joe/.ssh/google_compute_engine -A -p 22 joe@ --
11 Warning: Permanently added '' (ECDSA) to the list of known hosts.
12 Enter passphrase for key '/home/joe/.ssh/google_compute_engine':
13 Linux gcerocks-instance-1 3.2.0-4-amd64 #1 SMP Debian 3.2.54-2 x86_64

Note that it is always good practice to put in a strong passphrase when asked to do so. Never leave it blank. Also mind the security of the local machine you use to manage your Google Cloud.

With a Cloud SDK set up, you have a range of utilities to manage your cloud (Table 3). If you use Gcutil standalone, it automates the setup of key-based authentication for SSH access to your instance. Gcutil uploads and creates a public/private key and uploads your public key to the cloud. Finally, it associates the key with your Google account, giving you access to any instance you create. As always, setting up Gcutil with key-based authentication is helpful but means little if you fail to add a strong passphrase to protect your key and lock down your local machine.

Table 3

Google Cloud Utilities

Utility Function
appcfg Deploy and manage Google App Engine.
gcloud Manage cloud resources (e.g., authentication, configuration) and workflow [8].
gcloud sql Manage Google Cloud SQL.
gcutil Manage Google Compute Engine. Just as from the web console, you can manage from the CLI [9]. A few examples of how to use this tool are:
   gcutil version Show current version of Gcutil
   gcutil addinstance <instancename> Add an Google Compute Engine instance of specified name.
   gcutil deleteinstance <instancename> Remove a GCE instance.
   gcutil listinstances List current GCE instances.
   gcutil help List all available commands.
gsutil Manage Google Cloud storage.
Gcutil standalone (deprecated) Gcutil is the central tool used to manage your Google Compute Engine, but it was once distributed as a standalone tool. As of late, Google is encouraging the use of the Google Cloud SDK over the previous standalone Gcutil utility to consolidate development tools under one suite of tools [10].

Firewall in the Cloud

Next, you need to set up your cloud instance by configuring a firewall and adding persistent storage. All new instances by default block all external traffic, which is a smart security move from Google; default deny is always a good idea. To make the services you install available, you need to open up the firewall rules to that newly created instance.

To create a new firewall rule, click Networks, choose the default network (created with this instance), and go to Firewall | Create a new Firewall. Where you see default rules, click Create new. For example, Figure 3 shows an Nginx web server with HTTP on port 80 and HTTPS (SSL/TLS) on 443.

Figure 3: Creating a new firewall rule.

Buy this article as PDF

Express-Checkout as PDF
Price $2.95
(incl. VAT)

Buy ADMIN Magazine

Get it on Google Play

US / Canada

Get it on Google Play

UK / Australia

Related content

  • Moving HPC to the Cloud

    HPC has a unique set of requirements that might not fit into standard clouds. However, plenty of commercial options, including cloud-like services, provide the advantages of real HPC without the capital expense of buying hardware.

  • Interview: Hardware or Cloudware?

    Altair makes software for local high-performance computing systems and also provides HPC services through the cloud. We asked Bill Nitzberg, CTO of Altair’s PBS Works division, about the changing market and the relative benefits of cloud versus local HPC.

  • The Cloud’s Role in HPC

    Cloud computing is most definitely here – there are even commercials about it – but does it have a role in HPC? In this article, we discuss changes in HPC that could be solved effectively by cloud computing.

  • StarCluster Toolkit: Virtualization Meets HPC

    Cloud computing has become a viable option for high-performance computing. In this article, we discuss the use case for cloud-based HPC, introduce the StarCluster toolkit, and show how to build a custom machine image for compute nodes.

  • Desktop Supercomputers: Past, Present, and Future

    Desktop supercomputers give individual users control over compute power to run applications locally at will.

comments powered by Disqus