In the coming years, the cloud computing market is easily expected to exceed US$ 100 billion. Many organizations have found using the cloud has a clear advantage over in-house hardware. Indeed, shifting what was a capital expense to an operational expense has many advantages, including instant availability and the ability to scale up (and down) rapidly.

The advantage of pay-as-you-go computing has been an industry goal for many years. In HPC, the Globus project [1] has shown the power of grid-based computing and has spawned many successful production computation grids. Cloud computing takes grid computing to a whole new level by using virtualization to encapsulate an operating system (OS) instance. Users can construct their own operating system instance and run it in a cloud whenever they need computational resources. In addition to cloud computing, cloud storage also can be used independently or combined with OS instances.

Cloud computing would seem to be an HPC user's dream offering almost unlimited storage and instantly available and scalable computing resources, all at a reasonable metered cost. For all but the basic HPC applications, however, the use of a typical cloud needs a bit of due diligence because remote HPC services can range from shared HPC clusters to fully virtualized cloud environments.

Not All Clouds Look the Same

A "traditional" cloud offers features that are attractive to the general public. These services comprise single, loosely coupled instances (an instance of an OS running in a virtual environment) and storage systems backed by service level agreements (SLAs) that provide the end user guaranteed levels of service. These clouds offer the following features:

• Instant availability – Cloud offers almost instant availability of resources.
• Large capacity – Users can instantly scale the number of applications within the cloud.
• Software choice – Users can design instances to suit their needs from the OS up.
• Virtualized – Instances can be easily moved to and from similar clouds.
• Service level performance – Users are guaranteed a certain minimal level of performance.

Although these features serve much of the market, HPC users generally have a different set of requirements:

• Close to the "metal" – Many man-years have been invested in optimizing HPC libraries and applications to work closely with the hardware, thus requiring specific OS drivers and hardware support.
• Userspace communication – HPC user applications often need to bypass the OS kernel and communicate directly with remote user processes.
• Tuned hardware – HPC hardware is often selected on the basis of communication, memory, and processor speed for a given application set.
• Tuned storage – HPC storage is often designed for a specific application set and user base.
• Batch scheduling – All HPC systems use a batch scheduler to share limited resources.

Depending on the user's application domain, these two feature sets can make a big difference in performance. For example, applications that require a single node (or threaded applications) can work in a cloud. In this case, the user might have a single program that must be run with a wide range of input parameters (often called parametric processing), or they might have dataflow jobs, such as the Galaxy suite [2] used in biomedical research. These types of applications can benefit from most cloud computing resources.

Some applications can utilize highly parallel systems but do not require a high-performance interconnect or fast storage. One often cited example is digital rendering, in which many non-interacting jobs can be spawned across a large number of nodes with almost perfect scalability. These applications often work well with standard Ethernet and do not require a specialized interconnect for high performance.

Moving up the HPC tree, you'll find interconnect-sensitive applications that require low latency and high throughput interconnects not found in the traditional cloud. Indeed, most of these interconnects (e.g., InfiniBand and High-Performance Ethernet) require "userspace" communication pathways that do not involve the OS kernel. This method makes the use of cloud virtualization very difficult because most virtualization schemes cannot manage "kernel by-pass" applications (i.e., these are "on the wire" data transfers that are hard to virtualize). If high-performance networks are not available, many HPC applications run slowly and suffer from poor scalability (i.e., they see no performance gain when adding nodes).

Also in the tree are many I/O-sensitive applications that, without a very fast I/O subsystem, will run slowly because of storage bottlenecks. To open up these bottlenecks, most HPC systems employ parallel filesystems that drastically increase the I/O bandwidth of computing nodes.

Another growing branch includes performance accelerators or SIMD units (parallel computing Single-Instruction Multiple Data processors) from NVidia and AMD/ATI. This type of hardware is very specific to HPC systems and therefore is not found on typical cloud hardware.

At the top of the tree are applications that push on all levels of performance (compute, interconnect, and storage). These applications require fast computation (possible a SIMD unit), fast interconnects, and high-performance storage. Clearly, this computing environment is not found in a typical cloud and is unique to the HPC market. Attempting to run this level of application on a typical cloud will provide sub-par performance. A deeper treatment of these issues can be found in a recent article called "Will HPC Work in the Cloud?" [3].

Finally, any remote computation scheme needs to address the "moving big data problem." Many HPC applications require large amounts of data. Many clouds, even those that offer HPC features, cannot solve the problem easily. In particular, if the time to move large datasets to the cloud outweighs the computation time, then the cloud solution is now the slow solution. Interestingly, the fastest way to move data in these cases is with a hard disk and an overnight courier. (It seems the station wagon full of tapes is still the fastest way to transport data.)

The HPC Cloud Is Out There

With all the differences between the traditional cloud and HPC applications, users will be interested to know that HPC clouds and cloud-like resources are available. A number of companies, including Penguin, R-HPC, Amazon, Univa, SGI, Sabalcore, and Gompute offer specialized HPC clouds. Notably absent is IBM who, at this time, does not offer public HPC clouds. The company, however, does provide many options for constructing internal or private clouds.

As with many performance-based systems, the devil is often in the details and the word "cloud" (like the word "grid" in the past) can mean almost anything when it comes to performance. To provide a sense of the current HPC cloud capabilities, the previously mentioned companies were contacted and asked to answer some very specific HPC questions about their cloud offerings. The questions covered several categories including:

• System setup
• Interconnects
• Storage
• Workflow management
• Approximate cost

Note that the survey did not include academic clouds, such as those in the Future Grid project. These clouds are increasing, but access, cost, and capabilities can vary and might require research proposals. As with any new HPC project, due diligence will produce the best results. The following survey of HPC cloud vendors will help get you started.

Penguin Computing

One the first vendors to introduce a true HPC cloud was Penguin Computing. The Penguin On Demand [4], or POD cloud, was one of the first remote HPC services. From the beginning, POD has been a bare-metal compute model similar to an in-house cluster. Each user is given a virtualized login node that does not play a role in code execution. The standard compute node has a range of options, including dual four-core Xeon, dual six-core Xeon, or quad 12-core AMD processors ranging in speed from 2.2 to 2.9GHz with 24 to 128GB of RAM per server and up to 1TB of scratch local storage per node.

Getting applications running POD HPC clouds can be quite simple, because Penguin has more than 150 commercial and open source applications installed and ready to run on the system. Installing other applications is straightforward and available to users. Nodes with two NVidia Tesla C2075 computing processors are available.

In terms of network, POD nodes are connected via nonvirtualized, low-latency 10Gb Ethernet (GigE) or QDR IB networks. The network topology is local to ensure maximum bandwidth and minimum latency between nodes. Storage systems are made available via 10GigE to the local compute cluster. Additionally, POD has redundant high-speed Internet with remote connectivity ranging from 50Mbps to 1Gbps.

Several storage options are also available, starting with high-speed NFS using 10GigE attached storage. List pricing for storage on POD is US$ .10/GB per month. Beyond NFS, there are parallel filesystem options attached via multiple 10GigE links and InfiniBand. Lustre and Panasas high-performance storage systems also can be provided. Finally, dedicated storage servers are available. These systems can isolate data and facilitate encryption/decryption of high volumes of data by using physical shipping rather than Internet transfer.

POD offers a suite of tools to help manage your computation. Aptly called PODTools, Penguin offers a collection of command-line utilities for interacting with their HPC cloud. Beyond the standard SSH login, PODTools provide the ability to submit jobs, transfer data, and generate reports. Additionally, Penguin POD can be seamlessly integrated into existing on-site clusters to outsource excess workloads – often known as "cloud bursting." All these capabilities are encrypted and offer a high level of security.

In terms of cost, Penguin has an online pricing estimator [5]. I proposed two usage cases. The first was for a small case consisting of 80 cores with 4GB of RAM per core with basic storage of 500GB. POD pricing is based on cores/hour and would work out to be US$ 6,098.00/month or US$ .101/core per hour. A large example of 256 cores with 4GB of RAM per core and 1TB of parallel storage would cost US$ 18,245.00/month with the same US$ .101/core per hour.

Perhaps Penguin's biggest asset is a long history of delivering on-site HPC solutions. This experience has allowed them to develop a staff of industry domain experts. They also have long list of additional services that supplement their POD offering. These include on-premises provision of cloud bursting to the POD cloud, remote management of on-premises HPC services, cloud migration services, private remote HPC as service environments, and private internal clouds.