Moving HPC to the cloud

Scale Up

Article from ADMIN 11/2012

By Douglas Eadline

HPC has a unique set of requirements that might not fit into standard clouds. However, plenty of commercial options, including cloud-like services, provide the advantages of real HPC without the capital expense of buying hardware.

In the coming years, the cloud computing market is easily expected to exceed US$ 100 billion. Many organizations have found using the cloud has a clear advantage over in-house hardware. Indeed, shifting what was a capital expense to an operational expense has many advantages, including instant availability and the ability to scale up (and down) rapidly.

The advantage of pay-as-you-go computing has been an industry goal for many years. In HPC, the Globus project [1] has shown the power of grid-based computing and has spawned many successful production computation grids. Cloud computing takes grid computing to a whole new level by using virtualization to encapsulate an operating system (OS) instance. Users can construct their own operating system instance and run it in a cloud whenever they need computational resources. In addition to cloud computing, cloud storage also can be used independently or combined with OS instances.

Cloud computing would seem to be an HPC user's dream offering almost unlimited storage and instantly available and scalable computing resources, all at a reasonable metered cost. For all but the basic HPC applications, however, the use of a typical cloud needs a bit of due diligence because remote HPC services can range from shared HPC clusters to fully virtualized cloud environments.

Not All Clouds Look the Same

A "traditional" cloud offers features that are attractive to the general public. These services comprise single, loosely coupled instances (an instance of an OS running in a virtual environment) and storage systems backed by service level agreements (SLAs) that provide the end user guaranteed levels of service. These clouds offer the following features: