Hadoop has been growing clusters in data centers at a rapid pace. Is Hadoop the new corporate HPC?

Adding and subtracting software from a running cluster can be tricky; however, many application packages can be added or removed easily with a few tools and some simple tricks.

Cloud computing has become a viable option for high-performance computing. In this article, we discuss the use case for cloud-based HPC, introduce the StarCluster toolkit, and show how to build a custom machine image for compute nodes.

As core counts increase in both CPUs and GPUs, the HPC lifestyle could become a bit more difficult.

Parallel programming is not easy, but one tool you can use to help parallelize your application is OpenMP. Most compilers are compatible with OpenMP and allow you to parallelize your code on a single node.

The Modules package makes life easier in the HPC world (and beyond).

HPC systems are really designed to be shared by several users. One way to share them is through a software tool called a resource manager. Openlava is an open source version of the commercial scheduler LSF. It shares the robustness of LSF while being freely available, very scalable, and easy to install and customize.

Gathering data on various aspects of your HPC system is a key step toward developing information about the system and one of the first steps toward tuning your system for performance and reporting on system use. It can tell how users are using the system and, at a high level, what they are doing. In this article, I present a method for gathering data on how users are using Environment Modules, such as which modules are being used, how often, and so on.

The Julia language is a very powerful parallel computing model that works across multiple cores and cluster nodes.

Installing and configuring Warewulf on the master node and booting the compute nodes creates a basic cluster installation; however, a little more configuration to the master remains and a few other tools must be installed and configured for the Warewulf cluster to become truly useful for running HPC applications.