The YARN Invitation

Hadoop version 2 expands Hadoop beyond MapReduce and opens the door to MPI applications operating on large parallel data stores.

Why Isn’t Your Application Scaling?

Your parallel application is running fine, but you want it to run faster. Naturally, you use more and more cores, and everything is great; however, suddenly performance starts decreasing. What just happened?

The Road to End-of-Scale

The quest for exascale performance by the year 2020 is on. Delivering 1018 FLOPS could be slowed by the speed of light, among other things.

Getting Started with HPC Clusters

Getting started in the HPC world requires learning to write parallel applications and learning to administer and manage clusters. We take a look at some ways to get started.

Moving Your Data – It’s Not Always Pleasant

The world is swimming in data, and the pool is getting deeper at an alarming rate. At some point you will have to migrate data from one set of storage devices to another. Although it sounds easy, is it? We take a look at some tools that can help.

Is Hadoop the New HPC?

Hadoop has been growing clusters in data centers at a rapid pace. Is Hadoop the new corporate HPC?

The Cloud’s Role in HPC

Cloud computing is most definitely here – there are even commercials about it – but does it have a role in HPC? In this article, we discuss changes in HPC that could be solved effectively by cloud computing.

The RADOS Object Store and Ceph Filesystem: Part 3

We look at the new features in Ceph version 0.56, alias “Bobtail,” talk about who would benefit from CephX Ceph encryption, and show you how a Ceph Cluster can be used as a replacement for classic block storage in virtual environments.

Monitoring Client NFS Storage with nfsiostat

Previously we talked about using iostat to monitor local storage on your server or compute nodes, but what if you use NFS in your compute nodes to run jobs? The nfsiostat tool can help you understand the kinds of loads applications running on an NFS client put on the NFS storage server.

Monitoring Storage Devices with iostat

One tool you can use to monitor the performance of storage devices is iostat. In this article, we talk a bit about iostat, introduce a Python script that takes iostat data and creates an HTML report with charts, and look at a simple example of using iostat to examine storage device behavior while running IOzone.