Hadoop is a large and complex software framework involving a number of components interacting with each other across multiple hardware systems. Bottlenecks in a subset of the hardware systems within the cluster can cause overall poor performance of the underlying Hadoop workload.
In this tuning guide, we attempt to provide the audience with a holistic approach of Hadoop performance tuning methodologies and best practices. Using these methodologies we have been able to achieve as much as 5.6X performance improvements. We discuss hardware as well as software tuning techniques including OS, JVM and Hadoop configuration parameters tuning.
Panasas ActiveStor running the PanFS filesystem is an ideal storage solution for customers with varied big data workloads.
AMD FirePro S10000 is the World’s First Professional Graphics Card to Exceed One TeraFLOPS of Peak Double Precision Performance and Unparalleled Single Precision Performance.
Oak Ridge National Laboratory’s “Titan” Supercomputer Enables Cutting-Edge Research for Vital Science and Technology Disciplines, Including Energy and Climate Change.
Formula racecar engineers study destabilizing wake that interferes with high-speed passing maneuvers.
The SGI ICE 8400 blade system helps minimize overhead and communication bottlenecks that can rob efficiency and scalability, especially for data-intensive workflows.
Scientists help automate the search for hurricanes in huge datasets
Cray XK6 system helps scientists accelerate research into soft matter
Materials modeling on ORNL’S “Jaguar” shows big future for boron nitride.
Researchers use Cray XT “Jaguar” supercomputer to understand solar storms.