Tuning your filesystem's cache

Tune-Up

Article from ADMIN 07/2012

By Federico Lucifredi

Keeping your key files in RAM reduces latency and makes response time more predictable.

Anyone who has spent time tuning the performance of a system is keenly aware of the bottlenecks existing between permanent storage on disk and working storage in RAM. As a rule, one should think of a hard drive as being three orders of magnitude slower than random-access memory. Because of this disparity, modern operating systems are designed to cache the reads and buffer the writes of the storage subsystem.

In Linux, the kernel makes generous allocation of all memory not used by any program for behind-the-scene purposes; I focus on the disk page cache [1] here. If system load increases at the start of a new application, the system will evict caches as required to provide working memory for the new process – the illusion of free memory is preserved, even though those chips were most likely hard at work for the kernel all along.

The disk page cache uses memory pages in RAM to access rapidly data stored in disk blocks, with the kernel using "temporal locality" as the organizing principle to determine what to cache. The assumption that recently accessed blocks of data are more likely to be accessed again drives the kernel's preferences. Given that on most systems not all permanent storage can be cached simultaneously, it makes sense to optimize the disk cache according to your workload. For example, you can ensure a uniform performance experience for clients by pre-caching the directory tree yourself, rather than letting page cache misses cause the first load.

Enter Amazon EC2: If you often launch new instances, those new VMs will initially exhibit this uneven performance behavior. Elastic Block Storage (EBS) volumes are far from high-performance IOPS devices, making the disparity more obvious. The team behind the smartphone app Instagram illustrated this in the story of their successful EC2 scaling experience [2].

...

Use Express-Checkout link below to read the full article (PDF).