Orcea David, 123RF

Orcea David, 123RF

TKperf – Customized performance testing for SSDs and HDDs

Stopwatch

Article from ADMIN 15/2013
By
SSD manufacturers try to impress customers with performance data. If you want to know more, why not try your own performance measurements with a standardized test suite that the free TKperf tool implements.

Solid State Drives (SSDs) have grown beyond their status as a niche product and now increasingly compete with traditional hard disk drives (HDDs) as an attractive option for server systems in terms of capacity and price. Whether SSDs really do boost performance depends on the particular application, because the requirements placed on the storage systems can differ fundamentally. Users must first decide on SSDs, HDDs, or a combination of the two technologies. Once you've decided which medium to use, you still need to choose an appropriate model.

Test for Yourself

To begin, you will want to make sure that the speed of the selected hardware meets your current and future performance requirements. Meaningful and transparent performance tests considerably facilitate your search for the right SSD or HDD. From the perspective of the end user, the more information a test delivers, the more valuable it is. Often the manufacturer's performance specifications are not authoritative, because neither the software used nor the test method is specified.

The Storage Networking Industry Association (SNIA) has released the "Solid State Storage Performance Test Specification," a document that provides a specific test description for SSDs in the enterprise space [1]. In its tests, the specification specifically targets the unique properties of SSDs and describes the approach to achieving accurate and reproducible results. TKperf, an open source tool developed by Thomas-Krenn.AG, implements this specification and prepares the results in a test report [2]. In addition to testing for SSDs, some initial basic tests for hard disks are also available; they will be expanded in future versions of TKperf.

Fio and Python

In the background, TKperf uses the Flexible I/O Tester (Fio) [3] developed by Jens Axboe, the maintainer of the Linux block layer. Fio offers many options and the flexibility to implement the SNIA performance tests: Features include run-time limits, direct I/O, number of parallel jobs, I/O depth, or the use of non-compressible test data.

TKperf requires Fio 2.0.3 or higher. In older versions of Fio, some terse output information is missing; this is minimal output that can be easily processed by other programs [4]. Distributions like Ubuntu 12.04 or Debian squeeze have older Fio versions in their repositories (v1.59 or v1.38), but Fio can be compiled easily from the source code in the Git repository  [5].

Other components of TKperf rely on the hdparm utility and Python. Hdparm uses hdparm -I to provide automated information about a device under test. The tool can also perform a secure erase on SATA drives. For SSDs, this means that all flash chips are erased, thus restoring the SSD's as-received condition performance. Secure erase, which is a key part of the initialization phase of the tests, is also the reason why some manual attention is still needed when testing SAS or PCIe SSDs.

For SAS devices, you need to modify the sg_format secure erase method in sg3_utils (the tools for executing SCSI commands). Secure erase is even more complicated for PCIe SSDs because they require vendor-specific, command-line tools. For Intel's PCIe SSD 910, for example, you need to launch secure erase via the isdct Datacenter tool.

The Python scripts in TKperf start the Fio jobs, prepare the results, and generate charts. All Fio calls are recorded in a logfile. At the end of a test, the test results are also logged. The logged performance values are stored in XML format and used to create the charts. With the help of the Python Matplotlib library, TKperf generates the graphical representations of the results required by the specification  [6].

TKperf Test Scenarios

A bandwidth test, an input/output operations per second (IOPS) test, a latency test, and a saturation test are all available for SSDs. Listing 1 shows the output of TKperf during testing of an Intel DC S3700 200GB SSD.

Listing 1

TKperf Result Set

$ sudo tkperf ssd intelDCS3700 /dev/sdb -nj 2 -iod 16 -rfb
!!!Attention!!!
All data on /dev/sdf will be lost!
Are you sure you want to continue? (In case you really know what you are doing.)
Press 'y' to continue, any key to stop:
y
Starting SSD mode...
Model Number:       INTEL SSDSC2BA200G3
Serial Number:      BTTV241303MG200GGN
Firmware Revision:  5DV10206
Media Serial Num:
Media Manufacturer:
device size with M = 1000*1000:      200049 MBytes (200 GB)
Starting test: lat
Starting test: iops
Starting test: writesat
Starting test: tp

The bandwidth, IOPS, and latency tests show some similarities that are reflected in the graphics. All of these tests are performed in rounds, with measurements taken in each round. Taking into account a dependent variable, the test calculates whether the SSD is in a stable state. This approach avoids influencing the measurements in a "fresh out of the box" state or undefined transitions between different workloads. Only if the values of the dependent variable are stable over the last several rounds the values of this measurement window are incorporated into the test results.

Buy ADMIN Magazine

SINGLE ISSUES
 
SUBSCRIPTIONS
 
TABLET & SMARTPHONE APPS
Get it on Google Play

US / Canada

Get it on Google Play

UK / Australia

Related content

  • Tuning SSD RAID for optimal performance
    Hardware RAID controllers are optimized for the I/O characteristics of hard disks; however, the different characteristics of SSDs require optimized RAID controllers and RAID settings.
  • Optimizing Windows 10 for SSDs
    Not all PC settings – traditionally optimized for magnetic hard disks – are suitable for solid state drives. We show you how to configure Windows 10 for optimal operation with SSDs.
  • SDS configuration and performance
    Software-defined storage promises centrally managed, heterogeneous storage with built-in redundancy. We examine how complicated it is to set up the necessary distributed filesystems. A benchmark shows which operations each system is best at tackling.
  • Fundamentals of I/O benchmarking
    Admins often want to know how to measure the performance of a specific solution. Care is needed, however: Where there are benchmarks, there are pitfalls.
  • Predicting Drive Doom

    Modern drives use S.M.A.R.T. (self-monitoring, analysis, and reporting technology) to gather information and run self-tests. Smartmontools is a Linux tool for interacting with the S.M.A.R.T. features of drives. But is S.M.A.R.T. the best way to predict impending drive doom?

comments powered by Disqus