Linux File System Read Write Performance Test

System administrators responsible for handling Linux servers get confused at times when they are told to benchmark a file system's performance. But the main reason that this confusion happens is because, it does not matter whatever tool you use to test the file system's performance, what matter's is the exact requirement.File system's performance depends upon certain factors as follows.

• The maximum rotational speed of your hard disk
• The Allocated block size of a file system
• Seek Time
• The performance rate of the file system's metadata
• The type of read/Write

Seriously speaking its wonderful to realize that various different technologies made by different people and even different companies are working together in coordination inside a single box, and we call that box a computer. And its even more wonderful to realize that hard disk's store's almost all the information available in the world in digital format. Its a very complex thing to understand how really hard disks stores our data safely. Explaining different aspects of how a hard disk, and a file system on top of it, work together is beyond the scope of this article(But i will surely give it a try with couple of my posts about them)

So Lets begin our tutorial on file system benchmark test.

Its advised that during this file system performance test, you must not run any other disk I/O intensive tasks. Otherwise your results about performance will be heavily deviated. Its better to stop all other process during this test.

The Simplest Performance Test Using dd command

The simplest read write performance test in Linux can be done with the help of dd command. This command is used to write or read from any block device in Linux. And you can do a lot of stuff with this command. The main plus point with this command, is that its readily available in almost all distributions out of the box. And is pretty easy to use.

With this dd command we will only be testing sequential read and sequential write.I will test the speed of my partition /dev/sda1 which is mounted on "/" (the only partition i have on my system)so can write the data to any where in my filesystem to test.

[root@slashroot2 ~]# dd if=/dev/zero of=speetest bs=1M count=100
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 0.0897865 seconds, 1.2 GB/s
[root@slashroot2 ~]

In the above command you will be amazed to see that you have got 1.1GB/s. But dont be happy thats false. Becasue the speed that dd reported to us is the speed with which data was cached to RAM memory, not to the disk. So we need to ask dd command to report the speed only after the data is synced with the disk.For that we need to run the below command.

[root@slashroot2 ~]# dd if=/dev/zero of=speetest bs=1M count=100 conv=fdatasync
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 2.05887 seconds, 50.9 MB/s

As you can clearly see that with the attribute fdatasync the dd command will show the status rate only after the data is completely written to the disk. So now we have the actual sequencial write speed. Lets go to an amount of data size thats larger than the RAM. Lets take 200MB of data in 64kb block size.

[root@slashroot2 ~]# dd if=/dev/zero of=speedtest bs=64k count=3200 conv=fdatasync
3200+0 records in
3200+0 records out
209715200 bytes (210 MB) copied, 3.51895 seconds, 59.6 MB/s

as you can clearly see that the speed came to 59 MB/s. You need to note that ext3 bydefault if you do not specify the block size, gets formatted with a block size thats determined by the programes like mke2fs . You can verify yours with the following commands.

tune2fs -l /dev/sda1

dumpe2fs /dev/sda1

For testing the sequential read speed with dd command, you need to run the below command as below.

[root@myvm1 sarath]# dd if=speedtest of=/dev/null bs=64k count=24000
5200+0 records in
5200+0 records out
340787200 bytes (341 MB) copied, 3.42937 seconds, 99.4 MB/s

Performance Test using HDPARM

Now lets use some other tool other than dd command for our tests. We will start with hdparm command to test the speed. Hdparm tool is also available out of the box in most of the linux distribution.

[root@myvm1 ~]# hdparm -tT /dev/sda1
 
/dev/sda1:
 Timing cached reads:   5808 MB in  2.00 seconds = 2908.32 MB/sec
 Timing buffered disk reads:   10 MB in  3.12 seconds =   3.21 MB/sec

There are multiple things to understand here in the above hdparm results. the -t Option will show you the speed of reading from the cache buffer(Thats why its much much higher).

The -T option will show you the speed of reading without precached buffer(which from the above output is low 3.21 MB/sec as shown above. )

the hdparm output shows you both the cached reads and disk reads separately. As mentioned before hard disk seek time also matters a lot for your speed you can check your hard disk seek time with the following linux command. seek time is the time required by the hard disk to reach the sector where the data is stored.Now lets use this seeker tool to find out the seek time by the simple seek command.

[root@slashroot2 ~]# seeker /dev/sda1
Seeker v3.0, 2009-06-17, http://www.linuxinsight.com/how_fast_is_your_disk.html
Benchmarking /dev/sda1 [81915372 blocks, 41940670464 bytes, 39 GB, 39997 MB, 41 GiB, 41940 MiB]
[512 logical sector size, 512 physical sector size]
[1 threads]
Wait 30 seconds..............................
Results: 87 seeks/second, 11.424 ms random access time (26606211 < offsets < 41937280284)
[root@slashroot2 ~]#

its clearly mentioned that my disk did a 86 seeks for sectors containing data per second. Thats ok for a desktop Linux machine but for servers its not at all ok.

Read Write Benchmark Test using IOZONE:

Now there is one tool out there in linux that will do all these test in one shot. Thats none other than "IOZONE". We will do some benchmark test against my /dev/sda1 with the help of iozone.Computers or servers are always purchased keeping some purpose in mind. Some servers needs to be highend performance wise, some needs to be fast in sequencial reads,and some others are ordered keeping random reads in mind. IOZONE will be very much helpful in carrying out large number of permance benchmark test against the drives. The output produced by iozone is too much brief.

[root@myvm1 ~]# iozone -a /dev/sda1
             Auto Mode
        Command line used: iozone -a /dev/sda1
        Output is in Kbytes/sec
        Time Resolution = 0.000001 seconds.
        Processor cache size set to 1024 Kbytes.
        Processor cache line size set to 32 bytes.
        File stride size set to 17 * record size.
creditors you never heard
                                                            random  random    bkwd   record   stride
              KB  reclen   write rewrite    read    reread    read   write    read  rewrite     read   fwrite frewrite   fread  freread
              64       4  172945  581241  1186518  1563640  877647  374157  484928   240642   985893   633901   652867 1017433  1450619
              64       8   25549  345725   516034  2199541 1229452  338782  415666   470666  1393409   799055   753110 1335973  2071017
              64      16   68231  810152  1887586  2559717 1562320  791144 1309119   222313  1421031   790115   538032  694760  2462048
              64      32  338417  799198  1884189  2898148 1733988  864568 1421505   771741  1734912  1085107  1332240 1644921  2788472
              64      64   31775  811096  1999576  3202752 1832347  385702 1421148   771134  1733146   864224   942626 2006627  3057595
             128       4  269540  699126  1318194  1525916  390257  407760  790259   154585   649980   680625   684461 1254971  1487281
             128       8  284495  837250  1941107  2289303 1420662  779975  825344   558859  1505947   815392   618235  969958  2130559
             128      16  277078  482933  1112790  2559604 1505182  630556 1560617   624143  1880886   954878   962868 1682473  2464581
             128      32  254925  646594  1999671  2845290 2100561  554291 1581773   723415  2095628  1057335  1049712 2061550  2850336
             128      64  182344  871319  2412939   609440 2249929  941090 1827150  1007712  2249754  1113206  1578345 2132336  3052578
             128     128  301873  595485  2788953  2555042 2131042  963078  762218   494164  1937294   564075  1016490 2067590  2559306

Note: All the output you see above are in KB/Sec

The first column shows the file size used and second column shows the length of the record used.

Lets understand the output in some of the columns

The third Column-Write:This column shows the speed Whenever a new file is made in any file system under Linux. There is more overhead involved in the metadata storing. For example the inode for the file, and its entry in the journal etc. So creating a new file in a file system is always comparatively slower than overwriting an already created file.

Fourth column-Re-writing:This shows the speed reported in overwriting the file which is already created

Fifth column-Read:This reports the speed of reading an already existing file.

sixth column-Re-Read:This reports the speed of rereading a file which is already read. In all operating system there is disk locations cached for faster access. So Rereading a file which is already read, is faster.

Seventh Column-Random Read:This will show the speed when random area's of a single file is accessed. This is slower at times.

Eighth column-Random Write:random write is similar to random read. But instead will write to an already existing file by accessing random areas in the file.

Ninth Column-Random Mix:This column tells the random read and random write speeds combined together.

Backwards Read: This column reports the speed of doing a backward read. For example start reading from the end to the beginning of the file.

For more information Please reffer to : url=http://www.iozone.org/You can save the output of the iozone using -b option avialable in iozone as follows.

[root@myvm1 ~]# iozone -a -b speedtest.xls

Where "speedtest.xls" is the output file. You can now open this output file in excel sheet using open office and have a complete detailed report regarding the speed test.-s option present in iozone can be used to specify the filesize to be used for iozone test. As shown below.

[root@slashroot2 ~]# iozone -a -s 4048 -i 0

The above command will run the iozone utility with a filesize of 4MB and only will do a write and rewrite test because we have given it 0 option.You can pass numberical options to do only selective tests. Such as 0 for write and rewrite and 1 for 1 read and reread,2 for random read and write etc(you need to use the -i option before the numeric option).

Another tool that can be used for filesystem performance test is bonnie

Read Write Performance Test using Bonnie

Bonnie is also a very nice tool that can be used for performance benchmarking the hardisk. It can be installed by enabling the rpmforge repo.Now run the command as shown below for the test. run the command in the directory where your desired partition is mounted. for example am testing the /dev/sda1(which is my /).

[root@slashroot2 ~]# bonnie
Bonnie: Warning: You have 503MiB RAM, but you test with only 100MiB datasize!
Bonnie:          This might yield unrealistically good results,
Bonnie:          for reading and seeking and writing.
Bonnie 1.5: File './Bonnie.4614', size: 104857600, volumes: 1
Writing with putc()...         done:  24712 kiB/s  98.2 %CPU
Rewriting...                   done: 789544 kiB/s  83.3 %CPU
Writing intelligently...       done: 253001 kiB/s  80.5 %CPU
Reading with getc()...         done:  26101 kiB/s  99.5 %CPU
Reading intelligently...       done: 3723094 kiB/s 101.8 %CPU
Seeker 1...Seeker 2...Seeker 3...start 'em...done...done...done...
Estimated seek time: raw 0.006ms, eff 0.001ms

              ---Sequential Output (nosync)--- ---Sequential Input-- --Rnd Seek-
              -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --04k (03)-
Machine    MB kiB/s %CPU kiB/s %CPU kiB/s %CPU kiB/s %CPU kiB/s %CPU   /sec %CPU
slashr 1* 100 24712 98.2253001 80.5 789544 83.3 26101 99.53723094  102 173145.2 95.2

the final line in the above output shows sequencial output,sequencial input and random seek etc.you need to pass the -y option to sync the data to the disk. You can specify the filesize used to test with -s option.

[root@slashroot2 ~]# bonnie -y -s 1000

How to use FIO for IO benchmark Test

Fio is also another nice and small tool that can be used for benchmarking the IO usage of a particular drive. Unlike previous tools we saw, the fio tool does not run with command line options.

A file with full of parameters regarding the type of test is made. And then this file is passed to the "fio" command as an argument.

So lets begin by making our fio test file and its argument.

the file that will be passed as an argument will begin by

; -- start job file -- 

and will end with

; -- end job file --

Lets do a test with FIO and a file with argument as the parameter.

[root@myvm1 sarath]# cat fiotest
; -- start job file --
[random-writers]
rw=randrw
size=50m
numjobs=1
; -- end job file --

The output of the fio command is too much descriptive..but (there is a different documentation altogether for understanding the output for fio command)

The result of the At the end of the fio command with "the argument file" You will see the summery of the test.

Run status group 0 (all jobs):
   READ: io=25708KB, aggrb=306KB/s, minb=306KB/s, maxb=306KB/s, mint=83860msec, maxt=83860msec
  WRITE: io=25492KB, aggrb=303KB/s, minb=303KB/s, maxb=303KB/s, mint=83860msec, maxt=83860msec
 
Disk stats (read/write):
  sda: ios=6422/5964, merge=0/238, ticks=81457/2073064, in_queue=2173820, util=98.75%

the summery shows the below following things.

Aggrb: Aggregate bandwidth of all the process threads used in the test

Minb: Minimum Bandwidth

Maxb: Maximum bandwidth

Ios: Total input output operation performed with the disks.

in Queue: the total time spend being in queue

util: this shows the percentage of duration in which the disk was kept busy with IO operations during the test. In our example output the util is 98 percent which mens, during the task the disk was kept almost full busy with IO operations.

FIO is a very good tool if you want to test the performance of your disk during the heavy intensice IO operations.

Almost all the tools explained in the above tutorial are very much vast in themselfs. Hope this inroductory tutorial for those tools was helpful for benchmark test.