Thứ Hai, 20 tháng 3, 2017

Determining Linux IO Utilization

The iostat command is used for monitoring system input/output device loading by observing the time the devices are active in relation to their average transfer rates. The iostat command generates reports that can be used to change system configuration to better balance the input/output load between physical disks.
The first report generated by the iostat command provides statistics concerning the time since the system was booted. Each subsequent report covers the time since the previous report. All statistics are reported each time the iostat command is run. The report consists of a CPU header row followed by a row of CPU statistics. On multiprocessor systems, CPU statistics are calculated system-wide as averages among all processors. A device header row is displayed followed by a line of statistics for each device that is configured. When option -n is used, an NFS header row is displayed followed by a line of statistics for each network filesystem that is mounted.
The interval parameter specifies the amount of time in seconds between each report. The first report contains statistics for the time since system startup (boot). Each subsequent report contains statistics collected during the interval since the previous report. The count parameter can be specified in conjunction with the interval parameter. If the count parameter is specified, the value of count determines the number of reports generated at interval seconds apart. If the intervalparameter is specified without the count parameter, the iostat command generates reports continuously.
  • -d : Display the device utilization report (d == disk)
  • -x : Display extended statistics including disk utilization
[root@ldapmaster ~]# iostat -d -x
Linux 2.6.18-274.3.1.el5 (ldapmaster.ardclb.uta.edu)     11/04/2011

Device:         rrqm/s   wrqm/s   r/s   w/s   rsec/s   wsec/s avgrq-sz avgqu-sz   await  svctm  %util
sda               0.14    28.99  0.49  6.57    15.85   284.61    42.52     0.10   13.50   2.33   1.65
sda1              0.01     0.00  0.00  0.00     0.03     0.00    30.00     0.00   13.64  11.99   0.00
sda2              0.02     0.08  0.00  0.00     0.06     0.64   315.29     0.00   15.83   5.60   0.00
sda3              0.12    28.91  0.49  6.57    15.76   283.97    42.43     0.10   13.50   2.33   1.65
sdb               1.12   133.01  7.68 197.09   456.28  2640.95    15.13     3.41   16.66   0.38   7.73
dm-0              0.00     0.00  0.09  4.77     1.69    38.13     8.20     0.07   14.03   0.54   0.26
dm-1              0.00     0.00  0.26 26.47     8.74   211.76     8.25     0.56   20.85   0.46   1.23
dm-2              0.00     0.00  0.25  3.21     5.23    25.71     8.93     0.06   16.08   1.09   0.38
dm-3              0.00     0.00  0.01  0.01     0.06     0.07     8.81     0.00    9.70   4.17   0.01
dm-4              0.00     0.00  0.00  1.04     0.03     8.30     8.00     0.01    9.83   1.59   0.17
  • r/s and w/s--- respectively, the number of read and write requests issued by processes to the OS for a device.
  • rsec/s and wsec/s — sectors read/written (each sector 512 bytes).
  • rkB/s and wkB/s — kilobytes read/written.
  • avgrq-sz — average sectors per request (for both reads and writes). Do the math — (rsec + wsec) / (r + w) = (456.28+2640.95)/(7.68+197.09)=15.1254 If you want it in kilobytes, divide by 2.
    If you want it separate for reads and writes — do you own math using rkB/s and wkB/s.
  • avgqu-sz — average queue length for this device.
  • await — average response time (ms) of IO requests to a device. The name is a bit confusing as this is the total response time including wait time in the requests queue (let call it qutim), and service time that device was working servicing the requests (see next column — svctim).So the formula is await = qutim + svctim.
  • svctim — average time (ms) a device was servicing requests. This is a component of total response time of IO requests.
%util is a pretty confusing value. The man page defines it as, Percentage of CPU time during which I/O requests were issued to the device (bandwidth utilization for the device). Device saturation occurs when this value is close to 100%. A bit difficult to digest. Perhaps it’s better to think of it as percentage of time the device was servicing requests as opposed to being idle. To understand it better here is the formula:
utilization = ( (read requests + write requests) * service time in ms / 1000 ms ) * 100% or
For /dev/sdb %util = ( r + w ) * svctim /10 = ((7.68+197.09) 3.8 ) = 7.78
Traditionally, it’s common to assume that the closer to 100% utilization a device is, the more saturated it is. This might be true when the system device corresponds to a single physical disk. However, with devices representing a LUN of a modern storage box, the story might be completely different.
Rather than looking at device utilization, there is another way to estimate how loaded a device is. Look at the non-existent column I mentioned above — qutim — the average time a request is spending in the queue. If it’s insignificant, compare it to svctim — the IO device is not saturated. When it becomes comparable to svctim and goes above it, then requests are queued longer and a major part of response time is actually time spent waiting in the queue.
The figure in the await column should be as close to that in the svctim column as possible. If await goes much above svctim, watch out! The IO device is probably overloaded.
You can add an internal and count to repeat the test over time. This should be useful for identifying trends.
  • interval : It is time period in seconds between two samples . iostat 2 will give data at each 2 seconds interval.
  • count : It is the number of times the data is needed . iostat 2 5 will give data at 2 seconds interval 5 times
[root@ldapmaster ~]#  iostat -d -x 5 3
Linux 2.6.18-274.3.1.el5 (ldapmaster.ardclb.uta.edu)     11/04/2011

Device:         rrqm/s   wrqm/s   r/s   w/s   rsec/s   wsec/s avgrq-sz avgqu-sz   await  svctm  %util
sda               0.14    28.99  0.49  6.57    15.85   284.61    42.52     0.10   13.50   2.33   1.65
sda1              0.01     0.00  0.00  0.00     0.03     0.00    30.00     0.00   13.64  11.99   0.00
sda2              0.02     0.08  0.00  0.00     0.06     0.64   315.29     0.00   15.83   5.60   0.00
sda3              0.12    28.91  0.49  6.57    15.76   283.97    42.43     0.10   13.50   2.33   1.65
sdb               1.12   133.01  7.68 197.09   456.28  2640.95    15.13     3.41   16.66   0.38   7.73
dm-0              0.00     0.00  0.09  4.77     1.69    38.13     8.20     0.07   14.03   0.54   0.26
dm-1              0.00     0.00  0.26 26.47     8.74   211.76     8.25     0.56   20.85   0.46   1.23
dm-2              0.00     0.00  0.25  3.21     5.23    25.71     8.93     0.06   16.08   1.09   0.38
dm-3              0.00     0.00  0.01  0.01     0.06     0.07     8.81     0.00    9.70   4.17   0.01
dm-4              0.00     0.00  0.00  1.04     0.03     8.30     8.00     0.01    9.83   1.59   0.17

Device:         rrqm/s   wrqm/s   r/s   w/s   rsec/s   wsec/s avgrq-sz avgqu-sz   await  svctm  %util
sda               0.00    10.80  0.00  3.00     0.00   110.40    36.80     0.00    1.07   0.93   0.28
sda1              0.00     0.00  0.00  0.00     0.00     0.00     0.00     0.00    0.00   0.00   0.00
sda2              0.00     0.00  0.00  0.00     0.00     0.00     0.00     0.00    0.00   0.00   0.00
sda3              0.00    10.80  0.00  3.00     0.00   110.40    36.80     0.00    1.07   0.93   0.28
sdb               0.00     0.80  0.00  0.60     0.00    11.20    18.67     0.00    1.00   1.00   0.06
dm-0              0.00     0.00  0.00  4.40     0.00    35.20     8.00     0.00    1.00   0.09   0.04
dm-1              0.00     0.00  0.00  4.80     0.00    38.40     8.00     0.01    1.04   0.25   0.12
dm-2              0.00     0.00  0.00  2.80     0.00    22.40     8.00     0.00    0.93   0.07   0.02
dm-3              0.00     0.00  0.00  0.20     0.00     1.60     8.00     0.00    3.00   3.00   0.06
dm-4              0.00     0.00  0.00  1.60     0.00    12.80     8.00     0.00    1.00   0.25   0.04

Device:         rrqm/s   wrqm/s   r/s   w/s   rsec/s   wsec/s avgrq-sz avgqu-sz   await  svctm  %util
sda               0.00     6.40  0.00  4.60     0.00    88.00    19.13     0.01    1.43   0.43   0.20
sda1              0.00     0.00  0.00  0.00     0.00     0.00     0.00     0.00    0.00   0.00   0.00
sda2              0.00     0.00  0.00  0.00     0.00     0.00     0.00     0.00    0.00   0.00   0.00
sda3              0.00     6.40  0.00  4.60     0.00    88.00    19.13     0.01    1.43   0.43   0.20
sdb               0.00     0.60  0.00  0.60     0.00     9.60    16.00     0.00    1.33   1.33   0.08
dm-0              0.00     0.00  0.00  7.40     0.00    59.20     8.00     0.02    2.49   0.16   0.12
dm-1              0.00     0.00  0.00  3.60     0.00    28.80     8.00     0.00    0.89   0.28   0.10
dm-2              0.00     0.00  0.00  0.00     0.00     0.00     0.00     0.00    0.00   0.00   0.00
dm-3              0.00     0.00  0.00  0.00     0.00     0.00     0.00     0.00    0.00   0.00   0.00
dm-4              0.00     0.00  0.00  0.00     0.00     0.00     0.00     0.00    0.00   0.00   0.00

[root@ldapmaster ~]#
When option -n is used, an NFS header row is displayed followed by a line of statistics for each network filesystem that is mounted.
Device:                  rBlk_nor/s   wBlk_nor/s   rBlk_dir/s   wBlk_dir/s   rBlk_svr/s   wBlk_svr/s     ops/s    rops/s    wops/s
axiomfs01:/web-vault/web         0.26         0.00         0.00         0.00         0.03         0.00      0.01      0.00      0.00
axiomfs01:/web-vault/httpd         0.26         0.00         0.00         0.00         0.03         0.00      0.01      0.00      0.0
rBlk_nor/s Indicate the number of blocks read by applications via the read(2) system call interface. A block has a size of 512 bytes.
  • wBlk_nor/s Indicate the number of blocks written by applications via the write(2) system call interface.
  • rBlk_dir/s Indicate the number of blocks read from files opened with the O_DIRECT flag.
  • wBlk_dir/s Indicate the number of blocks written to files opened with the O_DIRECT flag.
  • rBlk_svr/s Indicate the number of blocks read from the server by the NFS client via an NFS READ request.
  • wBlk_svr/s Indicate the number of blocks written to the server by the NFS client via an NFS WRITE request.
  • rkB_nor/s Indicate the number of kilobytes read by applications via the read(2) system call interface.
  • wkB_nor/s Indicate the number of kilobytes written by applications via the write(2) system call interface.
  • rkB_dir/s Indicate the number of kilobytes read from files opened with the O_DIRECT flag.
  • wkB_dir/s Indicate the number of kilobytes written to files opened with the O_DIRECT flag.
  • rkB_svr/s Indicate the number of kilobytes read from the server by the NFS client via an NFS READ request.
  • wkB_svr/s Indicate the number of kilobytes written to the server by the NFS client via an NFS WRITE request.
  • rMB_nor/s Indicate the number of megabytes read by applications via the read(2) system call interface.
  • wMB_nor/s Indicate the number of megabytes written by applications via the write(2) system call interface.
  • rMB_dir/s Indicate the number of megabytes read from files opened with the O_DIRECT flag.
  • wMB_dir/s Indicate the number of megabytes written to files opened with the O_DIRECT flag.
  • rMB_svr/s Indicate the number of megabytes read from the server by the NFS client via an NFS READ request.
  • wMB_svr/s Indicate the number of megabytes written to the server by the NFS client via an NFS WRITE request.
  • ops/s Indicate the number of operations that were issued to the filesystem per second.
  • rops/s Indicate the number of 'read' operations that were issued to the filesystem per second.

Không có nhận xét nào:

Đăng nhận xét