--leak-check

When enabled, Memcheck searches for memory leaks when the client program finishes. The default value is summary, which outputs the number of leaks found. Other possible values are yes and full, both of which give details of each individual leak, and no, which disables memory leak checking.

--undef-value-errors

When enabled (set to yes), Memcheck reports errors when undefined values are used. When disabled (set to no), undefined value errors are not reported. This is enabled by default. Disabling it speeds up Memcheck slightly.

--ignore-ranges

Allows the user to specify one or more ranges that Memcheck should ignore when checking for addressability. Multiple ranges are delimited by commas, for example, --ignore-ranges=0xPP-0xQQ,0xRR-0xSS.

--I1

Specifies the size, associativity, and line size of the first-level instruction cache, separated by commas: --I1=size,associativity,line size.

--D1

Specifies the size, associativity, and line size of the first-level data cache, separated by commas: --D1=size,associativity,line size.

--LL

Specifies the size, associativity, and line size of the last-level cache, separated by commas: --LL=size,associativity,line size.

--cache-sim

Enables or disables the collection of cache access and miss counts. The default value is yes (enabled).

Note that disabling both this and --branch-sim leaves Cachegrind with no information to collect.

--branch-sim

Enables or disables the collection of branch instruction and misprediction counts. This is set to no (disabled) by default, since it slows Cachegrind by approximately 25 per-cent.

Note that disabling both this and --cache-sim leaves Cachegrind with no information to collect.

--heap

Specifies whether to perform heap profiling. The default value is yes. Heap profiling can be disabled by setting this option to no.

--heap-admin

Specifies the number of bytes per block to use for administration when heap profiling is enabled. The default value is 8 bytes per block.

--stacks

Specifies whether to perform stack profiling. The default value is no (disabled). To enable stack profiling, set this option to yes, but be aware that doing so will greatly slow Massif. Also note that Massif assumes that the main stack has size zero at start-up in order to better indicate the size of the stack portion over which the program being profiled has control.

--time-unit

Specifies the unit of time used for the profiling. There are three valid values for this option: instructions executed (i), the default value, which is useful in most cases; real time (ms, in milliseconds), which can be useful in certain instances; and bytes allocated/deallocated on the heap and/or stack (B), which is useful for very short-run programs, and for testing purposes, because it is the most reproducible across different machines. This option is useful when graphing Massif output with ms_print.

Tunables

back_seek_max

Backward seeks are typically bad for performance, as they can incur greater delays in repositioning the heads than forward seeks do. However, CFQ will still perform them, if they are small enough. This tunable controls the maximum distance in KB the I/O scheduler will allow backward seeks. The default is 16 KB.

back_seek_penalty

Because of the inefficiency of backward seeks, a penalty is associated with each one. The penalty is a multiplier; for example, consider a disk head position at 1024KB. Assume there are two requests in the queue, one at 1008KB and another at 1040KB. The two requests are equidistant from the current head position. However, after applying the back seek penalty (default: 2), the request at the later position on disk is now twice as close as the earlier request. Thus, the head will move forward.

fifo_expire_async

This tunable controls how long an async (buffered write) request can go unserviced. After the expiration time (in milliseconds), a single starved async request will be moved to the dispatch list. The default is 250 ms.

fifo_expire_sync

This is the same as the fifo_expire_async tunable, for for synchronous (read and O_DIRECT write) requests. The default is 125 ms.

group_idle

When set, CFQ will idle on the last process issuing I/O in a cgroup. This should be set to 1 when using proportional weight I/O cgroups and setting slice_idle to 0 (typically done on fast storage).

group_isolation

If group isolation is enabled (set to 1), it provides a stronger isolation between groups at the expense of throughput. Generally speaking, if group isolation is disabled, fairness is provided for sequential workloads only. Enabling group isolation provides fairness for both sequential and random workloads. The default value is 0 (disabled). Refer to Documentation/cgroups/blkio-controller.txt for further information.

low_latency

When low latency is enabled (set to 1), CFQ attempts to provide a maximum wait time of 300 ms for each process issuing I/O on a device. This favors fairness over throughput. Disabling low latency (setting it to 0) ignores target latency, allowing each process in the system to get a full time slice. Low latency is enabled by default.

quantum

The quantum controls the number of I/Os that CFQ will send to the storage at a time, essentially limiting the device queue depth. By default, this is set to 8. The storage may support much deeper queue depths, but increasing quantum will also have a negative impact on latency, especially in the presence of large sequential write workloads.

slice_async

This tunable controls the time slice allotted to each process issuing asynchronous (buffered write) I/O. By default it is set to 40 ms.

slice_idle

This specifies how long CFQ should idle while waiting for further requests. The default value in Red Hat Enterprise Linux 6.1 and earlier is 8 ms. In Red Hat Enterprise Linux 6.2 and later, the default value is 0. The zero value improves the throughput of external RAID storage by removing all idling at the queue and service tree level. However, a zero value can degrade throughput on internal non-RAID storage, because it increases the overall number of seeks. For non-RAID storage, we recommend a slice_idle value that is greater than 0.

slice_sync

This tunable dictates the time slice allotted to a process issuing synchronous (read or direct write) I/O. The default is 100 ms.

Tunables

fifo_batch

This determines the number of reads or writes to issue in a single batch. The default is 16. Setting this to a higher value may result in better throughput, but will also increase latency.

front_merges

You can set this tunable to 0 if you know your workload will never generate front merges. Unless you have measured the overhead of this check, it is advisable to leave it at its default setting (1).

read_expire

This tunable allows you to set the number of milliseconds in which a read request should be serviced. By default, this is set to 500 ms (half a second).

write_expire

This tunable allows you to set the number of milliseconds in which a write request should be serviced. By default, this is set to 5000 ms (five seconds).

writes_starved

This tunable controls how many read batches can be processed before processing a single write batch. The higher this is set, the more preference is given to reads.

/sys/block/sdX/queue tunables

add_random

In some cases, the overhead of I/O events contributing to the entropy pool for /dev/random is measurable. In such cases, it may be desirable to set this value to 0.

max_sectors_kb

By default, the maximum request size sent to disk is 512 KB. This tunable can be used to either raise or lower that value. The minimum value is limited by the logical block size; the maximum value is limited by max_hw_sectors_kb. There are some SSDs which perform worse when I/O sizes exceed the internal erase block size. In such cases, it is recommended to tune max_hw_sectors_kb down to the erase block size. You can test for this using an I/O generator such as iozone or aio-stress, varying the record size from, for example, 512 bytes to 1 MB.

nomerges

This tunable is primarily a debugging aid. Most workloads benefit from request merging (even on faster storage such as SSDs). In some cases, however, it is desirable to disable merging, such as when you want to see how many IOPS a storage back-end can process without disabling read-ahead or performing random I/O.

nr_requests

Each request queue has a limit on the total number of request descriptors that can be allocated for each of read and write I/Os. By default, the number is 128, meaning 128 reads and 128 writes can be queued at a time before putting a process to sleep. The process put to sleep is the next to try to allocate a request, not necessarily the process that has allocated all of the available requests.

If you have a latency-sensitive application, then you should consider lowering the value of nr_requests in your request queue and limiting the command queue depth on the storage to a low number (even as low as 1), so that writeback I/O cannot allocate all of the available request descriptors and fill up the device queue with write I/O. Once nr_requests have been allocated, all other processes attempting to perform I/O will be put to sleep to wait for requests to become available. This makes things more fair, as the requests are then distributed in a round-robin fashion (instead of letting one process consume them all in rapid succession). Note that this is only a problem when using the deadline or noop schedulers, as the default CFQ configuration protects against this situation.

optimal_io_size

In some circumstances, the underlying storage will report an optimal I/O size. This is most common in hardware and software RAID, where the optimal I/O size is the stripe size. If this value is reported, applications should issue I/O aligned to and in multiples of the optimal I/O size whenever possible.

read_ahead_kb

The operating system can detect when an application is reading data sequentially from a file or from disk. In such cases, it performs an intelligent read-ahead algorithm, whereby more data than is requested by the user is read from disk. Thus, when the user next attempts to read a block of data, it will already by in the operating system's page cache. The potential down side to this is that the operating system can read more data from disk than necessary, which occupies space in the page cache until it is evicted because of high memory pressure. Having multiple processes doing false read-ahead would increase memory pressure in this circumstance.

For device mapper devices, it is often a good idea to increase the value of read_ahead_kb to a large number, such as 8192. The reason is that a device mapper device is often made up of multiple underlying devices. Setting this value to the default (128 KB) multiplied by the number of devices you are mapping is a good starting point for tuning.

rotational

Traditional hard disks have been rotational (made up of spinning platters). SSDs, however, are not. Most SSDs will advertise this properly. If, however, you come across a device that does not advertise this flag properly, it may be necessary to set rotational to 0 manually; when rotational is disabled, the I/O elevator does not use logic that is meant to reduce seeks, since there is little penalty for seek operations on non-rotational media.

rq_affinity

I/O completions can be processed on a different CPU from the one that issued the I/O. Setting rq_affinity to 1 causes the kernel to deliver completions to the CPU on which the I/O was issued. This can improve CPU data caching effectiveness.