ZFS Tuning Parameters: A Practical Reference
What happens when you raise or lower each ZFS dataset property and kernel tunable, and how to pick a value for your workload.
5-9 minutes(1108 words)complex
Quick Navigation
Difficulty: Advanced
Estimated Time: 20-30 minutes
Prerequisites: Working ZFS pool, Linux command line, Basic storage concepts, Root access
Most ZFS tuning advice is a list of values to paste in. This reference takes the opposite approach: for each parameter, what actually degrades when you push it in either direction, and how to reason about the value for your workload.
Two of these settings are permanent. Get those right first.
Dataset and Pool Properties
| Parameter | Raising it | Lowering it | How to choose |
|---|---|---|---|
ashift (permanent) | 13 gives 8K blocks: wasted space on small files, RAIDZ padding overhead | 9 gives 512B: read-modify-write on 4K drives, severe performance loss | Use 12 (4K) for nearly all modern disks and SSDs. Never leave it at 9 unless the drive is truly 512-native. Cannot be changed after vdev creation. |
recordsize (default 128K) | 1M: better compression, less metadata, strong sequential throughput; bad partial-write amplification | 16K: matches database page size, low write amplification; more metadata, worse compression, more IOPS | Match the dominant I/O size of the application. Media and backup, 1M. PostgreSQL, 16K to 32K. General files, 128K. Only affects newly written data. |
volblocksize (zvol, permanent) | 64K and above: better throughput and compression, worse random write amplification | 8K: good for VM random I/O, but heavy padding overhead on RAIDZ | 16K is a sane default. On RAIDZ prefer 32K or larger; on mirrors 8K to 16K is fine. Set at creation only. |
compression | zstd-9 or zstd-19: better ratio, much more CPU, higher write latency | off: no CPU cost, but more I/O and worse throughput | lz4 is the safe default and nearly free. zstd-3 if CPU is idle and you want ratio. Never off on spinning disks. |
atime | on: a metadata write on every read | off: fewer writes, less fragmentation | off for VMs, databases, and backups. relatime for home directories or mail servers that need it. |
sync | always: every write is durable, slow without a SLOG | disabled: large speedup, but roughly the last 5 seconds of writes are lost on power cut (the pool itself stays intact) | Leave at standard. Use disabled only for scratch or rebuildable data. For NFS and VM stores, add a SLOG device rather than disabling sync. |
logbias | throughput: bypasses the SLOG, better for large streaming sync writes | latency: uses the SLOG, better for small sync writes | latency (the default) when you have a SLOG. throughput for big sequential sync writes such as a database bulk load. |
primarycache | all: caches both data and metadata in ARC | metadata or none: frees ARC, but cold reads | all normally. metadata when the application does its own caching (a large database buffer pool) or for one-pass sequential scans. |
special_small_blocks | 64K: more small blocks land on the SSD special vdev, much faster small I/O | 0 or off: everything stays on the main vdevs | Start at 32K if you have a special vdev. Watch its fill level: once it fills, writes spill back to HDD. Losing a special vdev loses the pool, so mirror it. |
copies | 2 to 3: survives block-level corruption, multiplies space used | 1: normal | Leave at 1. Redundancy belongs at the vdev level, not here, except on single-disk laptops. |
Kernel Module Tunables
| Parameter | Raising it | Lowering it | How to choose |
|---|---|---|---|
zfs_arc_max | More cache hits, less disk I/O; starves applications of RAM | More RAM for apps and VMs; higher read latency | On a dedicated NAS, 75 to 85 percent of RAM. On a host also running databases or VMs, cap it explicitly: the 50 percent default is often too greedy. |
zfs_dirty_data_max | Absorbs write bursts, better aggregation; longer stalls when the transaction group finally flushes | Smoother, more predictable latency; less throughput on bursts | The default (10 percent of RAM, capped at 4G) is fine. Lower it if you see periodic multi-second write stalls. |
zfs_txg_timeout (5s) | 10s and above: bigger, better-aggregated commits; larger latency spikes and more data in flight | 1 to 2s: smoother latency, more frequent smaller writes | Raise it for spinning disks under sequential load; lower it for latency-sensitive workloads. |
zfs_vdev_*_max_active (queue depth) | More parallelism, higher throughput on SSD and NVMe | Lower per-I/O latency, less queue buildup on HDD | HDDs like small queues (default around 10). NVMe benefits from raising the async read and write limits. |
zfs_prefetch_disable | 1 disables prefetch: helps pure-random workloads, wastes less ARC | 0 (default) leaves prefetch on: a large win for sequential reads | Leave it enabled unless you have measured prefetch hurting a random-read database. |
l2arc_write_max | Warms the L2ARC faster; more SSD wear, steals bandwidth from normal I/O | Gentler on the SSD, slow cache warm-up | Only raise it if the L2ARC hit rate is still climbing after days. L2ARC also consumes ARC RAM for its headers, so it can hurt small-RAM systems. |
How to Choose
- Get the permanent settings right first.
ashiftandvolblocksizecannot be changed later. Everything else is fixable. - Match block size to workload I/O size. This is the single biggest lever. A mismatch causes read-modify-write amplification that no other tuning will fix.
- Change one thing, measure, repeat. Use
zpool iostat -w 1for latency histograms andarc_summaryfor cache behaviour, rather than guessing. - Remember what is retroactive.
recordsizeandcompressiononly apply to newly written blocks. You must rewrite or send/receive the data to see the effect.
Quick Presets
VM images
zfs set volblocksize=16K tank/vms # at creation only
zfs set compression=lz4 tank/vms
zfs set atime=off tank/vms
zfs set sync=standard tank/vms # pair with a SLOG device
PostgreSQL or MySQL
zfs set recordsize=16K tank/pgdata
zfs set compression=lz4 tank/pgdata
zfs set atime=off tank/pgdata
zfs set primarycache=metadata tank/pgdata # only if the buffer pool is large
Media and backup
zfs set recordsize=1M tank/media
zfs set compression=zstd-3 tank/media
zfs set atime=off tank/media
Verifying Your Changes
Check what a dataset is actually using, including whether the value is inherited or set locally:
zfs get recordsize,compression,atime,primarycache tank/pgdata
Watch latency distribution while the workload runs:
zpool iostat -w 1
Check ARC efficiency before deciding that more cache is the answer:
arc_summary | head -40
Tags: #ZFS #Storage #Linux #Performance #Tuning