Swap that does not wait for the disk

Swap gets a bad name, but a little of it is healthy. The problem is where it lives. On most cloud servers the swap file sits on a network-attached volume, so every page the kernel brings back has to make a trip over that network. When a database or Redis ends up partly in swap, real requests wait on that trip. Redis in swap rather defeats the point of Redis.

Find out what is swapped

# Processes with more than 10 MB in swap, biggest first
for p in /proc/[0-9]*; do
  s=$(awk '/VmSwap/{print $2}' "$p/status" 2>/dev/null)
  [ "${s:-0}" -gt 10240 ] && echo "$s kB $(cat "$p/comm")"
done | sort -rn | head

# How often you are actually paging back in
vmstat 5 5   # watch the si column

If si is regularly above zero and the culprits are your database or cache, you are paying the disk penalty on real requests.

Set up zram

zram creates a swap device in RAM and compresses whatever lands in it. On Ubuntu:

sudo apt install zram-tools

sudo tee /etc/default/zramswap > /dev/null <<'EOF'
ALGO=lz4
PERCENT=25
PRIORITY=100
EOF
sudo systemctl restart zramswap

printf 'vm.swappiness=60\nvm.page-cluster=0\n' | sudo tee /etc/sysctl.d/99-zram.conf
sudo sysctl --system

swapon --show   # zram at priority 100, the old swap file lower
zramctl         # size, data stored and compression

Keep your existing swap file as a low-priority fallback. Nothing is taken away, and the kernel only reaches for the disk when the compressed device is full.

Why lz4 and not zstd

zstd compresses harder, but it costs more CPU. On burstable instances that run on CPU credits, cheap and fast beats small. lz4 typically squeezes memory to around half its size, which is plenty.

What to expect

Measured on a typical cloud server:

Faster swap-ins are nice, but there is a quieter win too. If your swap file shares a volume with your database, swapping competes with real queries for the same disk throughput. With zram, it does not.

If you need to undo it

sudo systemctl disable --now zramswap
sudo rm /etc/sysctl.d/99-zram.conf
sudo sysctl vm.swappiness=10

Check again after a week of normal traffic: zramctl for the ratio, and redis-cli --latency if you run Redis. Both should stay flat.