Steal time, and why yours reads zero

Where the number comes from

%st in top is the share of time your virtual CPU was runnable but not scheduled on a physical core, as reported by the hypervisor. The counter exists because a guest cannot otherwise tell the difference between a slow core and a core it never received.

top -bn1 | head -3
vmstat 1 5
mpstat -P ALL 1 5

The raw value, for anyone graphing it themselves, is the eighth field of the cpu line in /proc/stat, counted in USER_HZ:

grep -w cpu /proc/stat

Why it is zero here

Cores are allocated once. A vCPU on your instance is pinned to a physical core that is in nobody else’s allocation, on a host that is not oversubscribed, so there is no queue to wait in. Not a low figure, not a credit balance that runs out on the fourth day of the month — nothing to steal.

If yours is not zero, that is a defect and we want the ticket. Include a mpstat -P ALL 5 12 run and the instance ID.

Why other people’s figures are not zero

Burstable plans are built out of steal time. Several guests share a core, each holds a credit balance, and when the balance empties the scheduler hands the core to somebody else. Sold honestly, that is a legitimate product for workloads that idle. It is not this one. We deleted the cheap tier in 2021 and the overselling left with it.

What to check when the machine feels slow anyway

Steal is the first thing everyone looks at and rarely the answer. In order of likelihood:

Run queue

vmstat 1 5

More runnable threads than cores, in the r column, means you are simply out of CPU. No amount of tuning invents another core.

Disk

iostat -x 1 5

%iowait rising while user time stays low points at storage, not the processor.

Your own cgroup

This is the one that catches people. A container with a CPU quota feels exactly like steal time from inside the process and reports zero steal, because the limit is yours rather than ours:

cat /sys/fs/cgroup/cpu.stat

nr_throttled and throttled_usec climbing means the quota set by your own orchestrator is the ceiling. Raise it or remove it.

Frequency

Boost clocks move with the workload, and a single-threaded test is the honest way to see where you actually sit:

apt install -y sysbench
sysbench cpu --cpu-max-prime=20000 --threads=1 run

Compare the events-per-second figure against another instance rather than against a number from somebody’s blog post in 2019.

Guest and nested columns

%guest and %gnice are non-zero only when you are running virtual machines inside your instance. Nested virtualisation is available on the larger EPYC plans and on bare metal; check for the device before you design around it:

ls -l /dev/kvm

The short version

High r with user time near a hundred percent is the CPU. High %iowait is the disk. cpu.stat counting throttles is your own quota. Non-zero steal is us, and it should not happen.