Capping a Radeon RX 7900 XTX for LLM inference: a step-by-step

One of the machines in the house runs the language model my tools point at: a Radeon RX 7900 XTX behind a llama.cpp llama-server (with an Ollama-compatible shim in front), serving a 27B-parameter quantised model, one request at a time. Its power limit sat at 315 W and, under load, it used every watt of it. I wanted about 250 W, and a number for what that costs in tokens per second.

An AI agent did the digging over SSH — it had no root, so the sudo lines were mine — and the answer turned out to be 283 W, 8 °C cooler at the junction, and about a one-percent throughput cost. This is the whole session, dead ends included, because the dead ends are where the actual information was.

The workload, and why it predicts the result

The model is a 27B quantised at IQ4_XS, roughly 16 GB, fully offloaded to VRAM, with multi-token-prediction speculative decoding (up to two draft tokens accepted per step). Generating one token is, to a good approximation, one full stream of the weights through 2,640 GB/s of GDDR6. Throughput is therefore set by memory bandwidth — bytes per token over gigabits per second — and the compute, and most of the power that goes with it, sits mostly on the bench.

That one sentence predicts nearly everything that follows.

Step 1 — find the knobs

lspci | grep -i vga     # Navi 31 [Radeon RX 7900 XT/XTX/GRE]

The card is card1 in /sys/class/drm, and the amdgpu driver publishes its sensors under hwmon. Find it by name:

for h in /sys/class/hwmon/hwmon*; do
  [ "$(cat "$h/name" 2>/dev/null)" = "amdgpu" ] && ls "$h" | grep -E 'power|temp'
done

The numbers, all in µW — the first trap; 315000000 is 315 watts, not 0.315:

The 250 W target died in this table: the range is fixed in the VBIOS, and 283 W is the lowest the card will accept. The plan became: set the cap to 283 W, benchmark, and see whether there was anything else the card would let us do.

Writing the cap needs root (the file is root-owned, and the box has no passwordless sudo):

sudo sh -c 'echo 283000000 > /sys/class/hwmon/hwmon5/power1_cap'

(hwmon5 — an index, and as we’ll find in step 8, an index that can move between boots.)

Step 2 — a baseline before touching anything

The protocol, identical for every configuration so the only variable is the power setting:

• a POST /api/generate to the shim with a fixed 39-token prompt and num_predict: 1024;
• tokens per second = eval_count / (eval_duration / 1e9), straight from the response;
• meanwhile a background loop sampling power1_average and gpu_busy_percent every second;
• two runs per configuration.

Baseline at the stock 315 W cap: 71.6 t/s, draw pinned at 314 W, the GPU at 84–95 % busy while it ran.

Step 3 — the first cap costs nothing

The capped runs came back faster, within noise. The explanation is the workload note: token generation is bandwidth-bound, and 290 W was throttling nothing. The 315 W runs were hitting their limit at 314 W, but the card at 290 W simply drew less, ran a touch cooler, and lost none of the tokens.

Rule of thumb learned the hard way: benchmark before and after every change. The cap itself is one line; the measurement is the work.

Step 4 — the clock pin that would not take

To get below 283 W the clock ceiling has to come down, not the power budget. In the amdgpu driver the knob for that is pp_dpm_sclk:

sudo sh -c 'echo 0 > /sys/class/drm/card1/device/pp_dpm_sclk'
# sh: 1: echo: echo: I/O error

The first guess was a locked profile, so the power profile was switched to CUSTOM (pp_power_profile_mode, value 6) and the pin retried. Same error. Then catting the file three times in a row turned out to be revealing — the “table” is live:

0: 500Mhz     1: 1385Mhz *   2: 2431Mhz
0: 500Mhz     1: 1857Mhz *   2: 2431Mhz
S: 0Mhz *     1: 500Mhz      2: 2431Mhz

The starred line is just the card’s current core clock, moving with whatever load is running — the 0 MHz was the GPU at idle between samples. There are two real levels here (500 and 2,431 MHz) and the write was being refused.

Step 5 — reading the driver

The box runs a 7.0 kernel, where the driver’s power-management code has moved out of drivers/gpu/drm/amd/amdgpu/amdgpu_pm.c into a new drivers/gpu/drm/amd/pm/ directory — the first fetch 404’d because the file had relocated, and listing the old directory is how the new one was found.

Following the write through pm/amdgpu_pm.c:

The kernel parses the number as a mask of DPM levels and calls amdgpu_dpm_force_clock_level, which hands it to the SMU — the GPU’s own management firmware. This card’s firmware declines to force the low clock level, and the driver surfaces the refusal as an EIO. There is nothing to argue with: the sysfs file is a display and a request, and on this silicon the request gets a no.

Step 6 — the amd-smi detour

AMD’s userspace tool, amd-smi, is the proper way to pin clocks, so the install was attempted without root:

• the amdsmi package on PyPI turned out to be a pure-Python ctypes wrapper — it needs libamdsmi.so, which ships with ROCm, and the box has no ROCm;
• the apt repositories have no amdsmi package, and the standalone .deb repository paths that could be found all 404’d.

It was abandoned as a lost cause — which is what it would have been: amd-smi’s set-clock goes through the same driver API, and the same firmware would have said the same no.

Step 7 — what the power profile actually does

One knob remained: pp_power_profile_mode, stock value 0 (BOOTUP_DEFAULT), with 2 being POWER_SAVING. It was set to 2 and the benchmark rerun at the 290 W cap:

Identical. Of course: under full load the cap is the binding constraint, and the profile only changes the clock policy below the cap. What the profile did change was the light-load behaviour — with just the chat or a small job running, the core clock sat around 788 MHz instead of around 2,235. So the profile is a light-load saver and a generation no-op. It was left on.

The end state

31 W off the wall (10 %), about one t/s off (1.4 %), 8 °C off the junction.

Two notes on the temperature. The 91 °C stock runs were not thermally throttling — consumer RDNA3 starts protecting around 100–110 °C at the junction — which is why the t/s stayed flat: the clocks were never given anything to trade away. And the hottest thing on the card now is the VRAM, 86–90 °C under load (the GDDR6 limit is 105 °C). That is the number to watch if longer sustained loads ever come along.

Step 8 — making it survive a reboot

None of it persists: the cap and the profile revert to VBIOS defaults on a reboot. A small systemd oneshot reapplies both at boot. The trap is that hwmon5 and card1 are indices, and indices can shuffle between boots, so the script finds the card by identity rather than by number:

#!/bin/sh
# /usr/local/bin/gpu-power-limit.sh
ok=1
for h in /sys/class/hwmon/hwmon*; do
    [ "$(cat "$h/name" 2>/dev/null)" = "amdgpu" ] || continue
    if echo 283000000 > "$h/power1_cap" 2>/dev/null; then
        echo "gpu-power-limit: 283W cap applied ($h)"
    else
        echo "gpu-power-limit: FAILED to set cap on $h" >&2
        ok=0
    fi
done
for dev in /sys/class/drm/card[0-9]*/device; do
    [ -e "$dev/driver" ] || continue
    [ "$(basename "$(readlink -f "$dev/driver")")" = "amdgpu" ] || continue
    if echo 2 > "$dev/pp_power_profile_mode" 2>/dev/null; then
        echo "gpu-power-limit: POWER_SAVING profile applied ($dev)"
    else
        echo "gpu-power-limit: FAILED to set profile on $dev" >&2
        ok=0
    fi
done
[ "$ok" = "1" ]

# /etc/systemd/system/gpu-power-limit.service
[Unit]
Description=AMD GPU power limit (283W cap + POWER_SAVING profile)
After=multi-user.target

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/usr/local/bin/gpu-power-limit.sh
Restart=on-failure
RestartSec=10
StartLimitIntervalSec=600
StartLimitBurst=20

[Install]
WantedBy=multi-user.target

If the script exits non-zero — no amdgpu card found, or a write failed — systemd retries every 10 seconds for up to three minutes, which is slack enough for a slow driver. The first version of the script failed on exactly this: on this kernel the per-device hwmon entry resolves to a directory called hwmon, not hwmon5, so it tried to write /sys/class/hwmon/hwmon/power1_cap and the journal said Directory nonexistent. The journal told the truth and the retry loop meant it did not have to be caught live.

What this is not

It is not an undervolt. No voltage changed, and there was no way to change one: the firmware would not let the clocks be pinned, and the remaining lever for real voltage work is a modified VBIOS flash. That was not done — bricking a 24 GB card that is serving the house for roughly 30 watts is not a trade worth making.

The transferable finding is the shape of the workload. A bandwidth-bound model does not care about a power cap until the cap bites, and once it bites it barely cares about it. The saving was heat and wall watts, not speed. The same 31 watts on a compute-bound workload would have shown up straight in the t/s.

Manual override at any time:

sudo sh -c 'echo 315000000 > /sys/class/hwmon/hwmon5/power1_cap'   # back to stock