What prefill costs when the weights do not fit

Every measured prefill in the sybil-solutions campaigns, with the storage tier that served it. 517 rows from 34 datasets, 12 models. Generated from data/prefill.jsonl; every row cites the file it came from.

1. The same prompt, four storage tiers

GLM-5.3-Flash EXL3 3.05 bpw on one RTX 3090, campaign N129, 2026-10-08. Identical image, identical settings, only the store changes. The 7 GB/s arms are the same four-drive array with its bandwidth capped to one drive's, so they separate bandwidth from drive count.

8k prompt, C1. TTFT = 8192 / prefill tok/s. I/O columns are the prefill phase's device counters.

2. What the next thousand prompt tokens cost

Prefill speed is not constant in prompt length: the first chunk pays for warm-up, the marginal chunks run near the GPU's own rate. Marginal tok/s is the speed of the tokens added over the row above, computed from the TTFT column.

DeepSeek-V4.1-Flash

EXL3 3.0 bpw on one 3090, experts pinned in host RAM, run D119

Qwen3.8-Flash-Next

EXL3 3.05 bpw on one 3090 whose experts live on a Mac mini's SSD over USB4, config C

3. What the wider set adds

Thirty-four datasets, most of them one host and one protocol each. These bullets are the fastest single-stream prefill in each of the ones worth singling out.

• 7,817 tok/s at 32k tokens on DeepSeek-V4.1-Flash with the Engram tables on local NVMe (deepseek-v4.1-flash-4x-rtx-pro-6000, 4x RTX PRO 6000). That repo's metric includes queueing, so treat it as an upper bound; it still reports 5,845 tok/s at 500k.
• 4,397 tok/s at 32k, and flat from 8k to 64k (4,143-4,397), for Qwen3.8-Flash-Next MXFP4-FP8-GPTQ on 2x R9700 with the experts in host RAM (local-ai-recipe-kit-submissions-2x-r9700-32gb, a community submission to the kit).
• 2,664 tok/s at 106,294 tokens for Qwen3.8-Flash-Next Q2_0 on one RTX 3090 reading a Strata NVMe expert pack (local-ai-registry-gates-rtx-3090-24gb). The IQ3_S pack on the same prompt measures 2,174.
• 2,421 tok/s at ~4k for Qwen3.8-27B EXL3 on one Arc Pro B70 with W8A8 int8 prefill linears, against 1,666 for the same card without them (trellis-serve-xpu-b70-int8 vs trellis-serve-xpu-b70). On a VRAM-resident model, prefill is a kernel problem, not a storage one.
• 1,015 tok/s at 981,841 tokens, 967 s to first token, for GLM-5.3-Flash on 2x DGX Spark (glm53-tensorfold-2x-spark). The only sub-million-token prefill in the set, and the clearest picture of what a million-token prompt costs.
• 70 tok/s at 106k tokens on a Radeon 8060S iGPU (local-ai-registry-gates-radeon-8060s-64gb), against 2,664 for the 3090 above. Same prompt shape, 38x apart.

All datasets

4. Caveats

• Prefill numbers are only comparable inside one dataset: protocols differ (sweep medians vs one request per length), and so do hosts.
• Arms of one campaign often ran hours apart. On this host that drifts up to ~17 %, so a few-percent gap between arms in different sessions is not a result.
• transcribed rows were read off a Markdown table by hand; everything else is machine-read from the source repo's JSON.
• Time to first token marked with an asterisk is computed as prompt tokens / prefill tok/s, because the source reports only the rate.

5. Reproduce

git clone https://github.com/sybil-solutions/prefill-cost && cd prefill-cost
git clone --depth 1 https://github.com/sybil-solutions/moetier ../moetier && git clone --depth 1 https://github.com/sybil-solutions/glm-flash-lite ../glm-flash-lite && git clone --depth 1 https://github.com/sybil-solutions/dsv41-flash-offload ../dsv41-flash-offload && git clone --depth 1 https://github.com/sybil-solutions/qwen38-flash-next-3090-usb-offload ../qwen38-flash-next-3090-usb-offload
python3 tools/extract.py --sources ..   # staging rows
python3 tools/merge.py                  # validate + data/prefill.jsonl
python3 tools/render.py                 # README tables + docs/index.html

Provenance