Every number is a receipt.

Metrale Engine Metrale Engine 1.0.0-beta-preview

Round 11 throughout, except C=2. C=2 is the 2026-08-18 hardening run (c2_metrale_dgx2_20260818.json), not round 8: the certified table originally carried round 8's 38.95 there, which won by 1.004x -- a margin inside either engine's run-to-run spread. Both legs were re-measured back to back on dgx2 on 2026-08-18 (Metrale Engine 15:05, vLLM+MTP 15:20) and that PAIR replaced the earlier pair, which is why the Metrale Engine number rose to 41.02 while the paired vLLM number fell to 37.11 and the margin moved to 1.105x with non-overlapping distributions. A round-11 C=2 exists at 40.42 (l38_r11_c2.json) but has no same-day vLLM leg, so it is not publishable as a pair. See RESULTS.md, C=2 HARDENED.

Certified at 4012c9b7e1, which differs only in doc comments and gate machinery -- no executable change. Merged to main as 60370b9532.

vLLM + MTP vLLM 0.27.1

The apples-to-apples reference: ctx 2048, batch cap 128, util 0.85, fp8 KV, prefix caching, thinking off and MTP K=4 -- every axis matched to Metrale Engine. vLLM at its best, not a handicapped baseline.

vLLM, no speculation vLLM 0.27.1

Not matched to Metrale Engine: bf16 KV cache (Metrale Engine and the MTP leg use fp8); ctx 4096 (the matched legs use 2048); no speculative decoding. Shown for completeness. It is not the published denominator.

Unmatched fingerprint: bf16 KV, ctx 4096, no speculation. It beats vLLM+MTP at C=128 (390.42 vs 358.57) because MTP verification costs vLLM more than it gains 128-wide. Plotted for completeness. The published ratio uses the matched MTP leg, not this one.

Per-rung detail

ITL is inter-token latency as AIPerf defines it — (request_latency − TTFT) / (OSL − 1), the numerator ending at the final response chunk. It is the same quantity usually called TPOT; the record keys keep that name (tpot_p50_ms) because renaming a recorded key would orphan every measurement already committed.

Harness bench/ladder38/harness_w55_conc_ladder.py. 5 harness revisions appear above: 6412b12d4d — vLLM legs. Does not send the penalty keys.; 1c77e1d8e9 — Metrale Engine legs (rounds 7-11). Adds explicit presence_penalty/frequency_penalty = 0.0.; 1f10d4887b — The 2026-08-18 C=2 hardening pair (c2_metrale_dgx2_20260818.json and c2_vllm_mtp_dgx2_20260818.json). Both raw files carry this one driver sha, which is what makes them a pair: same harness revision, same day, same box.; 664302439b — The ENERGY leg (vllm-mtp-energy), 2026-09-21. harness_w56_conc_ladder.py -- the w55 harness plus power_window.py, which is the half that records gpu_rail_energy_j; w55 has no power sampling at all, which is why the throughput-only legs above carry no joules. This is the sha256 of the copy committed at bench/ladder38/harness_w56_conc_ladder.py and asserted equal to the raw file's driver_sha256 at the time it was added, so the two are checkable against each other -- and gen-ladder.mjs re-hashes that file at every build as harness_repo_sha256.; 55a5963e4b — The ENERGY leg's wide rungs (vllm-mtp-energy C=32/64/128), 2026-09-28: the same harness_w56_conc_ladder.py at a later revision, the copy committed at bench/ladder38/harness_w56_conc_ladder.py since 7cd54d45. Each raw file's driver_sha256 equals it (the harness hashes itself at startup). It differs from 664302439b in comments and in detecting in-band SSE error frames and truncated streams (both turn a would-be silent success into a counted error), not in the request body; power_window.py, which does the integral, differs only in comments. The only difference is those two keys. vLLM already defaults both to 0.0, so its sampling is byte-identical either way; the keys exist to stop Metrale Engine's non_thinking preset injecting presence_penalty=1.5. Both engines therefore ran identical sampling.

Full campaign log, including every rung we lost on the way and the three claims we retracted: bench/ladder38/RESULTS.md. Generated 2026-09-28T22:12:33Z from the committed measurements.