- Checkpoint
- unsloth/Qwen3.8-27B-NVFP4
- Hardware
- NVIDIA GB10 Grace Blackwell, 121.7 GB unified — dgx2 (spark-43fa)
- Workload
- ISL 128 / OSL 1024,
3 timed reps + 1 warmup, temperature 0, seed 42
- Parity
- disabled on both engines via chat_template_kwargs.enable_thinking=false; presence_penalty and frequency_penalty pinned to 0.0 on both engines
Atlas Atlas 1.0.0-beta-preview
Round 11 throughout, except C=2. C=2 is the 2026-08-18 hardening run (c2_atlas_dgx2_20260818.json), not round 8: the certified table originally carried round 8's 38.95 there, which won by 1.004x -- a margin inside either engine's run-to-run spread. Both legs were re-measured back to back on dgx2 on 2026-08-18 (Atlas 15:05, vLLM+MTP 15:20) and that PAIR replaced the earlier pair, which is why the Atlas number rose to 41.02 while the paired vLLM number fell to 37.11 and the margin moved to 1.105x with non-overlapping distributions. A round-11 C=2 exists at 40.42 (l38_r11_c2.json) but has no same-day vLLM leg, so it is not publishable as a pair. See RESULTS.md, C=2 HARDENED.
Buildbce152b18
Certified at 4012c9b7e1, which differs only in doc comments and gate machinery -- no executable change. Merged to main as 60370b9532.
SpeculationMTP, K=4 (--num-drafts 3); self-disables above 32 concurrent sequences
EnvATLAS_PREFILL_CODISPATCH=1 ATLAS_FP8_ROWWISE=1 ATLAS_MTP_DCUT_RATIO=1.0 ATLAS_MTP_K_LADDER=1:3,2:1,4:2,8:2,16:1
Commandspark serve unsloth/Qwen3.8-27B-NVFP4 --host 0.0.0.0 --port 8888 --model-name unsloth/Qwen3.8-27B-NVFP4 --max-seq-len 2048 --max-batch-size 128 --gpu-memory-utilization 0.85 --kv-cache-dtype fp8 --enable-prefix-caching true --ssm-cache-slots 8 --ssm-checkpoint-interval 32 --speculative --num-drafts 3 --mtp-quantization bf16 --scheduling-policy fifo --tool-call-parser qwen3_coder --disable-tool-grammar true --disable-thinking --request-timeout 0 --ssm-h-dtype f16-pool --gdn-fused-norm --ssm-batched-recurrent --ssm-tail-midchunk false --mtp-gate force --prefill-varlen-batch --no-tui
vLLM + MTP vLLM 0.27.1
The apples-to-apples reference: ctx 2048, batch cap 128, util 0.85, fp8 KV, prefix caching, thinking off and MTP K=4 -- every axis matched to Atlas. vLLM at its best, not a handicapped baseline.
Buildvllm/vllm-openai:latest @ sha256:0a51ea5b4ae2dc5d81890e5173f54203d2a3ae0cfffe51b8fd2afd4391bfd967
SpeculationvLLM's own Qwen3_5MTP, K=4 (num_speculative_tokens=3)
EnvHF_HUB_OFFLINE=1
Commandvllm serve --model unsloth/Qwen3.8-27B-NVFP4 --served-model-name unsloth/Qwen3.8-27B-NVFP4 --host 0.0.0.0 --port 8001 --max-model-len 2048 --max-num-seqs 128 --gpu-memory-utilization 0.85 --enable-prefix-caching --dtype bfloat16 --kv-cache-dtype fp8 --speculative-config '{"method":"mtp","num_speculative_tokens":3}'
vLLM, no speculation vLLM 0.27.1
Not matched to Metrale: bf16 KV cache (Atlas and the MTP leg use fp8); ctx 4096 (the matched legs use 2048); no speculative decoding. Shown for completeness. It is not the published denominator.
Unmatched fingerprint: bf16 KV, ctx 4096, no speculation. It beats vLLM+MTP at C=128 (390.42 vs 358.57) because MTP verification costs vLLM more than it gains 128-wide. Plotted for completeness. The published ratio uses the matched MTP leg, not this one.
Buildvllm/vllm-openai:latest @ sha256:0a51ea5b4ae2dc5d81890e5173f54203d2a3ae0cfffe51b8fd2afd4391bfd967
Speculationnone
EnvHF_HUB_OFFLINE=1
Commandvllm serve --model unsloth/Qwen3.8-27B-NVFP4 --served-model-name unsloth/Qwen3.8-27B-NVFP4 --host 0.0.0.0 --port 8001 --max-model-len 4096 --max-num-seqs 128 --gpu-memory-utilization 0.85 --enable-prefix-caching --dtype bfloat16 --kv-cache-dtype bfloat16
Per-rung detail
| Series | C | tok/s | median | spread | TTFT p50 | TPOT p50 | source file |
|---|
| Atlas | 1 | 23.59 | 23.55 | 2.78% | 682.54 ms | 41.84 ms | l38_r11_c1.json |
| Atlas | 2 | 41.02 | 41.37 | 4.15% | 795.2 ms | 47.53 ms | c2_atlas_dgx2_20260818.json |
| Atlas | 4 | 74.21 | 74.29 | 2.79% | 1296.3 ms | 51.76 ms | l38_r11_c4.json |
| Atlas | 8 | 125.95 | 125.83 | 2.92% | 2331.47 ms | 59.01 ms | l38_r11_c8.json |
| Atlas | 16 | 203.36 | 203.45 | 1.23% | 4252.58 ms | 72.67 ms | l38_r11_c16.json |
| Atlas | 32 | 291.01 | 292.50 | 2.43% | 8281.81 ms | 98.91 ms | l38_r11_c32.json |
| Atlas | 64 | 386.63 | 387.48 | 1.68% | 16052.93 ms | 148.81 ms | l38_r11_c64.json |
| Atlas | 128 | 478.11 | 480.05 | 2.05% | 31760.85 ms | 234.6 ms | l38_r11_c128.json |
| vLLM + MTP | 1 | 19.72 | 19.72 | 1.89% | 286.71 ms | 50.37 ms | vllm_fp8_mtp_reference.json |
| vLLM + MTP | 2 | 37.11 | 37.18 | 2.94% | 390.84 ms | 53.22 ms | c2_vllm_mtp_dgx2_20260818.json |
| vLLM + MTP | 4 | 71.61 | 72.05 | 2.82% | 604.84 ms | 54.51 ms | vllm_fp8_mtp_reference.json |
| vLLM + MTP | 8 | 124.48 | 124.31 | 1.25% | 936.14 ms | 60.8 ms | vllm_fp8_mtp_reference.json |
| vLLM + MTP | 16 | 197.03 | 197.85 | 3.51% | 1620.35 ms | 76.25 ms | vllm_fp8_mtp_reference.json |
| vLLM + MTP | 32 | 283.48 | 284.20 | 1.27% | 3066.96 ms | 107.45 ms | vllm_fp8_mtp_reference.json |
| vLLM + MTP | 64 | 361.39 | 361.18 | 0.21% | 5278.74 ms | 167.48 ms | vllm_fp8_mtp_reference.json |
| vLLM + MTP | 128 | 358.57 | 358.61 | 0.44% | 8500.7 ms | 269.54 ms | vllm_fp8_mtp_reference.json |
| vLLM, no speculation | 1 | 11.04 | 11.05 | 0.32% | 155.95 ms | 90.4 ms | vllm_latest_reference.json |
| vLLM, no speculation | 2 | 21.34 | 21.34 | 0.04% | 235.35 ms | 93.52 ms | vllm_latest_reference.json |
| vLLM, no speculation | 4 | 41.20 | 41.20 | 0.03% | 419.59 ms | 96.77 ms | vllm_latest_reference.json |
| vLLM, no speculation | 8 | 78.18 | 78.18 | 0.02% | 729.44 ms | 101.71 ms | vllm_latest_reference.json |
| vLLM, no speculation | 16 | 137.11 | 137.11 | 0.04% | 1176.06 ms | 115.56 ms | vllm_latest_reference.json |
| vLLM, no speculation | 32 | 219.50 | 219.45 | 0.09% | 2240.16 ms | 143.58 ms | vllm_latest_reference.json |
| vLLM, no speculation | 64 | 312.26 | 312.31 | 0.12% | 4009.21 ms | 200.87 ms | vllm_latest_reference.json |
| vLLM, no speculation | 128 | 390.42 | 390.15 | 0.31% | 6710.78 ms | 320.84 ms | vllm_latest_reference.json |
Harness bench/ladder38/harness_w55_conc_ladder.py. Two harness revisions appear above: 6412b12d4d — vLLM legs. Does not send the penalty keys.; 1c77e1d8e9 — Atlas legs (rounds 7-11). Adds explicit presence_penalty/frequency_penalty = 0.0. The only difference is those two keys. vLLM already defaults both to 0.0, so its sampling is byte-identical either way; the keys exist to stop Atlas's non_thinking preset injecting presence_penalty=1.5. Both engines therefore ran identical sampling.
Full campaign log, including every rung we lost on the way and the three claims we retracted: bench/ladder38/RESULTS.md. Generated 2026-09-16T23:47:53Z from the committed measurements.