▲ MADE POSITIVE PROGRESS

Benchmark #1 - baseline (random 1024/256)

First local benchmark on our box, TP4xPP2, random 1024-in/256-out, server-counted: c1 21.71 out tok/s (TPOT 43.0 ms, DSpark acceptance 6.80% on random tokens), c8 74.60 (peak 80.0). TTFT includes per-shape JIT warmup.

When (PT)2026-09-10 20:24 PT

ServingTP4xPP2, DSpark k=5, fp8_ds_mla, 1M ctx, CUDA graphs

Enginelocalhost/vllm-backport-v41:sm80 (vLLM 0.12.0-sm80 + PR#56201 + Ampere shims + PP relay)

Benchmarks

Metric	Value	Δ vs previous	Unit	Context	Note
`agg_decode_c8_tok_s` (agg_decode_c8_tok_s)	74.6 tok/s	-28.4%	tok/s	random 1024-in/256-out, TP4xPP2 (peak 80.0)
`c1_decode_tok_s` (c1_decode_tok_s)	21.71 tok/s	-50.4%	tok/s	random 1024-in/256-out, TP4xPP2	first local measurement
`dspark_accept_pct` (dspark_accept_pct)	6.8 %	first point	%	random tokens - misleadingly low on random data

raw JSON: /api/reports/6