Corrected decode benchmark (ignore_eos) - important

Earlier per-concurrency decode numbers were EOS-truncation artifacts (no ignore_eos -> early stop, per-request overhead dominated). Corrected, ignore_eos + cached shared prefix: 4k c32 302.8, 128k c32 214.8, 512k c64 129.6; single-stream ~105 tok/s pure decode. The 500 @512k target is a…

When (PT)2026-09-11 11:22 PT

ServingTP4xPP2, DSpark k=5, fp8_ds_mla, 1M ctx, CUDA graphs

Enginelocalhost/vllm-backport-v41:sm80 (vLLM 0.12.0-sm80 + PR#56201 + Ampere shims + PP relay)

Benchmarks

Metric	Value	Δ vs previous	Unit	Context	Note
`agg_decode_128k_tok_s` (agg_decode_128k_tok_s)	214.8 tok/s	0.0%	tok/s	128k ctx c32 (corrected)
`agg_decode_4k_tok_s` (agg_decode_4k_tok_s)	302.8 tok/s	first point	tok/s	4k ctx c32 (corrected)	corrected upward from understated numbers
`agg_decode_512k_tok_s` (agg_decode_512k_tok_s)	129.6 tok/s	0.0%	tok/s	512k ctx c64 (corrected)
`c1_decode_tok_s` (c1_decode_tok_s)	105 tok/s	143.6%	tok/s	pure decode, corrected (earlier '43' was an artifact)

raw JSON: /api/reports/18