{"model":"deepseek-4.1-flash","title":"Corrected decode benchmark (ignore_eos) - important","status":"negative","summary":"Earlier per-concurrency decode numbers were EOS-truncation artifacts (no ignore_eos -\u003e early stop, per-request overhead dominated). Corrected, ignore_eos + cached shared prefix: 4k c32 302.8, 128k c32 214.8, 512k c64 129.6; single-stream ~105 tok/s pure decode. The 500 @512k target is a…","occurred_at":"2026-09-11T18:22:00Z","config":{"serving":"TP4xPP2, DSpark k=5, fp8_ds_mla, 1M ctx, CUDA graphs","engine":"localhost/vllm-backport-v41:sm80 (vLLM 0.12.0-sm80 + PR#56201 + Ampere shims + PP relay)"},"benchmarks":[{"metric":"agg_decode_4k_tok_s","value":302.8,"unit":"tok/s","context":"4k ctx c32 (corrected)","note":"corrected upward from understated numbers"},{"metric":"agg_decode_128k_tok_s","value":214.8,"unit":"tok/s","context":"128k ctx c32 (corrected)"},{"metric":"agg_decode_512k_tok_s","value":129.6,"unit":"tok/s","context":"512k ctx c64 (corrected)"},{"metric":"c1_decode_tok_s","value":105,"unit":"tok/s","context":"pure decode, corrected (earlier '43' was an artifact)"}]}