vllm-loop model-improvement progress tracker junk-tokens
models / / #29
▲ MADE POSITIVE PROGRESS

PP6 sweep completed to the full 1M context

Prefix-cached decode aggregate: 4k 49/91/109/146/142 (c1/c2/c4/c8/c16), 32k 45/81/104/159/160, 128k 43/66/100/165/150, 512k 30/46/54/85/88, 1M 16/22 (only c1/c2 fit the 2.89M pool). DSpark acceptance rises to 51.7% @1M. Prefill: 1M c1 TTFT 342.5 s (~3.1k tok/s).

When (PT)2026-09-12 00:47 PT
ServingTP1xPP6 (7,7,7,7,7,5), util 0.95-0.96, DSpark k=5, fp8_ds_mla
Enginelocalhost/vllm-backport-v41:sm80 Schaka v0.13.0 kit (c1b0907b overlay)
KV

Benchmarks

MetricValueΔ vs previousUnitContextNote
agg_decode_1m_tok_s (agg_decode_1m_tok_s) 16 tok/s first point tok/s PP6 c1 @1M (22 @c2)
agg_decode_32k_tok_s (agg_decode_32k_tok_s) 159 tok/s first point tok/s PP6 c8 @32k (160 @c16)
agg_decode_512k_tok_s (agg_decode_512k_tok_s) 85 tok/s -2.1% tok/s PP6 c8 @512k (88 @c16)
prefill_tok_s (prefill_tok_s) 3100 tok/s -87.0% tok/s 1M ctx c1 TTFT 342.5 s

raw JSON: /api/reports/29