KV accounting measured (PP8)

Zanooda PP8, fp8_ds_mla: 6,170,576-token KV pool in 135.72 GiB total (per-rank 6.48-20.95 GiB) -> 23,617 B/token whole-model; 1M tokens ~23.06 GiB (~26x the model card's 890 B/tok) because fp8_ds_mla keeps uncompressed indexer-K caches. Goal reprioritized to PP8/PP6 only.

When (PT)2026-09-11 21:22 PT

ServingTP1xPP8 (Zanooda 13-patch stack), fp8_ds_mla

Enginezanooda/vllm-sm80-ds41f:v41-sm80 + KV-layout intersection fix

Benchmarks

Metric	Value	Δ vs previous	Unit	Context	Note
`kv_gib_per_1m` (kv_gib_per_1m)	23.06 GiB	first point	GiB	PP8 (23,617 B/token) - vs 10.95 GiB on PP6
`kv_pool_tokens` (kv_pool_tokens)	6170576 tokens	-56.1%	tokens	PP8 pool (5.88x concurrency @1M)

raw JSON: /api/reports/24