Perf finding - the prefill bottleneck is the TP all-reduce

Got Schaka's TP1xPP8 profile booting (2 patches: intersect per-rank KV layouts; guard relay warmup index-K write): PP8 prefill 5,738 tok/s @32k / 8,264 @128k = ~3.7x TP4xPP2 (link-bound all-reduce over Gen2 x4) - but the relay was numerically wrong, reverted.

When (PT)2026-09-11 06:32 PT

ServingTP4xPP2, DSpark k=5, fp8_ds_mla, 1M ctx, CUDA graphs

Enginelocalhost/vllm-backport-v41:sm80 (vLLM 0.12.0-sm80 + PR#56201 + Ampere shims + PP relay)

Benchmarks

Metric	Value	Δ vs previous	Unit	Context	Note
`prefill_tok_s` (prefill_tok_s)	5738 tok/s	256.8%	tok/s	TP1xPP8 @32k (~3.7x TP4xPP2)
`prefill_tok_s` (prefill_tok_s)	8264 tok/s	413.9%	tok/s	TP1xPP8 @128k

What didn't work

PP8 accuracy gate rejected it: needle 0/3, coding 1/5 with garbled identifiers - relay does not reproduce indexer-K cache write on every stage

raw JSON: /api/reports/16