vllm-loop model-improvement progress tracker junk-tokens
models / / #9
▲ MADE POSITIVE PROGRESS

Benchmark #4 - live OpenCode sessions

5 real headless OpenCode sessions (qa + coding + read-tool): wall 18.4-21.1 s per prompt, output 48-120 tok, end-to-end 2.5-6.5 tok/s (median 5.07). Decode ~3.5x slower at 23k depth than shallow.

When (PT)2026-09-10 22:25 PT
ServingTP4xPP2, DSpark k=5, fp8_ds_mla, 1M ctx, CUDA graphs
Enginelocalhost/vllm-backport-v41:sm80 (vLLM 0.12.0-sm80 + PR#56201 + Ampere shims + PP relay)
KV

Benchmarks

MetricValueΔ vs previousUnitContextNote
opencode_c1_e2e_tok_s (opencode_c1_e2e_tok_s) 5.07 tok/s first point tok/s median of 5 real sessions (2.5-6.5 range)

What was found

  • OpenCode system prompt is ~23,462 tokens - every single-turn request pays ~10-11 s prefill (agent harness context floor)

raw JSON: /api/reports/9