deepseek-4.1-flash
▼ MADE NEGATIVE PROGRESS
Tested prefill-chunk size as the ITL-tail fix: MAXBATCH=8192 collapses KV (7.86M->3.23M) and fails c1; MAXBATCH=1024 is rejected by the multimodal min. No free ITL fix; baseline restored (2026-09-12 13:02 PT)
Benchmark progress over time
One chart per metric. Hover a point for date (PT), value, and what happened; click to open the report.
Status log
| When (PT) | Status | Summary | |
|---|---|---|---|
| 2026-09-12 13:02 PT | ▼ negative | Tested prefill-chunk size as the ITL-tail fix: MAXBATCH=8192 collapses KV (7.86M->3.23M) and fails c1; MAXBATCH=1024 is rejected by the multimodal min. No free ITL fix; baseline restored | report → |
| 2026-09-12 12:33 PT | ▲ positive | Built the one-command 32K master table; PP8 baseline c1 20.6 / c4 53.4 / c8 78.9 Gen tok/s, 512K@c16 98.9 tok/s; util 0.96 OOMs 512K so lowered to 0.94 | report → |
| 2026-09-12 10:46 PT | ▲ positive | The layout-registration hazard AGENTS.md warns about works as designed on heterogeneous PP stages: LMCache group_tokens_per_block resolved per cache group, KV cache group edits applied, non-prefix-cacheable engine group excluded - hourly #17. | report → |
| 2026-09-12 10:06 PT | ● stuck | VLLM_SPARSE_DECODE_MAXNREG is dead code (_decode_maxnreg_kwargs has no caller) - the earlier MAXNREG test measured nothing. Wired real knobs and swept on PP8/512k: HEAD_BLOCK_SIZE=32 gives consistent, bit-identical +2-4% (c1 53.0->54.3, 512k c8 45.7->47.4, c32 60.2->61.6) - folded in as PP8… | report → |
| 2026-09-12 08:18 PT | ▲ positive | Layout-resolver intersection fix makes PP8 work: 9.35M-token KV pool (2.75x PP6), 8.9x concurrency @1M; 512k aggregate still ~60 tok/s @c32 -> step-cost bound. Leads: vLLM #56120 (SM80 Triton knobs), SGLang #38646 (NVFP4 sparse-MLA, 384 B/tok, SM120-only). | report → |
| 2026-09-12 05:50 PT | ▲ positive | Shared-487k-prefix pure decode (server-counted): c1 25.1 / c2 38.1 / c4 44.1 / c8 52.3 / c12 54.6 aggregate - saturates from c8 (per-step cost grows ~linearly with batch at depth). 500 tok/s @512k is ~9x away and blocked by depth step-cost scaling + KV-capped concurrency: kernel/engine work, not… | report → |
| 2026-09-12 04:16 PT | ▼ negative | Plain PP6 65.1 tok/s (512-tok gen, thinking off) vs PP6 + LMCacheMPConnector 58.0 - the LMCache path costs ~11% c1 decode; real-OpenCode c1 median drops ~30 -> 26.5. Decision: plain PP6 stays the c1 served default; LMCache is a validated optional arm for cross-restart/evicted-prefix reuse (vLLM… | report → |
| 2026-09-12 03:56 PT | ▲ positive | In-process LMCacheConnectorV1 is single-group by construction -> HMA disabled -> vLLM cannot unify V4.1 KV specs (MLA state_content_bytes=584 + SWA compressor group) and startup fails. Correct path: LMCacheMPConnector (group-aware, SupportsHMA) + lmcache MP server WITH GPU access +… | report → |
| 2026-09-12 01:29 PT | ▲ positive | nvidia-smi dmon during sustained PP6 c1 decode: SM ~100% but memory-controller only 4-5% -> c1 is SM-occupancy/latency-bound, not DRAM- or power-bound. | report → |
| 2026-09-12 00:47 PT | ▲ positive | Prefix-cached decode aggregate: 4k 49/91/109/146/142 (c1/c2/c4/c8/c16), 32k 45/81/104/159/160, 128k 43/66/100/165/150, 512k 30/46/54/85/88, 1M 16/22 (only c1/c2 fit the 2.89M pool). DSpark acceptance rises to 51.7% @1M. Prefill: 1M c1 TTFT 342.5 s (~3.1k tok/s). | report → |
| 2026-09-12 00:33 PT | ▲ positive | Rebuilt the PP6 overlay from Schaka's latest kit: the kv-group relay now owns the source indexer K cache (layers.20.attn.indexer.k_cache), fixing his prompt-corruption path (44k check 5/10 -> 0/10 wrong). Ours: coherence OK, needle 128k 3/3, multi-needle 44k 5/5, coding 5/5; pure c1… | report → |
| 2026-09-11 23:22 PT | ▲ positive | PP6 (TP1xPP6, partition 7,7,7,7,7,5, util 0.95) now served: c1 96.9 best-of-4 (107.6 with 93.8% DSpark acceptance) - within noise of TP8xPP1 (100.8), ~2.3x faster than PP8; needle 6/6 + 512k 3/3, coding 5/5; prefill 128k c1 7,716 / c16 23,780; decode 128k c8 164, 512k c8 85. | report → |
| 2026-09-11 23:16 PT | ▲ positive | Both accuracy-gated. PP6: c1 96.9 (8-tok) / 103.8 @4k / 49.4 @32k vs PP8 43.2 / 32.2 / 27.6; KV 10.95 GiB per 1M (PP8 23.06); 512k c16 88.3 (86.8); decode agg 128k c8 164 (128). PP8 only wins total pool (6.17M vs 2.89M tokens) and some prefill. PP8 is the slowest PP option at c1. | report → |
| 2026-09-11 22:04 PT | ▲ positive | PP8 (TP1xPP8) on our box: single-stream 43.2/40.9/32.2/27.6 @8/512/4k/32k; aggregate 4k c8 154.9, 32k c16 166.6, 128k c16 139.0, 512k c16 86.8; prefill 4k c1 1350 / c16 7067, 128k c16 30058. Conflict resolved by measurement: PP8 is ~2.3x SLOWER at c1 than TP8xPP1 (43 vs 100.8) - c1 -> TP8xPP1… | report → |
| 2026-09-11 21:22 PT | ▲ positive | Zanooda PP8, fp8_ds_mla: 6,170,576-token KV pool in 135.72 GiB total (per-rank 6.48-20.95 GiB) -> 23,617 B/token whole-model; 1M tokens ~23.06 GiB (~26x the model card's 890 B/tok) because fp8_ds_mla keeps uncompressed indexer-K caches. Goal reprioritized to PP8/PP6 only. | report → |
| 2026-09-11 19:21 PT | ▲ positive | Published measured layout comparison: PP layouts prefill ~2x (no all-reduce; Gen2 x4 caps TP4 prefill ~1,700 tok/s) but lose deep decode vs TP4xPP2 (92 vs 130 @c64/512k). Third-party PP8 claims (117 single / 532 agg) use tokens/(last-first) + staggered arrivals and read higher than wall-clock. | report → |
| 2026-09-11 16:07 PT | ▲ positive | Forked wtdcode/vllm-backport -> roboalchemist/vllm-backport and pushed our work as branch cmp170hx-dsv41-flash. | report → |
| 2026-09-11 15:14 PT | ▲ positive | TP8xPP1 c1 ~100-103 tok/s (measured, best-of-N) vs shi3z 8x A100 sm80 52 tok/s and zebgop-ops 4xCMP170HX PP4 16.7 tok/s (both author-reported). Clock-lock is a no-op (CMP clamps); moe_backend=triton fails on MXFP4/SM80; DSpark k=5 pinned by checkpoint. Remaining c1 headroom is kernel-level. | report → |
| 2026-09-11 13:34 PT | ▲ positive | Single-stream TP8xPP1 (zero pipeline hops): 98.1/95.0/92.6/88.4 tok/s @8/512/4k/32k vs TP4xPP2 89.4/84.2/87.6/54.1 - decisive at 32k. Accuracy gates pass (needle 4/4, coding 5/5). Tradeoff: 512k aggregate 113 vs 130. TP8xPP1 becomes the served config for c1-first. | report → |
| 2026-09-11 11:35 PT | ▼ negative | Correction: the earlier 'flat TPOT' note read the aggregate field. True per-stream: 4k 106 ms | 128k 150 ms | 512k c32 294 ms | 512k c64 504 ms (~5x). 500 tok/s @512k needs ~5x lower per-token cost at depth or ~4x concurrency headroom. Upstream: #46994 MERGED (PP MTP + stale topk_indices_buffer… | report → |
| 2026-09-11 11:22 PT | ▼ negative | Earlier per-concurrency decode numbers were EOS-truncation artifacts (no ignore_eos -> early stop, per-request overhead dominated). Corrected, ignore_eos + cached shared prefix: 4k c32 302.8, 128k c32 214.8, 512k c64 129.6; single-stream ~105 tok/s pure decode. The 500 @512k target is a… | report → |
| 2026-09-11 08:09 PT | ▲ positive | Adopted Schaka's relay fix (topk_indices on the hop + index_k_cache guard): TP1xPP6 prefill 5.6k @32k / 7.2k @128k. TP4xPP2 remains decode-optimal (512k c64 129.6, correct 12/12). TP2xPP4 / TP1xPP8 still boot but needle 0/3 (relay cannot handle TP splits) - rejected. | report → |
| 2026-09-11 06:32 PT | ▲ positive | Got Schaka's TP1xPP8 profile booting (2 patches: intersect per-rank KV layouts; guard relay warmup index-K write): PP8 prefill 5,738 tok/s @32k / 8,264 @128k = ~3.7x TP4xPP2 (link-bound all-reduce over Gen2 x4) - but the relay was numerically wrong, reverted. | report → |
| 2026-09-11 04:19 PT | ▲ positive | OpenCode driven through Bifrost (bifrost-dsv41/dsv41-flash-cmp at bifrost.roboalch.com/v1) returned exactly BIFROST-OPENCODE-OK - direct and gateway access paths both live. | report → |
| 2026-09-11 02:22 PT | ▲ positive | 8-turn accumulating conversation (~13k tokens): fact planted turn 1 recalled exactly on turn 9 (MULTITURN-885440-K); recall turn 1.4 s via prefix cache. | report → |
| 2026-09-11 02:21 PT | ▲ positive | Needle planted at 50% depth in a 1,048,576-token prompt recalled exactly: NEEDLE-885440-X (wall 1113 s, prefill-dominated). The service genuinely uses its advertised context. | report → |
| 2026-09-11 01:24 PT | ▲ positive | Service HEALTHY (TP4xPP2, DSpark k=5, 1M ctx, CUDA graphs on, EP on): long-context recall 12/12 (4k-512k, 15/50/85% depth), coding 4/5 executed, Bifrost route dsv41-flash-cmp live. | report → |
| 2026-09-11 00:37 PT | ▲ positive | Interactive TUI session (not headless) verified exact-token replies - realness gate evidence for c1 claims. | report → |
| 2026-09-10 23:13 PT | ▲ positive | Shared-prefix + prefix-caching + ignore_eos, TP4xPP2: per-token decode FLAT with depth (TPOT 8-10 ms @512k, sparse CSA2), but aggregate saturates: 128k c32 214.8, 512k c64 129.6 (26% of the 500 goal). DSpark acceptance ~42% at depth. Cold 512k prefill 326 s (1,608 tok/s). | report → |
| 2026-09-10 22:25 PT | ▲ positive | 5 real headless OpenCode sessions (qa + coding + read-tool): wall 18.4-21.1 s per prompt, output 48-120 tok, end-to-end 2.5-6.5 tok/s (median 5.07). Decode ~3.5x slower at 23k depth than shallow. | report → |
| 2026-09-10 22:22 PT | ▲ positive | Random-token acceptance (4-8%) was misleading. Real prose (TP4xPP2, k=5): c1 43.1 tok/s (accept 41.2%), c4 111.0, c8 153.4 - c1 matches Schaka's 43.8. Arrangements: TP8xPP1 wins c1 (20.2), TP2xPP3 wins aggregate + TTFT. Long-context prefill 1,704 @8k to 2,873 tok/s @256k. | report → |
| 2026-09-10 20:47 PT | ▲ positive | Server runs with --tool-call-parser deepseek_v41 --reasoning-parser deepseek_v41 --enable-auto-tool-choice; clean content/tool-call split verified; OpenCode tmux proof PASSED. | report → |
| 2026-09-10 20:24 PT | ▲ positive | First local benchmark on our box, TP4xPP2, random 1024-in/256-out, server-counted: c1 21.71 out tok/s (TPOT 43.0 ms, DSpark acceptance 6.80% on random tokens), c8 74.60 (peak 80.0). TTFT includes per-shape JIT warmup. | report → |
| 2026-09-10 17:33 PT | ▲ positive | Built localhost/vllm-backport-v41:sm80 (base v0.12.0-sm80 + PR#56201 + fixups + Ampere shims + PP relay; 92 files) and launched dsv41-schaka on :8090 (TP4xPP2, TRITON_MLA_SPARSE_DSV41, fp8_ds_mla, DSpark k=5, 1M ctx). artlair in-place MXFP4 Marlin repack cuts repack peak 2100 -> 9.4 MiB. | report → |
| 2026-09-10 17:15 PT | ▲ positive | Schaka 170hx-journey (TP4xPP2) is the first working V4.1 on SM80: 43.8/72.9/104.2 tok/s @ c1/c4/c8, prefill 1,584/1,650/1,389 tok/s @ 39k/119k/319k, KV pool 14,058,003 tokens. His kit matches our bug: wo_a wrong-shaped under Marlin, and V4.0's 9-arg qnorm op dropping V4.1's apply_q_norm… | report → |
| 2026-09-10 16:47 PT | ▲ positive | Per-branch probe: attention 'o' absmax ~2.0 (sparse MLA works) but _o_proj output absmax ~0.013 - a ~150x shrink; residual goes FFN-only and explodes (0.6 -> 3.5e29 at KV-source layers). One bug: fused inverse-RoPE + wo_a einsum + wo_b. | report → |
| 2026-09-10 11:40 PT | ▲ positive | deepseek-v4.1-flash healthy on 8x CMP 170HX (TP8, eager, fp8_ds_mla, Engram in pinned host RAM; init engine 143.5s, ~5 tok/s) - first known V4.1 boot on SM80/CMP170HX anywhere. | report → |
| 2026-09-10 08:01 PT | ▲ positive | Engine built for SM80 (vLLM dsv41-feat@e47aa780: Ampere sparse-MLA backend + fp8 shims); 475.25 GiB checkpoint SHA-verified; TP8 chosen (layers 20-39 share one KV group); weight-load tail running. | report → |
All reports (38)
| When (PT) | Status | Title | Summary |
|---|---|---|---|
| 2026-09-12 13:02 PT | ▼ | Hourly update #19 — chunk-size loop: both hypotheses rejected | Tested prefill-chunk size as the ITL-tail fix: MAXBATCH=8192 collapses KV (7.86M->3.23M) and fails c1; MAXBATCH=1024 is rejected by the multimodal min. No free ITL fix; baseline restored |
| 2026-09-12 12:33 PT | ▲ | Hourly update #18 — 32K master table online (PP8) | Built the one-command 32K master table; PP8 baseline c1 20.6 / c4 53.4 / c8 78.9 Gen tok/s, 512K@c16 98.9 tok/s; util 0.96 OOMs 512K so lowered to 0.94 |
| 2026-09-12 10:46 PT | ▲ | LMCache validated on PP8 (heterogeneous stages) | The layout-registration hazard AGENTS.md warns about works as designed on heterogeneous PP stages: LMCache group_tokens_per_block resolved per cache group, KV cache group edits applied, non-prefix-cacheable engine group excluded - hourly #17. |
| 2026-09-12 10:06 PT | ● | Sparse-MLA decode knob sweep - dead code found, +2-4% folded in | VLLM_SPARSE_DECODE_MAXNREG is dead code (_decode_maxnreg_kwargs has no caller) - the earlier MAXNREG test measured nothing. Wired real knobs and swept on PP8/512k: HEAD_BLOCK_SIZE=32 gives consistent, bit-identical +2-4% (c1 53.0->54.3, 512k c8 45.7->47.4, c32 60.2->61.6) - folded in as PP8… |
| 2026-09-12 08:18 PT | ▲ | PP8 brought up - 2.75x the KV pool of PP6 | Layout-resolver intersection fix makes PP8 work: 9.35M-token KV pool (2.75x PP6), 8.9x concurrency @1M; 512k aggregate still ~60 tok/s @c32 -> step-cost bound. Leads: vLLM #56120 (SM80 Triton knobs), SGLang #38646 (NVFP4 sparse-MLA, 384 B/tok, SM120-only). |
| 2026-09-12 05:50 PT | ▲ | 512k depth sweep - saturation quantified | Shared-487k-prefix pure decode (server-counted): c1 25.1 / c2 38.1 / c4 44.1 / c8 52.3 / c12 54.6 aggregate - saturates from c8 (per-step cost grows ~linearly with batch at depth). 500 tok/s @512k is ~9x away and blocked by depth step-cost scaling + KV-capped concurrency: kernel/engine work, not… |
| 2026-09-12 04:16 PT | ▼ | LMCache c1 cost A/B - ~11% decode cost | Plain PP6 65.1 tok/s (512-tok gen, thinking off) vs PP6 + LMCacheMPConnector 58.0 - the LMCache path costs ~11% c1 decode; real-OpenCode c1 median drops ~30 -> 26.5. Decision: plain PP6 stays the c1 served default; LMCache is a validated optional arm for cross-restart/evicted-prefix reuse (vLLM… |
| 2026-09-12 03:56 PT | ▲ | LMCache blocker root-caused AND resolved - serving V4.1 | In-process LMCacheConnectorV1 is single-group by construction -> HMA disabled -> vLLM cannot unify V4.1 KV specs (MLA state_content_bytes=584 + SWA compressor group) and startup fails. Correct path: LMCacheMPConnector (group-aware, SupportsHMA) + lmcache MP server WITH GPU access +… |
| 2026-09-12 01:29 PT | ▲ | c1 bottleneck diagnosed (measurement) | nvidia-smi dmon during sustained PP6 c1 decode: SM ~100% but memory-controller only 4-5% -> c1 is SM-occupancy/latency-bound, not DRAM- or power-bound. |
| 2026-09-12 00:47 PT | ▲ | PP6 sweep completed to the full 1M context | Prefix-cached decode aggregate: 4k 49/91/109/146/142 (c1/c2/c4/c8/c16), 32k 45/81/104/159/160, 128k 43/66/100/165/150, 512k 30/46/54/85/88, 1M 16/22 (only c1/c2 fit the 2.89M pool). DSpark acceptance rises to 51.7% @1M. Prefill: 1M c1 TTFT 342.5 s (~3.1k tok/s). |
| 2026-09-12 00:33 PT | ▲ | PP6 relay fix adopted + validated (Schaka c1b0907b) | Rebuilt the PP6 overlay from Schaka's latest kit: the kv-group relay now owns the source indexer K cache (layers.20.attn.indexer.k_cache), fixing his prompt-corruption path (44k check 5/10 -> 0/10 wrong). Ours: coherence OK, needle 128k 3/3, multi-needle 44k 5/5, coding 5/5; pure c1… |
| 2026-09-11 23:22 PT | ▲ | PP6 adopted as the PP default (validated) | PP6 (TP1xPP6, partition 7,7,7,7,7,5, util 0.95) now served: c1 96.9 best-of-4 (107.6 with 93.8% DSpark acceptance) - within noise of TP8xPP1 (100.8), ~2.3x faster than PP8; needle 6/6 + 512k 3/3, coding 5/5; prefill 128k c1 7,716 / c16 23,780; decode 128k c8 164, 512k c8 85. |
| 2026-09-11 23:16 PT | ▲ | PP6 vs PP8 measured - PP6 wins c1 and KV efficiency | Both accuracy-gated. PP6: c1 96.9 (8-tok) / 103.8 @4k / 49.4 @32k vs PP8 43.2 / 32.2 / 27.6; KV 10.95 GiB per 1M (PP8 23.06); 512k c16 88.3 (86.8); decode agg 128k c8 164 (128). PP8 only wins total pool (6.17M vs 2.89M tokens) and some prefill. PP8 is the slowest PP option at c1. |
| 2026-09-11 22:04 PT | ▲ | PP8 current-state sweep + conflict flagged | PP8 (TP1xPP8) on our box: single-stream 43.2/40.9/32.2/27.6 @8/512/4k/32k; aggregate 4k c8 154.9, 32k c16 166.6, 128k c16 139.0, 512k c16 86.8; prefill 4k c1 1350 / c16 7067, 128k c16 30058. Conflict resolved by measurement: PP8 is ~2.3x SLOWER at c1 than TP8xPP1 (43 vs 100.8) - c1 -> TP8xPP1… |
| 2026-09-11 21:22 PT | ▲ | KV accounting measured (PP8) | Zanooda PP8, fp8_ds_mla: 6,170,576-token KV pool in 135.72 GiB total (per-rank 6.48-20.95 GiB) -> 23,617 B/token whole-model; 1M tokens ~23.06 GiB (~26x the model card's 890 B/tok) because fp8_ds_mla keeps uncompressed indexer-K caches. Goal reprioritized to PP8/PP6 only. |
| 2026-09-11 19:21 PT | ▲ | README: PP8/pipeline-parallel numbers documented | Published measured layout comparison: PP layouts prefill ~2x (no all-reduce; Gen2 x4 caps TP4 prefill ~1,700 tok/s) but lose deep decode vs TP4xPP2 (92 vs 130 @c64/512k). Third-party PP8 claims (117 single / 532 agg) use tokens/(last-first) + staggered arrivals and read higher than wall-clock. |
| 2026-09-11 16:07 PT | ▲ | Public fork published | Forked wtdcode/vllm-backport -> roboalchemist/vllm-backport and pushed our work as branch cmp170hx-dsv41-flash. |
| 2026-09-11 15:14 PT | ▲ | Hourly #9 - c1 leads the published ecosystem | TP8xPP1 c1 ~100-103 tok/s (measured, best-of-N) vs shi3z 8x A100 sm80 52 tok/s and zebgop-ops 4xCMP170HX PP4 16.7 tok/s (both author-reported). Clock-lock is a no-op (CMP clamps); moe_backend=triton fails on MXFP4/SM80; DSpark k=5 pinned by checkpoint. Remaining c1 headroom is kernel-level. |
| 2026-09-11 13:34 PT | ▲ | c1 optimization - TP8xPP1 wins at every depth | Single-stream TP8xPP1 (zero pipeline hops): 98.1/95.0/92.6/88.4 tok/s @8/512/4k/32k vs TP4xPP2 89.4/84.2/87.6/54.1 - decisive at 32k. Accuracy gates pass (needle 4/4, coding 5/5). Tradeoff: 512k aggregate 113 vs 130. TP8xPP1 becomes the served config for c1-first. |
| 2026-09-11 11:35 PT | ▼ | Hourly #7 - per-stream TPOT is NOT flat with depth | Correction: the earlier 'flat TPOT' note read the aggregate field. True per-stream: 4k 106 ms | 128k 150 ms | 512k c32 294 ms | 512k c64 504 ms (~5x). 500 tok/s @512k needs ~5x lower per-token cost at depth or ~4x concurrency headroom. Upstream: #46994 MERGED (PP MTP + stale topk_indices_buffer… |
| 2026-09-11 11:22 PT | ▼ | Corrected decode benchmark (ignore_eos) - important | Earlier per-concurrency decode numbers were EOS-truncation artifacts (no ignore_eos -> early stop, per-request overhead dominated). Corrected, ignore_eos + cached shared prefix: 4k c32 302.8, 128k c32 214.8, 512k c64 129.6; single-stream ~105 tok/s pure decode. The 500 @512k target is a… |
| 2026-09-11 08:09 PT | ▲ | PP relay correctness fix adopted - prefill triples | Adopted Schaka's relay fix (topk_indices on the hop + index_k_cache guard): TP1xPP6 prefill 5.6k @32k / 7.2k @128k. TP4xPP2 remains decode-optimal (512k c64 129.6, correct 12/12). TP2xPP4 / TP1xPP8 still boot but needle 0/3 (relay cannot handle TP splits) - rejected. |
| 2026-09-11 06:32 PT | ▲ | Perf finding - the prefill bottleneck is the TP all-reduce | Got Schaka's TP1xPP8 profile booting (2 patches: intersect per-rank KV layouts; guard relay warmup index-K write): PP8 prefill 5,738 tok/s @32k / 8,264 @128k = ~3.7x TP4xPP2 (link-bound all-reduce over Gen2 x4) - but the relay was numerically wrong, reverted. |
| 2026-09-11 04:19 PT | ▲ | Gateway path validated (OpenCode -> Bifrost) | OpenCode driven through Bifrost (bifrost-dsv41/dsv41-flash-cmp at bifrost.roboalch.com/v1) returned exactly BIFROST-OPENCODE-OK - direct and gateway access paths both live. |
| 2026-09-11 02:22 PT | ▲ | Accumulated multi-turn prefix validated | 8-turn accumulating conversation (~13k tokens): fact planted turn 1 recalled exactly on turn 9 (MULTITURN-885440-K); recall turn 1.4 s via prefix cache. |
| 2026-09-11 02:21 PT | ▲ | Full 1M-token context validated | Needle planted at 50% depth in a 1,048,576-token prompt recalled exactly: NEEDLE-885440-X (wall 1113 s, prefill-dominated). The service genuinely uses its advertised context. |
| 2026-09-11 01:24 PT | ▲ | Served, validated, benchmarked | Service HEALTHY (TP4xPP2, DSpark k=5, 1M ctx, CUDA graphs on, EP on): long-context recall 12/12 (4k-512k, 15/50/85% depth), coding 4/5 executed, Bifrost route dsv41-flash-cmp live. |
| 2026-09-11 00:37 PT | ▲ | Live OpenCode TUI validated | Interactive TUI session (not headless) verified exact-token replies - realness gate evidence for c1 claims. |
| 2026-09-10 23:13 PT | ▲ | Benchmark #5 - decode at depth (512k) | Shared-prefix + prefix-caching + ignore_eos, TP4xPP2: per-token decode FLAT with depth (TPOT 8-10 ms @512k, sparse CSA2), but aggregate saturates: 128k c32 214.8, 512k c64 129.6 (26% of the 500 goal). DSpark acceptance ~42% at depth. Cold 512k prefill 326 s (1,608 tok/s). |
| 2026-09-10 22:25 PT | ▲ | Benchmark #4 - live OpenCode sessions | 5 real headless OpenCode sessions (qa + coding + read-tool): wall 18.4-21.1 s per prompt, output 48-120 tok, end-to-end 2.5-6.5 tok/s (median 5.07). Decode ~3.5x slower at 23k depth than shallow. |
| 2026-09-10 22:22 PT | ▲ | Benchmark #3 - arrangements + DSpark real-text correction | Random-token acceptance (4-8%) was misleading. Real prose (TP4xPP2, k=5): c1 43.1 tok/s (accept 41.2%), c4 111.0, c8 153.4 - c1 matches Schaka's 43.8. Arrangements: TP8xPP1 wins c1 (20.2), TP2xPP3 wins aggregate + TTFT. Long-context prefill 1,704 @8k to 2,873 tok/s @256k. |
| 2026-09-10 20:47 PT | ▲ | OpenCode validated (tmux) | Server runs with --tool-call-parser deepseek_v41 --reasoning-parser deepseek_v41 --enable-auto-tool-choice; clean content/tool-call split verified; OpenCode tmux proof PASSED. |
| 2026-09-10 20:24 PT | ▲ | Benchmark #1 - baseline (random 1024/256) | First local benchmark on our box, TP4xPP2, random 1024-in/256-out, server-counted: c1 21.71 out tok/s (TPOT 43.0 ms, DSpark acceptance 6.80% on random tokens), c8 74.60 (peak 80.0). TTFT includes per-shape JIT warmup. |
| 2026-09-10 17:33 PT | ▲ | Update #2 - Schaka recipe built locally | Built localhost/vllm-backport-v41:sm80 (base v0.12.0-sm80 + PR#56201 + fixups + Ampere shims + PP relay; 92 files) and launched dsv41-schaka on :8090 (TP4xPP2, TRITON_MLA_SPARSE_DSV41, fp8_ds_mla, DSpark k=5, 1M ctx). artlair in-place MXFP4 Marlin repack cuts repack peak 2100 -> 9.4 MiB. |
| 2026-09-10 17:15 PT | ▲ | Schaka publishes first working recipe - pivoted | Schaka 170hx-journey (TP4xPP2) is the first working V4.1 on SM80: 43.8/72.9/104.2 tok/s @ c1/c4/c8, prefill 1,584/1,650/1,389 tok/s @ 39k/119k/319k, KV pool 14,058,003 tokens. His kit matches our bug: wo_a wrong-shaped under Marlin, and V4.0's 9-arg qnorm op dropping V4.1's apply_q_norm… |
| 2026-09-10 16:47 PT | ▲ | Correctness root cause localized | Per-branch probe: attention 'o' absmax ~2.0 (sparse MLA works) but _o_proj output absmax ~0.013 - a ~150x shrink; residual goes FFN-only and explodes (0.6 -> 3.5e29 at KV-source layers). One bug: fused inverse-RoPE + wo_a einsum + wo_b. |
| 2026-09-10 11:40 PT | ▲ | MILESTONE — first known SM80/CMP170HX boot | deepseek-v4.1-flash healthy on 8x CMP 170HX (TP8, eager, fp8_ds_mla, Engram in pinned host RAM; init engine 143.5s, ~5 tok/s) - first known V4.1 boot on SM80/CMP170HX anywhere. |
| 2026-09-10 08:01 PT | ▲ | Day-0 SM80 port complete, first boot in progress | Engine built for SM80 (vLLM dsv41-feat@e47aa780: Ampere sparse-MLA backend + fp8 shims); 475.25 GiB checkpoint SHA-verified; TP8 chosen (layers 20-39 share one KV group); weight-load tail running. |