Why My Local 80B Model Beats Claude at Coding Tasks, and It's Not About the Model

The Setup

I run a dual RTX 3090 Ti homelab (HL1) serving Qwen3-Coder-Next 80B MoE (3B active params) via llama-server. The model is quantized to Q3_K_XL (33.79 GiB, 3.64 BPW) by Unsloth, running at 262K context with Q8 KV cache. Both GPUs stay under 57C at ~300W each. The stack is dead simple: llama.cpp build b8492, systemd service, K8s ingress with TLS, and ccq alias to point Claude Code at the local endpoint.

Everything worked. Inference was fast: 84 t/s generation, 2200 t/s prompt eval at 262K context. But something was very wrong with real-world agentic coding performance.

The Symptom

Every turn of a Claude Code (ccq) session took ~15-20 seconds before the model even started generating. On a 31K token conversation, that's 20 seconds of pure waiting per tool call. An agentic task with 20 tool rounds = 7+ minutes of just prefill overhead.

The llama-server logs told the story:

n_past = 35, slot.prompt.tokens.size() = 31327
forcing full prompt re-processing due to lack of cache data
(likely due to SWA or hybrid/recurrent memory)

Every single turn: 35 tokens of prefix match. Full re-processing of 31K+ tokens from scratch. The checkpoints were being created, but immediately erased on the next request.

The Architecture Problem (Or So I Thought)

Qwen3-Coder-Next uses a hybrid architecture: 3 layers of Gated Delta Net (SSM/recurrent) alternating with 1 layer of Global Attention (3:1 ratio). Unlike pure Transformer models where KV cache handles prefix reuse natively, hybrid models need the recurrent state to be reconstructed from scratch when the token prefix diverges.

llama.cpp PR #20087 added --checkpoint-every-n-tokens N to snapshot the recurrent state during prefill, allowing restoration from the nearest checkpoint instead of reprocessing from position 0. I rebuilt to b8492, added --checkpoint-every-n-tokens 4096, and validated with curl: checkpoint restored, 63% tokens saved.

But in real agentic use? Still full re-processing every turn.

The Wrong Hypothesis: Parallel Slots

My first theory: with --parallel 4 (default), the server assigns requests round-robin across 4 LRU slots. Each slot has its own checkpoints. A single-session agentic client bounces between slots, so the checkpoints from the previous turn are on a different slot.

Fix: --parallel 1. Force everything onto one slot.

Result: no change. Still n_past = 34-35. Still full re-processing. The checkpoints are there, but the server can't use them because the token prefix diverges at position 35. (Spoiler: --parallel 4 turns out to matter later. Once the real problem is fixed, it cuts wall time nearly in half. But at this point in the investigation, it was a red herring.)

The Real Culprit: Claude Code's Prompt Serialization

The breakthrough came from testing with OpenCode (which uses the OpenAI /v1/chat/completions format) instead of Claude Code (which uses the Anthropic /v1/messages format via ANTHROPIC_BASE_URL).

Claude Code logs:

n_past = 35, slot.prompt.tokens.size() = 39933
forcing full prompt re-processing
prompt eval time = 19511ms / 39042 tokens

OpenCode logs:

selected slot by LCP similarity, sim_best = 0.995
prompt eval time = 52ms / 13 tokens

52 milliseconds vs 19 seconds. Same model, same server, same --parallel 1.

Claude Code re-serializes the entire prompt on every turn in a way that makes the token sequence diverge after ~35 tokens (roughly the end of the system prompt). The server sees a completely different token stream from position 36 onwards, so the KV cache, the recurrent state, and all checkpoints are useless.

OpenCode maintains a stable prefix. The server finds 99.5%+ similarity, processes only the new tokens (~13-100 per turn), and the recurrent state checkpoints accumulate naturally (up to 30/32 over a session).

The Benchmark

Same prompt, same task (Python interval scheduling library, 34 unit tests), all running in empty directories with --allowedTools to skip permission prompts:

10x faster than Opus. Clean first pass, zero fix rounds. The model had previously failed this exact benchmark (circular imports, multiple fix iterations) when running through Claude Code. The config improvements since then helped, but the real unlock was the client switch. And once the client was fixed, restoring --parallel 4 gave an additional ~2x speedup. The parallelism that was useless with CC's broken prefix became a real advantage with OpenCode's stable one.

What I Learned

1. The client matters as much as the model. Claude Code's prompt re-serialization is invisible to the user but catastrophic for incremental KV cache on hybrid SSM architectures. This is a CC-specific behavior. It doesn't affect pure Transformer models the same way because their KV cache is more forgiving of prefix mismatches.
2. Hybrid SSM models amplify cache misses. With a pure Transformer, a prefix mismatch just means recomputing KV for the divergent suffix. With a hybrid model (Gated Delta Net + Attention), the recurrent state must be reconstructed from scratch, or from the nearest checkpoint if the prefix matches that far.
3. --checkpoint-every-n-tokens works, but only if the client cooperates. The feature is sound. The curl tests proved it. But it's useless if the client rebuilds the prompt in a way that breaks prefix continuity.
4. --parallel N was a red herring, until it wasn't. With CC's broken prefix, slot count doesn't matter: you can't reuse a recurrent state built from a different token sequence. But once the client cooperates (OpenCode), --parallel 4 cuts wall time from 2:21 to 1:02. The server can now overlap prefill and generation across slots because the cache actually works. The same config change that did nothing with CC gave a ~2x boost with OpenCode.

The Workflow Now

alias cc='claude'                              # Opus/Sonnet via Anthropic API
alias ccq='OPENAI_API_KEY=... OPENAI_BASE_URL=https://code.llm.internal/v1 opencode run'

Design with Opus/Sonnet in Claude Code. Write the code with ccq (OpenCode + Qwen local). Review with Opus/Sonnet. The local model handles the heavy lifting at 10x the speed, zero API cost, and full 262K context. Claude Code stays for Anthropic models only.

Relevant llama.cpp Issues

• #18497: cache-reuse not effective for qwen3-next
• #20225: full prompt re-processing on every conversation turn
• #19794: --swa-full ineffective for Qwen3-Coder-Next
• PR #20087: --checkpoint-every-n-tokens (the fix that works, if the client cooperates)