Phantom · use cases

Your hardware. Your models. Real seats.

Local models are first-class seats in Phantom, not a demo lane. Discovery and dispatch are separate problems, and the numbers below come from the repository's own changelog, receipts and source files rather than a benchmark we cannot show.

Install Phantom Read the local inference guide

How local works

Discovery stays on Ollama. Dispatch is opt-in.

Most local setups collapse the two, then wonder why the model picker goes empty. Keeping them apart is what lets dispatch move onto the fast runtime without losing the seat list.

Discovery answers what you have

Installed tags, context window and tool capability come from Ollama's /api/tags and /api/show. Every tag whose capabilities include tools becomes a local:<tag> seat.

Source: docs/VERSE.md, docs/LOCAL-INFERENCE.md

A model without tools is hidden

A model that cannot call tools cannot drive an agent session, so Phantom hides it rather than letting you pick something that fails on the first turn.

Source: docs/VERSE.md

Dispatch is a separate, opt-in lane

Turn dispatch optionally goes to llama-server through a normalizing proxy that keeps the prompt prefix append-only. The proxy is a separate listener, so an unset or malformed value falls back to the lane that already works.

Source: docs/LOCAL-INFERENCE.md, src/core/local-runtime/llama/anthropic-shim.ts

Local engines are honest about their tier

The local-coder and openai-compat engines are defined in the engine registry with tier mid: they can work on branches but can never satisfy the frontier requirement for merging to main.

Source: docs/LOCAL-INFERENCE.md, src/core/run/engine-registry.ts

Context windows are read, not assumed

A local seat compacts at the window the runner actually serves, minus 33k of headroom. Tags under a 56k window are listed but disabled, with the reason shown.

Source: docs/VERSE-CONTEXT.md

Weights plus cache is the real budget

A 27B model at four-bit lists an 18 GB download on the Ollama tag pages, and a 27B model at eight-bit keeps about 27 GB of weights resident before the KV cache. The docs tell you to leave room for the cache and the rest of the system.

Source: docs/LOCAL-INFERENCE.md, model table

Measured, with receipts

Numbers we can point at.

Recorded runs on a 128 GB machine with a 27B local model. These describe what was measured where it says so, not a promise about your hardware.

Prompt tokens reprocessed per turn: 23,500 → 34 to 556

The Anthropic-compatible shim used to fold per-turn counters ahead of the whole conversation, invalidating the prefix cache every turn. The same fix took four parallel agents from 1386s to 540s.

Source: CHANGELOG.md, 3.3 local lane, src/core/local-runtime/llama/anthropic-shim.ts

Aggregate decode throughput

9.05, 15.61 and 22.79 tokens per second at 1, 2 and 4 llama-server batching slots, and four agents completing 4 of 4 correct in a 521s wall run, all on one GPU with one copy of the weights.

Source: site/README.md, provenance table

A public qualification receipt

One recorded online qualification on 2026-10-01: a multi-file rename fixture passed its checker in 392.5s, changing 3 files across 12 turns, on Qwen3.8 27B at Q8_0 with uncontrolled cache. One observed run, not a baseline or head comparison.

Source: benchmarks/local-qualification-2026-10-01.json

Measure your own setup

ashlr benchmark run explicitly starts local model work. ashlr benchmark --compare-reports BASE CANDIDATE is fully offline and refuses incompatible or uncontrolled receipts; it reports matched observations, not universal savings.

Source: docs/AGENT-HARNESS-EVOLUTION.md

Picking a model

Sizes the guide already recorded.

Four-bit download sizes as listed on the Ollama library tag pages in the setup guide, checked 2026-10-25. Download size is not total memory in use.

From docs/LOCAL-INFERENCE.md; each row names its Ollama library page in the guide.
Ollama tag	Download size	Notes
`qwen3.8:27b`	18 GB	dense 27B
`qwen3.6:27b`	18 to 19 GB	dense 27B; a coding tag is the same size
`qwen3.6:35b`	23 to 24 GB	35B-A3B MoE, same digest
`qwen3-coder:30b`	19 GB	30B-A3B MoE, q4_K_M digest

Sources: docs/LOCAL-INFERENCE.md, Ollama OpenAI compatibility

Try it

Pull a model, then open the console.

ollama pull qwen3.8:27b
npm i -g @ashlr/phantom
phm verse

Setup as documented: docs/LOCAL-INFERENCE.md, docs/QUICKSTART.md

Keep going