Kevin does not run your models. llama.cpp, Ollama, LM Studio and vLLM already do that, and they do it better than anything we would write. Kevin is the layer above them: one endpoint that dispatches a request to whichever backend should serve it, including a model on your own GPU at zero marginal cost, and then writes a row recording what that request cost and what caused it.
A single developer who wants a local model and a good interface is not our user. LM Studio's desktop experience is better than anything we would build, it is free for local use, and it needs no message bus, no service fleet and no ledger. Kevin is the wrong shape for that person. Installing it would cost you an afternoon and return nothing.
The threshold is not model size or VRAM. It is the moment there is more than one machine, more than one model, or anyone asking what a workload actually cost. The community's own answer to that is to assemble llama-swap plus LiteLLM plus a dashboard by hand. If you have already done that, you know the shape of the problem and you can judge whether this is a better version of it.
Every node below is a real module. The paths under them are real.
Claude Code, an agent, a script. Anything that speaks the Anthropic Messages API.
One API-compatible endpoint. You change a base URL, not your code.
fabric_proxy/server.py
Resolves the requested model to a provider adapter, applies tier policy when one is configured, and refuses the call if the estimated cost breaches the hard cap.
fabric_proxy/router.py
A local Ollama-compatible endpoint on your own GPU, or Anthropic, OpenAI, Groq, Gemini. Five provider adapters ship today.
fabric_proxy/providers/
Computes routed cost, the counterfactual baseline cost, compression ratio and latency for this exact request.
fabric_proxy/kevin_value.py
Append-only SQLite on your disk, stamped with the build version that wrote the row.
KEVIN_HOME/db/walt.db
Point a tool at it with ANTHROPIC_BASE_URL=http://localhost:$KEVIN_PORT_WALT. The service is called walt in the codebase. Anthropic-API compatibility is the reason this exists in that shape: this segment is almost entirely OpenAI-API-shaped, and a Claude Code session that wants a local fallback has no clean option today.
Inference here is done by llama.cpp and the GGUF format (MIT, ggml-org, now maintained with Hugging Face funding), by Ollama on top of it, and by vLLM (Apache-2.0, PyTorch Foundation) where a serving engine is wanted instead. Kevin contributes none of that work and depends on all of it. Every claim on this page is about the layer above: dispatch, attribution and accounting. If a page from us ever reads as though we built the runtime, that is a bug in the page.
Local inference is not free, it is unmetered. Power, amortised hardware and the opportunity cost of a box you bought for this are all real and none of them appear anywhere. Nothing in the self-hosted stack tells you what a request cost, what it would have cost on a hosted frontier model, or which workload was responsible. Kevin writes one row per request.
fabric_proxy/telemetry.py, table fabric_calls
| Column | Why it is there |
|---|---|
| model_routed provider | Which backend actually served it. |
| input_tokens output_tokens | Billable units as reported by the backend, not estimated. |
| cost_routed_usd | What this request cost. Exactly 0.0 for a local model, deliberately. |
| latency_ms | What the decision cost in time. |
| session_id tool_origin task_type | Who asked, and from what. tool_origin is derived from the user agent. |
| ticket holon | The work item that caused the spend, sent as request headers. This is what makes cost attributable after the fact. |
| pub_ver | Provenance: the build that wrote the row, so you can tell which code produced which numbers. |
The ledger is a local SQLite file. It is not shipped anywhere, there is no telemetry endpoint on our side, and the schema is in the file named above so you can query it yourself.
Field names are real. Values below are zeros, not measurements.
// shape produced by fabric_proxy/kevin_value.py "kevin_value": { "model_routed": "…", "model_baseline": "…", "provider": "…", "input_tokens": 0, "output_tokens": 0, "cost_routed_usd": 0.000000, "cost_baseline_usd": 0.000000, "routing_savings_usd": 0.000000, "compression_ratio": 0.00, "routing_reason": "…", "latency_ms": 0 }
We publish no savings percentage. Any number you see from us should come from your own traffic, out of your own ledger.
The counterfactual baseline is returned inline, as shown, and written to the ledger alongside the request, so a savings rollup is a query rather than an estimate. It starts from the day you install it. A row logged before those columns existed carries an unknown baseline, stored as unknown rather than zero, and is left out of the money columns instead of being counted as a request that saved nothing. The rollup reports that coverage next to the figure.
This is the case Kevin is actually shaped for, and it is worth being precise about which parts exist.
Each box writes its own append-only SQLite file at KEVIN_HOME/db/walt.db. "Where it ran" is which machine's ledger holds the row, plus the provider column inside it.
Machines are declared in daemon/config/machines.yaml and polled by a background loop that publishes transitions and serves a snapshot at GET /api/peers/health. The peer list is read once at startup; edits need a restart.
Local models are priced at exactly 0.0 in the price table. The local adapter also refuses the call rather than mislabel it if the upstream proxy quietly served the request from a paid backend instead. A free row that was actually billed is worse than no row.
Two consequences of pricing local inference at zero. It is arithmetically correct at the margin and it is economically incomplete: it excludes power, depreciation and the capital you already sank. We report the marginal number because it is the one we can measure without guessing at your electricity tariff. Treat the hardware line as yours to add.
We are not going to tell you that owning is always cheaper, because in 2026 it stopped being true by default. If you built a payback model in the spring, it is materially worse now.
Secondary and journalistic sources observed 2026-08-01, not vendor-confirmed. Re-check before you spend against them.
Owned iron wins at steady-state utilisation and loses badly on bursty or low-volume traffic, where scale-to-zero on rented capacity is not a close call. A box that idles most of the day was a defensible purchase when memory was cheap and is a harder one now.
So the honest position is narrower than "self-host and save money". It is: make the hardware you have already bought carry more of the load, measure what it carries, and route the overflow to rented capacity instead of pretending you never need any. Buying more hardware to run Kevin is not a case we will argue.
This segment gets Kevin free. There is no seat price, no cloud tier, and nothing to upsell you to, so there is also no reason for us to talk you into it. These are the cases where something else is the better answer.
One more, since you will find it yourself. Cheapest-capable-model routing was commoditised during 2026: a hyperscaler ships one free inside your account, a model-host shipped an Anthropic-compatible routing proxy in July, and there is a well-funded Apache-2.0 semantic router in the vLLM ecosystem. We are not claiming to have invented cost routing and we would not build a product whose headline was that. What none of them emit is a per-request record of what the decision cost against what it would have cost otherwise, on infrastructure with no vendor control plane in the path. That is the whole argument.