Own Your Stack/The Core/hybrid
Own your inference
hybrid
Answer the easy majority on a small model you run — and catch the cheap model when it's confidently wrong with a verifier that's free: exact arithmetic. Escalate only the queries that earn it.
Most of what you ask an LLM is easy. Facts, rewrites, plain Q&A, arithmetic — a small model running on your own machine nails those, free and private. The rare hard query — a proof, real code, multi-step reasoning — is where you actually want frontier quality. Paying the frontier rate for the whole stream, easy and hard alike, is the part nobody questions.
The catch isn't routing the easy queries home — it's knowing when the small model is confidently wrong. A router built on the model's own signals can't tell confident-right from confident-wrong. hybrid's answer is a verifier stronger than the model, and free: Python's exact arithmetic. It solves what it can outright (closed-form math and unit conversions, on-box and exact), and for word problems it has the model plug its own numbers back into the problem and re-derives every check — a false one is a hard escalate. The frontier key only ever leaves your machine on a query that earns it.
# the oracle settles what it can; the rest falls through query → router ─┬─ solve: arithmetic / conversion ─▶ SOLVED (exact, free) ├─ rule: hard category ─▶ ESCALATE ├─ rule: open-ended ─▶ LOCAL ├─ verify: plug the numbers back in │ checks hold ─▶ LOCAL │ false ─▶ ESCALATE └─ vote: self-consistency ─▶ LOCAL / ESCALATE
Solves what it can, exactly
Closed-form arithmetic, plus exact unit conversions, percentage-change, and multiples, are answered by a deterministic solver — on-box, instantly, correct by construction. Conversions are carried in fractions, so 1 in = 25.4 mm never drifts. Zero frontier calls, and it's the class a small model gets confidently wrong (17⁴ → 83521, never 6859).
Verifies the local answer
For any query with a number, the local model answers and plugs its own numbers back into the problem's relationships; we re-derive each one exactly. A false check is a hard escalate — the answer is provably inconsistent with the problem, not merely out-voted. Live, it caught five of six ugly multiplications a small model states confidently — the exact errors self-consistency waves through.
Escalates the known-hard by rule
Categories a small model is known to fail — code, proofs, puzzles — escalate to the frontier on a deterministic rule, with no point spent trying locally first. Open-ended tasks like rewrite and summarize stay local, because there's no single right answer for the verifier to check against.
Falls back to self-consistency
For what the oracle can't settle, the local model answers a few times; unanimous agreement means confident and stays local, while disagreement means uncertain and escalates. It catches genuine uncertainty cheaply — and the deterministic tiers above catch the confident wrongness it can't.
Keeps the limit visible
The line moved, it didn't vanish. The oracle checks the answer against the relationships the model transcribes — so a self-consistently-wrong setup still needs the frontier. A second small model as an independent vote was tested and rejected; it shares the classic blind spots. --demo and a benchmark keep that boundary on screen rather than papering over it.
Drops in front of any OpenAI client
Measured on qwen2.5:7b: 75% of a 20-query benchmark answered on-box, every confident-wrong multiplication caught. server.py exposes an OpenAI-compatible endpoint — Cursor, Cline, or any script gets local-first routing transparently, each reply carrying an x_hybrid field (route, reason, backend, latency). Dependency-free, stdlib-only Python, MIT-licensed.
Part of The Core.
hybrid owns the inference decision. The Core is the layer every model call flows through — a router you own and local-first inference.
Own your inference, not your invoice.
hybrid is open source and MIT-licensed. Read the code, read the build note, run it on your own box.
View hybrid on GitHub →