System one models take a state and a set of typed questions, and return decisions with probabilities instead of text. Nothing is generated, so nothing has to be parsed and no evidence can be invented. Six rules make them work in a real build: ask atomic questions, pack them into one pass, design the state small, gate on calibration, keep a deterministic layer owning the evidence, and measure the questions nobody looks at.
I spent a weekend trying to make a generative model do something it is bad at.
The plan was small. I was building an AI audit tool: read one web page, report what is wrong with it. So I sent the page to a model and asked for a structured report. What came back was prose wearing a JSON costume. Every field I needed carried a small chance of being invented, and I had no way to tell which ones.
The fix was not a better prompt. A different class of model was doing the job, one that never writes a sentence at all, and once I understood how it works most of my architecture fell out of that understanding.
What system one models actually are
The name comes from Kahneman’s Thinking, Fast and Slow. System 1 is the fast, intuitive judgment; System 2 is the slow deliberate one. These models are built for decisions a knowledgeable person could make in a few seconds.
Mechanically you send two things: a state, which is any messy context you have, and a set of typed questions. What comes back is typed answers with probabilities attached.
Asking a language model for a decision means coercing a text generator into emitting something your code can read, then parsing it back out. Each step of that coercion is a place to fail. A system one model returns the decision directly, because the shape of every answer is fixed by the question you asked.
Three primitives, one request
| Primitive | What it asks | What you get back |
|---|---|---|
| Choice | Pick one option from a list | The winning key, a probability per option, a confidence |
| Score | Place the input on an ordered scale | A fractional score plus the full distribution |
| Noul | Is this statement true? | One calibrated probability between 0 and 1 |
Three shapes cover an enormous amount of what software needs to decide: which queue, how urgent, allow or block, keep or drop, how risky. If you have written a branching rule and wished it were slightly smarter, that is the gap these fill. All three can be mixed into a single request.
Why one pass can answer six questions
The packing is the part I did not expect. The state goes into the sequence once. Each question then follows it as its own branch, under a mask that lets a question see the state and never see its siblings.
Two useful properties fall out of that. Every question is evaluated in isolation, so the answer to question three cannot be contaminated by question two, and adding questions does not degrade the earlier ones. And because the state is encoded once with no decoding step, response time barely moves as you add questions. You pay for the extra question tokens and almost nothing else.
The open implementation called kev reports packed and separate requests agreeing to within about four parts in a million, which is the kind of number that makes the isolation claim believable rather than merely asserted.
Two open implementations appeared within days
Jev is the original and it is closed. Within a week of its launch, two independent open versions of the same idea were on GitHub under permissive licences, which is fast even for this field.
One is kev, a small family of decision models you can train and run yourself. The other is Laya, a 421 million parameter decision engine that crossed twenty thousand stars in under a week, with weights on Hugging Face and a single forward pass that carries a whole page.
Laya comes from an independent researcher, Nandakishor M, who describes his background as a former principal investigator at IIT Palakkad. That is a Kerala institution and I am from Kerala, so I read it first for that reason. I stayed for a better one: its README spends an unusual amount of space explaining where the model fails. That is rare enough to be worth saying plainly, and it is what made the model useful to me.
The six rules, in the order I learned them
They come from one build. SiteClarity is an open source tool that reads a page and reports what is working against it, with a quote backing every finding. It runs inside a free cloud tier against a 10 millisecond CPU budget, and the four interchangeable backends behind one interface are what let me swap between the closed original and both open implementations without touching a call site.
1. Ask atomic questions, then do the reasoning in code
This is the rule that changed how I build. TypeSafe’s own documentation states it plainly:
“System One models work best when each question asks one specific, well-scoped thing. If the question you want to ask would require extended reasoning or weighs multiple independent factors, decompose it. Ask each factor as a separate question, then combine the results with logic in your code.”
That sentence is from TypeSafe’s documentation on how System One models work, and it reorganised my design. Instead of asking one model to judge a whole page and hoping the judgment hangs together, I ask six narrow questions and do the reasoning myself. The intelligence gets narrower and the system gets more reliable.
The logic also becomes inspectable. When my priorities shift, I change a coefficient in TypeScript rather than rewriting a prompt and hoping nothing else moved.
2. Pack everything into one pass
Six narrow questions could easily have meant six round trips. They are one request instead, because the state is encoded once and the questions ride along as isolated branches.
The practical effect is that question seven is nearly free. On the hosted service, adding a question to a batch costs its own tokens and a sliver of time, where a second generative call would cost the whole context again. My analysis runs a catalogue of questions over the same page state in a single call, and the latency I measure is dominated by fetching the page.
3. Design the state small before you design the prompt
This is the tradeoff the announcement posts skip, and it shaped my architecture more than anything else.
The hosted model gives you roughly 32,000 tokens of state per question branch. Laya’s English checkpoint gives you 512 total, and 1,024 on its multilingual and typed-decision checkpoints. Hand it too much and it rejects the request rather than truncating quietly.
So the question becomes what belongs in the state, and the obvious answer is wrong in an interesting way.
Chunking the page and feeding it in pieces breaks at the boundaries and does not scale past one page. What I do instead is scope-partitioned state: every question gets the smallest state that can answer it. A page-level question gets the title, the headings and the opening paragraph. A question about whether a section answers its own heading gets that section and nothing else.
That second part matters for correctness rather than for size. Asking whether a section stands on its own while padding the state with the rest of the page corrupts the question you are asking.
My section states come out around 293 tokens, which is why a 512 token model can answer questions about them at all. The same code and the same questions run against two models with a sixty-fold difference in capacity, because the state was designed small in the first place.
Quick question: do I need a GPU to try this?
No. The hosted model needs nothing. A half-billion parameter local version trains in under two hours on Apple Silicon and serves a six question request in about 160 milliseconds. Everything here runs inside a free cloud tier with no card on file.
4. Gate on calibration, and know what it cannot catch
System one models are trained so their probabilities reflect real uncertainty. Across a hundred similar cases, a 0.8 should be right about eighty times. That property is called calibration, and it is what separates a number you can act on from a number that merely looks precise.
Calibration is also where honesty is required, and the documentation says so without hedging. It is measured across groups of predictions, so it says nothing about whether the individual answer in front of you is correct. A well calibrated model can still be confidently wrong.
Laya’s own README gave me the clearest example of that. Its English checkpoint does not read non-Latin scripts. Handed Khmer text, it scored 0.000 accuracy while reporting 0.952 confidence. Three characters of certainty attached to a completely wrong answer, and because the model stayed confident while being wrong, no threshold could have caught it.
Confidence gating protects you from uncertain wrong answers. It does nothing about certain ones. The fix is a language and script check before the model is asked anything, not a higher bar.
Quick question: so should I raise my confidence threshold to be safe?
Raising it mostly discards answers the model was correctly unsure about. Fix the input first, then tune. I keep mine at 0.60 because loosening it gained one case out of forty-five and forty-five cases cannot tell you a looser bar stays clean.
5. Keep a deterministic layer that owns the evidence
This is the reason I was drawn to system one models. The model cannot author a sentence, so it cannot invent a quote.
Every quote shown in a report from my tool is verified to be a verbatim substring of a stored passage before it is displayed. The model returns decisions. A separate layer, with no model in it, renders the sentence you read. That removes an entire category of failure structurally rather than by testing for it, and it is the pattern I would copy into any system where a wrong claim carries a cost.
6. Measure the questions nobody looks at
The lesson I keep relearning: the most visible output is the one most likely to be shipping unmeasured. My cataloguing question, the one that selects which suggestion a user reads, had two cases for weeks while much duller questions had nine.
Growing the corpus was uncomfortable and necessary. Nineteen cases scored 95 percent. At forty-five cases it fell to 84 percent, and a false positive appeared in the single question that gates everything downstream. The fixes brought agreement back into the high eighties with false positives counted separately, and the corpus is past seventy cases now. Every time I made it harder, the number fell before it rose.
I track false positives apart from agreement, because reporting a problem that is not there costs more trust than missing one costs value.
How this compares to using a general LLM
| System one model | General LLM | |
|---|---|---|
| Output | Typed decision plus probabilities | Prose you must parse |
| Failure mode | Wrong answer with a confidence attached | Wrong answer, invented field, or malformed JSON |
| Latency | One pass, no decoding | Token by token |
| Cost per call | Fractions of a cent at published rates | The vendor reports ten to a few hundred times more |
| Cost of adding a question | Nearly free, state is encoded once | Another full call |
| Reasoning depth | Shallow by design | Deep, and the reason it is slow |
| Good for | Routing, classification, scoring, gating, guardrails | Writing, planning, open-ended reasoning |
The vendor’s published comparison has system one models at 193 times faster and 444 times cheaper than an equivalent generative call. Those are their benchmarks rather than mine, and the open implementations disagree with each other depending on which checkpoint and which task you measure. Treat every speed number in this space as load-dependent and re-measure it on your own workload.
What I can say from my own build: six questions over one page state in a single request, inside a CPU budget I cannot exceed.
The honest framing is that the two are complements. Put the fast decision at the branch point, and call a generative model only on the path that actually needs prose or planning. It is the same argument I made about how AI search engines retrieve and synthesise, applied one layer down.
What I got wrong
Four things went wrong, and the first cost me the most.
I assumed an API contract and then wrote tests that confirmed my assumption. One adapter posts to an endpoint shape I inferred from a different platform’s conventions. My unit tests stub the network and assert that shape, so they pass while proving nothing about the real service. A test that encodes your guess is worse than no test, because it turns an open question into a false confidence. The fix is to call the real thing once and record what comes back.
I wrote calibration cases that calibrated nothing. The first version restated the question inside the case itself, so the harness was quietly measuring a duplicate. Sharpening the real question changed nothing, because the real question was never under test. Selection cases now supply only the candidate sentences and the harness builds the question from source.
I tuned a threshold against a corpus too small to certify it. Loosening the confidence bar gained one case out of forty-five, and I left the bar alone. Tuning until the number looks better is how you fit to your corpus instead of measuring against it.
I wrote tests for every module and none for the pipeline that wires them together. All of the careful logic is unit tested. The orchestration running through the middle of it is not, which is where the last real bug was hiding.
The limits worth knowing before you start
The model is shallow on purpose, so anything needing multi-step reasoning has to be decomposed into questions plus code. Calibration on your own task is unknown until you label outcomes from that task, and benchmark calibration says nothing about your workflow. A half-billion parameter local model also knows less about the world than a large one, which matters when a question depends on knowing things rather than reading the state in front of it.
None of those are reasons to avoid the approach. They are reasons to keep the questions narrow and the reasoning in code where you can see it.
Frequently Asked Questions
What is a system one model in plain terms?
A model that makes a judgment instead of writing text. You give it context and questions with fixed answer shapes, and it returns an answer with a probability per possibility. It cannot produce a sentence, which is why nothing downstream has to parse or validate prose.
Is a system one model just a small language model?
No, and the difference is the training objective rather than the size. Small language models still predict the next token, so generated text is the output. System one models are trained against labelled outcomes so their probabilities mean something, and text generation is not in the pipeline at all.
How is this different from structured output mode on an LLM?
Structured output constrains the shape of generated text. The model is still generating tokens, so it is still slow, still expensive per call, and can still put a confidently wrong value inside a valid structure. Removing the generation step changes the cost and the failure modes.
Can a system one model be wrong?
Yes, and you should assume it will be. It can be wrong while sounding confident, which is the case that matters, because no threshold catches it. Gate on checks you can run deterministically, then use the confidence to decide what a human reviews.
Do I still need a large language model?
Usually yes, for whatever needs prose, planning or multi-step reasoning. The pattern that works is a decision model at the branch points and a generative model behind the doors that need writing. You are replacing the small high-volume judgments, not the thinking.
What does this cost to run?
Published rates for the hosted model are a fraction of a cent per call, and the vendor reports a few hundred times cheaper than an equivalent generative call. The open implementations cost nothing but your hardware. The app I describe here runs in a free cloud tier with no card on file.
Can the page being analysed instruct the model?
It is a real risk, and it is why the output shape is fixed before the model ever sees the page. A page can try to say something flattering, but the model returns a value from a closed set I defined, and the report only renders quotes fetched from stored passages. Nothing the page says can become a sentence in the report.
Conclusion
The speed is real, but it is not what draws me to system one models. Removing text generation removes a category of problem, and a lot of otherwise fiddly architecture exists only to manage that category. Six rules carried the build: atomic questions, one packed pass, a state designed small, thresholds gated on real calibration, a deterministic layer owning the evidence, and a habit of measuring the questions nobody looks at.
If you want somewhere to start, Laya and kev are both Apache-2.0 and small enough to run on your own machine, and the hosted version is worth trying first if you would rather see the shape of the answers before committing hardware to it.If you would rather see them used than described, the seven AI answer readiness checks in a real build are the worked example.
For the organisational version of the same discipline, giving an AI agent a narrow job with a budget applies it to a workflow rather than a single call, and how to deploy AI agents covers where constrained roles fit in a team. That is also the shape of every agent I have set up at visibility.so: a narrow role, a typed output, and a human who approves rather than supervises.
