English
Book an intro call

Not every automation decision needs a large language model

Most steps in business automation are small decisions. Is this email spam? Which department should get this request? How urgent is it? Is the customer asking to cancel?

For the past two years, the usual answer to all of these was to send the text to a large language model and ask it to reply. That works, but you pay for writing ability nobody needs. The model produces text, your code has to extract an answer from it, and every so often the answer comes back in a format nobody expected.

The idea is older than it looks

Back in 2023, Stanford researchers showed in the FrugalGPT paper that a chain of models, running from the cheapest to the most expensive, could match GPT-4 at up to 98% lower cost. In 2024, researchers at LMSYS published RouteLLM, a router that decides for each query whether to send it to a strong model or a weak one. On one benchmark it cut costs by more than 85% while keeping about 95% of the performance. Anthropic’s own documentation also recommends starting with its small model, Haiku, for simple high-volume tasks.

What is new in 2026 is a different kind of model, built from the start for decisions only.

A model that returns a decision instead of text

In September 2026, TypeSafe introduced Jev, a model that returns a defined decision: yes or no, a choice from a list, or a score, together with a calibrated confidence level. It does not write text at all. The published price is $0.042 per million input tokens, and output is free. For comparison, the company cites a range of $0.20 to $10 per million tokens for typical language models. Response time, according to the company, is between 70 and 500 milliseconds. For now the model is available only through early access.

There is already an open-source competitor: Laya by ConvAI Innovations, a small open-weights model that answers in about 33 milliseconds. According to its authors, it reaches good results only after being tuned on the customer’s own data. Without tuning, it performs close to random guessing.

What happened when people tried it

The software company Pelcro published a detailed test. In October it described how it had Jev rate the relationship between every pair of pages on its site, 361 pages in all, to choose links to related articles. That came to 129,960 decisions, in under six minutes, for a total cost of 87 cents. After the team added rules, the precision of the links rose from 64% to 81%. A small language model would have cost about $2 for the same job. Embeddings, the long-established way to measure similarity between texts, would have cost about one cent. So here too, the choice depends on the question you need answered, and price is only part of it.

Vals AI, an independent evaluation firm, tested Jev on 400 claims drawn from company financial filings. It got 390 right, exactly the same as GPT-6 Astra, at roughly 1/498 of the cost. But all 12 models tested scored between 95% and 99.5%, so the test was not especially hard. On a set of legal questions, Jev came last of the 12, with the least reliable confidence scores.

Investor Tomasz Tunguz tested Jev on 98 of his own email threads. The language model he had been running in production was right 47% of the time, and Jev 80%. It is a small sample from one person. Still, it shows how quickly you can run a comparison like this on real data.

Where it does not fit

  • Questions that call for complex judgment, such as legal interpretation. This is where the benchmarks show a clear weakness.
  • Checking claims against documents when a single mistake is expensive. In an independent test on GitHub, Jev confirmed false claims at a rate of 23%, compared with 13% to 14% for a large model.
  • Any step that requires writing: a reply to a customer, a summary, a document.
  • A price that may change. Tunguz himself believes the current price is subsidized, and the model is still in early access.

How to build it into a process

In an automation project, this decision belongs in the specification phase. For each step in the process you define the input, the decision required, and what a mistake would cost. Only then do you choose a model. The pattern that works in most cases:

  • A cheap, fast decision step at the entry point, which classifies and routes.
  • A large model only at the step that genuinely needs to write or reason.
  • A confidence threshold. Anything below it goes to a person or to a stronger model.
  • A test set of a few hundred real examples with the correct answer, run against every model before you choose and every time you switch.

The same principle works for AI agents. In April 2026, Anthropic published an approach in which a strong model advises and a cheaper model carries out the work. In its tests, Sonnet with Opus as an advisor scored 2.7 points higher than Sonnet alone and cost 11.9% less per task.

Frequently asked questions

What is a decision model?
A model that takes text and returns a predefined answer, such as yes or no, a category or a score, together with how confident it is. It does not write text, so it is faster and cheaper, and your code does not have to parse the answer.
Should we replace all our calls to large models now?
No. Start with one high-volume step that has a clear answer, such as classifying incoming requests, and test it against real examples. If the accuracy is good enough, extend it to the next step.
What happens when the model is not sure of its answer?
That is why calibrated confidence matters. You set a threshold, and anything below it goes to a person or to a stronger model. That way the expensive mistakes do not slip through unnoticed.

Sources

Want to talk about this in the context of your system? Book a free one-hour intro call.