BERT, an LLM, or Jev: what should make the decision?

By Sandeep · 11factor

Research and diagrams assisted by Hermes Agent. Sources checked 21 September 2026. The support example and its outputs are illustrative, not a new benchmark.

A customer writes:

I was charged twice. Please refund the duplicate.

Your software needs to route the message. It might also need to identify whether the customer is requesting a refund. It does not need a beautifully written paragraph about billing.

You could fine-tune a BERT-style classifier. You could ask a generative LLM to return a category. Or you could give Jev the message, your questions and the allowed answers.

All three can be used to produce a label such as billing. They get there differently. Those differences affect what you train, what you can change at request time, what your output means and what you must test.

My interest in Jev is here: can a model designed around bounded judgments be a better component than a text generator for this particular job?

That puts it alongside traditional classifiers and LLMs used as classifiers. It does not establish that Jev has BERT's architecture, or that it is the cheapest or most accurate option. We have not run that three-way benchmark.

The diagram below traces the three decision paths described in this article.

First separate the model's job from its architecture

A classifier chooses among categories. An encoder turns an input into useful internal representations. A generative language model assigns probabilities to possible next tokens and can use them to build an output sequence.

These categories overlap. An encoder can feed a classifier; a generative model can be used to classify. A product that returns classifications has not thereby disclosed how its neural network works.

BERT is a documented encoder architecture. A typical GPT-style LLM uses a causal decoder architecture. Jev is a product with a documented decision interface and a high-level vendor account of its architecture and training.

We know more about the first two internal mechanisms than the third. An honest diagram should show that uneven level of knowledge.

1. BERT: read the message, then score the categories

BERT stands for Bidirectional Encoder Representations from Transformers. Jacob Devlin, Ming-Wei Chang, Kenton Lee and Kristina Toutanova describe its architecture and training in the original paper.[1]

Here is an ordinary BERT sequence classifier:

message
  → tokens and their embeddings
  → bidirectional Transformer encoder
  → a representation of the message
  → task-specific classification head
  → category scores and probabilities

A token is a piece of text, not necessarily a whole word. An embedding represents that piece as a vector: a list of learned numbers. Position information helps distinguish different arrangements of the same words.

Inside each encoder layer, self-attention lets a token's representation use information from other positions in the input. In BERT, that context can come from both directions. The representation at charged can take account of a later twice.

A feed-forward network then transforms each position's representation. Residual connections and normalization help carry and stabilize information through the stack. Repeating these layers produces representations that depend on the surrounding text.[1][2]

The original BERT classification setup uses the final representation of a special [CLS] token as an input to a classification head: a learned mapping from that representation to one score for each category.[1]

For our example, we configure that head for:

billing | technical | other

For mutually exclusive categories, a softmax converts the raw scores, called logits, into a normalized distribution over those categories. The application can select the highest-scoring label or send uncertain cases for review.

There is no need to generate the sentence “This request should be routed to billing.” The classifier directly produces the category scores.

What gets learned, and when?

Original BERT pretraining includes predicting masked tokens and a next-sentence task. So “BERT never predicts words” would be wrong. Masked-token prediction during pretraining is different from using a classification head to generate category scores at deployment.[1]

In conventional supervised fine-tuning, you supply examples from your task with labels. Training adjusts the classifier head and, commonly, the encoder weights. The head's output dimensions correspond to the categories you configured.

If you substantially change those categories, the old head does not magically acquire the new mapping. You may need to change it, train again and evaluate again.

This is an attractive baseline when the task is stable, labeled examples are available and deployment control matters. Open-weight encoders give you a route to running the model on your own infrastructure. Whether that is cheaper or faster depends on model size, hardware, traffic and the work needed to maintain it; it is not a property the word “BERT” guarantees.

DeBERTa is a related family worth considering. Pengcheng He and colleagues separate content and position information in attention. The original paper's “enhanced mask decoder” concerns masked-token pretraining; it does not turn a normal DeBERTa classifier into a GPT-style text generator.[3]

The exception that makes the comparison more interesting

It is too strong to say that encoder-based classification always needs fixed application labels.

An encoder trained for natural-language inference, or NLI, can evaluate whether one text supports another. You supply the message alongside hypotheses such as:

• “This request is about billing.”
• “This request is about technical support.”

The model scores the relationship. Its underlying entailment head remains fixed, but the descriptions of the categories arrive as input text. Wenpeng Yin, Jamaal Hay and Dan Roth demonstrated a BERT-based approach to zero-shot text classification using this idea.[4]

That gives us a genuine request-time classifier baseline. It is not necessary to invent a new fixed head every time a category description changes. It is also not free of design problems: a vague other category needs careful treatment, and scores depend on how the hypotheses and candidate set are constructed.[5]

Jev should be compared with this baseline too, not only with an old fixed-label classifier.

2. A generative LLM: produce an answer one token at a time

A typical decoder-only LLM also uses embeddings, attention and feed-forward transformations. The important difference in attention is the causal mask: a position cannot look ahead to later positions in the sequence.[2][6]

These simplified masks explain access to context. They are not measured attention weights, and they do not reveal Jev's internals.

View full-size attention masks diagram

In next-token training, the target is to predict what comes next from the preceding context. The training sequence is known, so many positions can be processed together while the mask prevents each position from seeing its future target. Causal training does not mean training one token in a separate invocation at a time.[2]

At inference, separate two stages:

Prefill: the prompt is already known. The model processes it, builds cached attention information and computes a distribution for the first new token.

Decode: a token is selected, added to the sequence, and used as context for the next step. A key/value cache, usually shortened to KV cache, reuses attention information from earlier positions rather than recomputing all of it for every new token.[7]

For our routing example, the prompt includes the message and a requested output structure. The model might produce:

{"route":"billing"}

That displayed JSON contains multiple text pieces; a whole word or field is not necessarily one token. The ordinary decoding process depends on the growing output prefix.

“Autoregressive” describes that dependency. It does not mean every GPU operation is serial: computation within a step, prompt processing and batches of requests use parallel hardware. Our diagram shows the basic dependency, not an implementation timing trace.

We no longer have to beg for JSON

The joke about “ONLY RETURN JSON, PLEASE” describes prompt-only formatting. It is not a fair description of modern structured generation.

A constrained decoder can mask out tokens that would violate a grammar or supported schema. With an enum containing billing, technical and other, it can constrain the answer to that set instead of merely hoping the model obeys a formatting instruction.[8][9]

Three things still differ:

• Valid JSON: the text parses.
• Schema-conforming JSON: the fields, types and allowed values match the contract.
• A correct decision: the selected value is right for the input.

A perfectly formed {"route":"technical"} could still misclassify our billing request. Refusals and incomplete outputs also need handling; schema-constrained generation is not a claim that every response is a successful classification.[9]

The reason to use a generative LLM is flexibility. The same model may classify the message, explain a complicated policy, draft a response or help resolve an ambiguous case. But if the application only needs a category, producing an explanation may be unnecessary work. That is a reason to test a smaller decision mechanism, not proof it will win.

3. Jev: ask bounded questions about supplied state

TypeSafe's Jev interface starts with state: the text or structured information the questions refer to. You then supply questions and their permitted kinds of answers.[10]

For the same support example, that could mean:

State:
“I was charged twice. Please refund the duplicate.”

Question A:
Which support queue fits this request?
Choice options: billing / technical / other

Question B:
Is the customer requesting a refund?
Noul: probability that the answer is yes

The question's meaning belongs in its instructions, not just in a suggestive identifier. TypeSafe says question IDs are for your code and are not sent to the model.[10]

Jev's documented primitives are:

• Choice: the selected option, a distribution over the supplied options and a confidence statistic.
• Noul: one number representing the probability that a proposition is true. It is not a boolean plus a separate confidence score. Application code chooses how to use that probability.
• Score: an assessment against supplied levels. The returned score is a probability-weighted mean of level indices, so it can fall between them. It is not a tool for extracting an exact invoice amount.[10][11]

TypeSafe says the same weights serve every account; users shape the task through state, instructions and criteria rather than training a new per-account classifier or LoRA adapter.[12]

That is the useful product distinction: the caller programs the questions and answer space through the request.

What happens inside Jev?

Here is what TypeSafe's founder Diogo Almeida says in the launch article:

“Jev outputs all probabilities in parallel instead of autoregressively generating by token.”[13]

The article describes a new model architecture, a parallel sampler and Reinforcement Learning for Calibrated Decisions, or RLCD. The documentation describes training probabilities against outcomes, with calibration assessed over groups of predictions.[13][14]

This is a training objective claim as well as an interface claim. It is not simply a different name for JSON-constrained decoding.

But the reviewed sources do not disclose enough to draw Jev's layers. They do not establish its attention topology, pooling method, head layout, exact RLCD loss or parameter count. We cannot conclude that it is BERT inside, that it uses one head per question, or that a request requires exactly one neural-network forward pass.

That is why the Jev box in the diagram is deliberately opaque.

The API does give us another useful boundary: questions in the same request share state, but one answer does not become context for another.[10] If a later judgment depends on an earlier answer, arrange that dependency in application code or a subsequent request. Do not draw an invisible chain of reasoning between parallel outputs.

This API independence does not imply that the real-world events are statistically independent. “A refund was requested” and “this is a billing problem” can be related even if you ask about them separately.

Probabilities, correctness and permission are different things

A number between zero and one looks reassuring. It is not automatically a trustworthy estimate of being right.

A BERT classifier's softmax probabilities are normalized, but normalization alone does not establish calibration. A generative LLM can output a confidence number without that number being calibrated. TypeSafe's RLCD training target is not, by itself, evidence that Jev is calibrated on your application's traffic.[15][14]

Calibration asks whether predicted probabilities agree with observed frequencies across relevant examples. If a model assigns a proposition a probability of 0.8 across many comparable cases, does it hold about 80% of the time? That is a population-level test, not a promise about the next customer message.

Also, Jev's separate confidence field is documented as a statistic derived from the returned distribution. Do not silently substitute it for the probability that the selected answer is correct.[11]

Now consider TypeSafe's “zero hallucinations” claim. The launch article explains:

“Our number is not empirical. Schema matching is guaranteed, thus we can confidently add 0% into the plots.”[13]

The claim concerns staying inside the defined output space. It does not rule out the wrong valid answer. TypeSafe's own failure-mode documentation describes semantic errors and prompt-injection susceptibility.[16]

In the earlier Jev evaluation I asked Hermes Agent to run, an unrelated request to explain a monad was assigned product_question with a reported confidence of 0.99. The offered options lacked a no-match choice. That was a failure in the recipe's fallback design, not evidence that a correct available label was ignored. Adding an escape option and a scope check helped on those known development cases; it did not establish generalization or calibration.[17]

The same design risk exists with other classifiers. A model cannot select an answer you have excluded from its allowed output set.

Even a correct classification does not authorize an action. Recognizing a refund request does not establish that a duplicate payment occurred. Before issuing a refund, ordinary software must check the account, transaction evidence, eligibility and permission.

The model supplies a judgment. The application owns the action.

Which one would I choose?

For a stable classification problem with enough labeled examples, I would include a supervised encoder in the comparison. If simple rules or a conventional linear classifier solve it, they belong in the baseline too. A language model is not mandatory because the input happens to contain words.

For categories that change through natural-language descriptions, I would test an NLI-based zero-shot classifier, Jev and an LLM used as a classifier. They offer different contracts and operational trade-offs; “dynamic labels” alone does not establish a winner.

For a task that genuinely needs open-ended explanation, synthesis or a multi-step investigation, I would consider a generative LLM. Jev's documentation says it is not trained to generate text.[16]

I would not build a complicated routing system that calls all three models merely because all three exist. Start with the simplest mechanism that passes the application's tests. Add escalation when the evidence justifies it.

For agent infrastructure, the interesting test cases include intent routing, tool suggestions, content triage and escalation. A model may advise a policy gate, but permission checks must not depend only on its recommendation.

The useful buying metric is cost per acceptable application decision. Include wrong answers, retries, abstentions, human review, deployment and maintenance. A cheaper API call can be an expensive decision if the application acts on it incorrectly.

A fair comparison would test the same job

1. Define the task, error costs and an explicit escape route before tuning anything.
2. Prepare representative labeled examples. Separate training, calibration or development data from the final held-out test; adjudicate ambiguous labels rather than treating model agreement as truth.
3. Compare the fixed-label encoder, a request-time encoder baseline, Jev and a structured-output LLM where each is appropriate. Pin versions and record prompt, option and threshold changes.
4. Test two contracts where relevant: the same probability-rich interface, and the best practical implementation if the application only needs a label. Do not force competitors to generate unnecessary probabilities and then call that their unavoidable cost.
5. Measure errors by category, performance on no-match and adversarial inputs, calibration, abstention, end-to-end latency and total decision cost. Validate thresholds on data not used to select them.

This article proposes that evaluation. It does not report results from one.

A short check on the mental model

Before looking at the answers, predict what happens in these situations:

• You add a category to a conventional BERT classifier's configuration but do not train or evaluate the changed head. Has it learned the new category?
• An LLM returns schema-valid technical for the duplicate-charge message. What passed, and what failed?
• Jev returns high probability for a refund request. Which checks are still needed before money moves?

The first needs a learned and tested mapping, not a renamed output slot. In the second, the interface contract passed while the classification was wrong. In the third, classification leaves transaction truth, policy and authority unresolved.

To explain the distinction to someone else: trace the same message through the encoder and task head, the causal generation loop, and Jev's request-and-answer interface. Then identify what we still do not know about Jev's internals.

My view is that calling Jev “just a classifier” gets closer to its practical role than calling it a general reasoning-model replacement. But “just” misses the product question: can programmable, probability-producing judgments be a useful general-purpose building block?

That is worth testing against strong classifier baselines. We do not need to invent its architecture to take the question seriously.

Sources and further reading

[1] Devlin, Chang, Lee and Toutanova, BERT, §§3–4: https://aclanthology.org/N19-1423.pdf

[2] Vaswani and colleagues, Attention Is All You Need, §§3–4. Source for Transformer components and causal masking; the paper itself describes an encoder–decoder model: https://arxiv.org/pdf/1706.03762

[3] He, Liu, Gao and Chen, DeBERTa, §§2.1–2.2: https://arxiv.org/pdf/2006.03654

[4] Yin, Hay and Roth, Benchmarking Zero-shot Text Classification, §5: https://aclanthology.org/D19-1404.pdf

[5] Hugging Face, zero-shot classification implementation, pinned to Transformers v4.44.0: https://raw.githubusercontent.com/huggingface/transformers/v4.44.0/src/transformers/pipelines/zero_shot_classification.py

[6] Hugging Face, Causal language modeling: https://huggingface.co/docs/transformers/tasks/language_modeling

[7] Kwon and colleagues, Efficient Memory Management for Large Language Model Serving with PagedAttention, §§2.1–2.3: https://arxiv.org/pdf/2309.06180

[8] MLC Community, Structured generation with XGrammar: https://blog.mlc.ai/2024/11/22/achieving-efficient-flexible-portable-structured-generation-with-xgrammar

[9] OpenAI, Structured model outputs: https://developers.openai.com/api/docs/guides/structured-outputs.md

[10] TypeSafe, Primitives: https://docs.typesafe.ai/primitives

[11] TypeSafe, Confidence and primitive semantics: https://docs.typesafe.ai/confidence · https://docs.typesafe.ai/primitives/choice · https://docs.typesafe.ai/primitives/noul · https://docs.typesafe.ai/primitives/score

[12] TypeSafe, Models, including customization: https://docs.typesafe.ai/models

[13] Diogo Almeida, Introducing System One models and Jev: https://typesafe.ai/blog/introducing-system-one-models-and-jev

[14] TypeSafe, AI primer and System One: https://docs.typesafe.ai/introduction/machine-learning-primer · https://docs.typesafe.ai/concepts/system-one

[15] Guo, Pleiss, Sun and Weinberger, On Calibration of Modern Neural Networks, §§2 and 4: https://proceedings.mlr.press/v70/guo17a/guo17a.pdf

[16] TypeSafe, Jev 1.13 failure modes: https://docs.typesafe.ai/model-jaggedness/jev-1.13

[17] Sandeep, Jev: useful decisions, not guaranteed answers, evaluation conducted 19 September 2026. The accompanying final report is linked in this draft package. These are earlier diagnostic observations, not a BERT–LLM–Jev comparison: read the report.