Jev-class model Kev is available on CArouter (+ benchmarks)

Kev 4B is now available on CArouter at /v1/decisions. It is a decision model: it reads a text and answers typed questions about it with a probability for each answer, the way TypeSafe's hosted Jev does, and it runs on our own hardware in Canada.

We measured it on 328 new requests in English and French, then used its answers to pick a model for each request among the chat models we serve. Here is what it gets right, what it costs, and what choosing a model this way is worth.

The short version

• On the 137 requests we kept aside from every design choice, Kev named the kind of task correctly 98.5% of the time, 98.6% in English and 98.4% in French.
• Kev gave 76 of those answers with at least 90% confidence, and not one of them was wrong.
• Picking a model per request with Kev's answers came within two requests of the best single model, at 25% less cost and 35% less waiting.
• A three-question decision takes 146 milliseconds on a short request and 1.2 seconds on a 4,000-word one.
• It costs CAD 0.0532 per million input tokens, and output is free.

What Kev does

A chat model answers in sentences that someone then has to read. Kev never writes anything. You send it a state, which can be a text or a JSON object, with up to 32 questions of three types: a choice among named options, a score on ordered levels, or a yes or no.

Each answer comes back with its probabilities and a confidence. Kev reads everything in one pass and generates nothing, so it cannot invent an answer that is not among your options.

Here is a support email sent to Kev with three questions about it.

curl https://carouter.ai/v1/decisions \
  -H "Authorization: Bearer $CAROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "kev-4b",
    "state": "Hi, I was charged twice for invoice 2026-0417 this month. Can you refund the second charge before Friday? Our accounts close then.",
    "questions": {
      "intent": {"type": "choice", "instructions": "What is the customer asking for?",
                 "criteria": {"refund": "money back for a charge",
                              "invoice_copy": "a copy of an invoice",
                              "cancellation": "ending their subscription"}},
      "urgent": {"type": "noul", "instructions": "Does the customer need an answer within the week?"},
      "tone":   {"type": "score", "instructions": "How upset is the customer?",
                 "criteria": ["calm", "annoyed", "angry"]}
    }
  }'

Kev answered that the customer wants a refund, with 98% probability, that the matter is urgent, at 88%, and that the customer is annoyed rather than angry. It read 111 tokens to do so.

{
  "object": "decision",
  "model": "kev-4b",
  "answers": {
    "intent": {"type": "choice", "choice": "refund", "confidence": 0.9752,
               "probabilities": {"refund": 0.9835, "invoice_copy": 0.0077, "cancellation": 0.0088}},
    "urgent": {"type": "noul", "noul": 0.8772},
    "tone":   {"type": "score", "score": 1.1696, "confidence": 0.8253,
               "legend": {"0": "calm", "1": "annoyed", "2": "angry"},
               "probabilities": {"0": 0.0899, "1": 0.6507, "2": 0.2594}}
  },
  "usage": {"prompt_tokens": 111, "total_tokens": 111},
  "served_region": "CA-ON"
}

The request follows the shape of TypeSafe's SystemOne API, plus the model field every CArouter request carries. The state can run to 8,192 tokens. Past that, Kev refuses the request rather than cutting it, because it would keep the start of a long text and drop its end.

Same answers as in our research

Before opening Kev to customers, we replayed the 1,020 requests of our previous benchmark on the machine that now serves it. It chose the same answer as the test machine on 1,017 of them, and its accuracy matched to the third decimal in both languages.

The three differences were near-ties, under 45% confidence on either machine. The model you call is the model we measured.

A new benchmark, closer to real work

Public datasets do not contain what our customers send: a code review, a tax question about a Quebec invoice, an agent's next step after a tool result, a statute or a statistics release pasted whole. So we wrote 328 requests of that kind, half in English and half in French.

The French was written for a reader in Quebec or Ontario, not translated from the English. The long documents come from openly licensed sources such as federal statutes, Statistics Canada releases, Wikipedia and public-domain books. Every request records where its words come from.

We asked Kev the three questions an automatic model selector needs: what kind of task this is, how hard it is, and whether a good answer will run long. It saw the start of the instructions, the tool names, and the beginning and end of the latest message, or the user's task when that message is a tool result.

We wrote those questions and the routing rules on three fifths of the requests. Every figure below comes from the other two fifths: 137 requests that none of those choices saw.

What we found

Kev named the kind of task correctly 98.5% of the time on those 137 requests. It was right on every request in seven of the nine kinds, and missed one research request and one chat request.

Agent steps are read from the request itself. A request that carries tool definitions is one step of an agent's work, and the router sees that the way it sees the language and the length. Kev alone recognized half of them, because the latest message is a tool result that reads like code or a web page.

The line between a quick question and a research question is a judgment call, and the labels are ours. A different labeller could move those two figures by a few points.

Honest about its doubts

For a model that decides on someone's behalf, the number that matters most is how often it is wrong while sure of itself. Kev gave 76 answers with at least 90% confidence, and all 76 were right.

Every answer Kev gave with at least 60% confidence was right, and 86% of its answers reached that bar. Below it, a selector should send the request to a default model rather than trust the reading.

Difficulty and long answers

Kev places a request within one level of our difficulty rating 85% of the time. It is the least precise of the three readings, and our routing table does not use it yet.

A request that states its own length, such as at least 1,000 words or a complete module, is read as needing a long answer from its wording. Together with Kev, that caught 3 of the 5 held-out requests that need one, and long answers still need work.

Picking the model for each request

We then ran every request through the four chat models we serve from Canada: Qwen3.6 35B A3B and Qwen3 0.6B on our own hardware, and Augure's Tofino 3 and Rosedale 1. Each answer was graded by a test, an exact result or a rubric.

Code ran against its tests, while math and extraction were compared with the right answer. Prose was scored against a rubric by Qwen3.6 35B A3B with its thinking off, a judge that failed 2 of 122 correct answers and passed none of 122 wrong ones.

From the three fifths used for design, we wrote a routing table by kind of work. Tofino 3 takes agent steps, math, translation and chat, where it ties or leads in seconds and at a fraction of the price. Qwen3.6 takes everything else, as well as any request Kev is unsure about.

On the 137 held-out requests, picking the model with Kev passed 81.8% of answers, against 83.2% for the best single model, Qwen3.6. That is two requests apart, at 25% less cost, and each answer took 33 seconds on average instead of 51.

The ceiling is higher. Choosing the best model for each request after the fact would pass 94.2% of answers, so a table by kind of work leaves quality that a finer policy could recover.

Speed and cost

Kev runs on an NVIDIA GB10 in Ontario that also serves our other open models. It holds under 9 GB of memory and handles one request at a time.

A short decision takes about 150 milliseconds, and a 4,000-word request about 1.2 seconds. The datacenter GPU of our previous test was faster on short requests and close on long ones.

The price is CAD 0.0532 per million input tokens, and output is free. That is the price TypeSafe publishes for Jev, less ten percent. A three-question decision on a 500-token request costs less than three thousandths of a cent.

Kev bills every token it reads: the state once, and each question with its options. TypeSafe does not publish how Jev counts its tokens, so the two prices are close but not proven identical.

Privacy

Kev keeps nothing. Each request is read once and forgotten with its answer, and nothing is shared between customers, not even a cache whose timing could reveal what someone else sent.

Requests are served in Canada. On a key that requires our PII filter, personal information in the state is pseudonymized before Kev sees it.

What this does not prove

• This is one run on 328 requests that we wrote and labelled ourselves, and the figures come from the 137 we kept aside. In a kind with 10 held-out requests, each mistake moves the figure by ten points.
• The judge is also one of the four models it grades, which may flatter Qwen3.6 on prose. We checked the judge on answers we knew to be right or wrong, not on subtle mistakes.
• The selection was simulated from Kev's answers and our routing table. The gateway's own auto still has to pass the same test before it picks anything for you.
• The latency figures come from a client on the same machine. A request through our API from elsewhere in Canada adds the network time.

What comes next

Kev is the first half of auto, the mode where a request names no model and CArouter picks one at the price or speed you prefer. Before auto picks anything on your behalf, we want a finer routing policy that closes more of the gap to 94%, better detection of long answers, and our own auto passing this test.

In the meantime, Kev is yours to call directly, for support triage, lead qualification, routing or any decision you would rather not leave to a chat model's prose. Every measurement behind this article is kept, and we will share it with anyone who asks.