# Clef and Strands Decider vs Kev-4B: benchmark on a GB10 > Cloudflare's Clef and Strands Labs' Strands Decider are two new open decision models. On a GB10, Clef named the kind of task right on 94.5% of our 328 requests against 92.4% for Kev-4B, at five times the wait and nearly four times the memory, and OpenJev's untrained Qwen3.5-4B still leads at 97.0%. - Published: 2026-10-01 - Author: Flavien Berwick - Category: Research - Web page: https://carouter.ai/blog/clef-strands-decider-vs-kev - In French: https://carouter.ai/fr/blog/clef-strands-decider-vs-kev.md ![Research: Clef and Strands Decider vs Kev-4B, benchmark on a GB10. Kind of task named right on our 328 requests: Clef 94.5% in 562 ms and 52 GB, Clef-flash 89.6% in 173 ms and 18 GB, Kev-4B 92.4% in 101 ms and 14 GB, Strands Decider 87.2% in 108 ms and 4 GB. OpenJev's untrained Qwen3.5-4B still leads at 97.0%, in 192 ms and 9 GB.](https://carouter.ai/blog/clef-strands-decider-vs-kev/illustration.png) Two new open decision models have just been released. Cloudflare published Clef and its smaller sibling Clef-flash, and Strands Labs published Strands Decider. All three answer typed questions about a text the way Jev and Kev do. We ran them beside Kev on our two benchmarks, on an NVIDIA GB10, the machine that serves Kev-4B at /v1/decisions. We measured how often each one is right, how often it is wrong while sure, how long a decision takes and how much memory it holds. [Read the first benchmark.](https://carouter.ai/blog/jev-kev-laya-decision-models) [Read the second, with Kev's launch.](https://carouter.ai/blog/kev-available-on-carouter) [Read the third, on OpenJev.](https://carouter.ai/blog/openjev-vs-kev) ## The short version - **On our requests.** Clef named the kind of task right on 94.5% of our 328 requests, against 92.4% for Kev-4B, 89.6% for Clef-flash and 87.2% for Strands Decider. - **Where Clef gains.** The gap is seven requests, and it comes from agent steps: Clef recognised 28 of 32 and Kev-4B 16. CARouter reads an agent step from the request itself, and scored that way Kev-4B is at 97.3% and Clef at 95.7%. - **On public prompts.** The three newcomers reach 80 to 82% in English, level with the hosted Jev and ahead of Kev-4B at 76%. In French, Clef-flash leads at 78% and Strands Decider falls to 68%. - **Confidence.** On public prompts, Clef and Clef-flash were wrong while at least 90% sure on 8% and 9% of requests, against 12% for Jev, 3% for Kev-4B and 0.2% for Strands Decider. - **Speed.** A three-question decision on a short request takes 101 milliseconds on Kev-4B, 108 on Strands Decider, 173 on Clef-flash and 562 on Clef. - **Memory.** Clef holds 52 GB of the GB10's 121, Clef-flash 18, Kev-4B 14 and Strands Decider 4. - **The model nobody trained.** OpenJev's Qwen3.5-4B, run again on the same machine, still leads on our requests at 97.0%, in 192 milliseconds and 9 GB. ## What the newcomers are Clef is Cloudflare's family of decision models, released under the Apache 2.0 licence with its weights on Hugging Face. Clef is built on Qwen3.8-27B and Clef-flash on Qwen3.5-9B. [Cloudflare's announcement.](https://blog.cloudflare.com/clef-decision-models/) [Clef's weights.](https://huggingface.co/Cloudflare/clef) Cloudflare added a small head on top of each chat model. It reads the model's last layer and scores every option of every question in one pass. Clef takes the same request as Jev, and it can also read images and video, which we did not test. Strands Decider comes from Strands Labs, the experimental side of the Strands Agents project, also under Apache 2.0. It has 1.9 billion parameters: an adapter and a small head on Qwen3.5-2B-Base. That is how Kev is built, on a base that sits between Kev-0.8B and Kev-4B. [Source code.](https://github.com/strands-labs/strands-decider) The three read different lengths of text. Strands Decider's window is 4,096 tokens, Kev's is 8,192 as we serve it, and Clef's is 16,384 as released. Clef and Strands Decider cut a text that is too long and keep its beginning, where our Kev engine refuses the request. None of our test requests reached those limits. ## How we tested The test machine is a GB10, the same model as the one that serves Kev, with 121 GB of memory shared between the processor and the GPU. Each model ran alone on it, one request at a time, through its authors' own code. Kev ran the way our server runs it. The two benchmarks are those of the earlier articles. On the 1,020 public prompts, each model answers one question with nine kinds of task. On our 328 requests it answers three: the kind of task, how hard it is, and whether the answer will be long. One script scored every model, the one of the previous article. Before scoring anything new, it recomputes the published figures of the earlier articles and stops if one differs. We checked the setup three ways. Strands Decider gave its model card's answers on the card's own example, to within 0.004. OpenJev gave its authors' published answer on all 144 of its test decisions. Kev-4B on the test machine chose the same answer as in production for 1,015 of the 1,020 public prompts and 327 of our 328 requests. Cloudflare publishes no answer-by-answer results for Clef, so we could not check it the same way. It loaded as its card describes and gave sensible answers to the card's examples. Jev is a paid hosted service that we did not call again. Its figures on the public prompts are those of the first benchmark, and it has never been run on our requests. One thing changed since the OpenJev article. We reworded two options of the question about the kind of task, chat and research, and an agent step now shows the user's task beside the tool's result. Every model here answered the new wording. That article's figures on our requests answered the old wording and do not compare with these. With the new one, Kev-4B went from 88.7% to 92.4%, and OpenJev's Qwen3.5-4B from 95.7% to 97.0%. ## What we found: accuracy ![Kind of task named right, in English and in French. On the 1,020 public prompts: Jev 81% and 76%, Kev-9B 74% and 67%, Kev-4B 76% and 70%, Kev-0.8B 70% and 64%, OpenJev Qwen3.5-4B 79% and 81%, Strands Decider 80% and 68%, Clef-flash 82% and 78%, Clef 82% and 76%. On our 328 requests: Kev-4B 93% and 92%, Kev-0.8B 80% and 81%, OpenJev Qwen3.5-4B 97% and 97%, Strands Decider 89% and 85%, Clef-flash 90% and 90%, Clef 95% and 94%. Jev and Kev-9B were not run on our requests.](https://carouter.ai/blog/clef-strands-decider-vs-kev/01-accuracy-both-benchmarks.png) On our requests, Clef is the most accurate of the trained models: 310 right out of 328, against 303 for Kev-4B. Clef-flash got 294 and Strands Decider 286. On the public prompts the order changes. Clef-flash, Clef and Strands Decider reach 82%, 82% and 80% in English, around the hosted Jev at 81%, and Kev-4B stays at 76%. French separates them. Clef-flash keeps 78% and Clef 76%, as Jev does. Strands Decider drops to 68%, below Kev-4B at 70%. On our requests it loses less: 89% in English and 85% in French. **Every model on both benchmarks, and on the GB10** | Model | Public prompts, English | Public prompts, French | Confidently wrong, public, English | Our 328 requests | Short decision | Memory | | --- | --- | --- | --- | --- | --- | --- | | Jev (hosted) | 80.9% | 76.0% | 11.7% | not run | not run | not run | | Kev-9B | 74.3% | 66.5% | 4.8% | not run | not run | not run | | Kev-4B | 75.7% | 69.8% | 3.3% | 92.4% | 101 ms | 14 GB | | Kev-0.8B | 69.6% | 63.7% | 0.0% | 80.2% | 23 ms | 4 GB | | OpenJev Qwen3.5-4B | 79.3% | 80.8% | 19.3% | 97.0% | 192 ms | 9 GB | | Strands Decider 2B | 80.4% | 67.7% | 0.2% | 87.2% | 108 ms | 4 GB | | Clef-flash (9B) | 82.0% | 77.9% | 8.9% | 89.6% | 173 ms | 18 GB | | Clef (27B) | 81.9% | 75.8% | 8.1% | 94.5% | 562 ms | 52 GB | We wrote the questions about our requests with Kev-4B's answers in front of us, on three fifths of the requests. On the 137 requests kept aside, the order is the same: OpenJev 98.5%, Clef 95.6%, Kev-4B 93.4%, Clef-flash 91.2% and Strands Decider 88.3%. ## Where the differences come from ![Kind of task named right on our 328 requests, by kind, for Kev-4B, OpenJev Qwen3.5-4B, Strands Decider, Clef-flash and Clef. Chat: 87%, 100%, 57%, 93% and 83%. Agent step: 50%, 94%, 59%, 44% and 88%. Research: 82%, 75%, 89%, 50% and 71%. Summarization: 100%, 100%, 95%, 100% and 100%. Extraction: 100%, 97%, 63%, 100% and 97%. All five reach 100% on code, math, business writing and translation.](https://carouter.ai/blog/clef-strands-decider-vs-kev/03-by-kind.png) Every model is perfect on code, math, business writing and translation. The differences sit in three kinds of request: agent steps, casual chat and research. Agent steps are where Clef gains. It recognised 28 of 32, where Strands Decider recognised 19, Kev-4B 16 and Clef-flash 14. The latest message of an agent step is a tool's result, which reads like code or a web page. It is also the one kind CARouter does not ask a model about. A request that carries tool definitions is an agent step, and our platform sees that the way it sees the language. Scored that way, Kev-4B names 97.3% of the requests, Clef 95.7%, Clef-flash 95.1% and Strands Decider 91.2%. On research, Strands Decider and Kev-4B do better than both Clef models: 89% and 82%, against 71% for Clef and 50% for Clef-flash, which files a research request under extraction or summarization. Strands Decider's weak kinds are chat, at 57%, and extraction, at 63%. ## What we found: confidence A model that decides on someone's behalf should act only when it is sure. So we count how often each one is wrong while at least 90% sure. ![Confidence on both benchmarks. On the public prompts in English, accuracy against wrong answers given at 90% confidence or more: Jev 81% right and 12% confidently wrong, Kev-9B 74% and 5%, Kev-4B 76% and 3%, Kev-0.8B 70% and 0%, OpenJev Qwen3.5-4B 79% and 19%, Strands Decider 80% and 0.2%, Clef-flash 82% and 9%, Clef 82% and 8%. On our 328 requests, answers given at 90% confidence or more: Kev-4B 187 with none wrong, Kev-0.8B 2 with none wrong, OpenJev Qwen3.5-4B 292 with 4 wrong, Strands Decider 131 with none wrong, Clef-flash 227 with 2 wrong, Clef 213 with none wrong.](https://carouter.ai/blog/clef-strands-decider-vs-kev/02-confidence.png) On the public prompts, Clef and Clef-flash are confidently wrong on 8% and 9% of the English requests. That is better than Jev at 12% and worse than Kev-4B at 3%. Strands Decider was confidently wrong once in 540 requests. On our requests almost nobody is wrong while sure, and the models differ in how often they are sure. Clef-flash gave 227 answers at that level and two were wrong. Clef gave 213, Kev-4B 187 and Strands Decider 131, with none wrong. Each model computes its confidence its own way. Kev and Strands Decider correct it for the number of options, and Clef reports the probability of its first choice. Recomputed with Kev's formula, Clef is that sure of 197 answers and Clef-flash of 218, and the order does not change. ## Difficulty and long answers Clef and Clef-flash are the best at rating difficulty. Both place a request within one level of our rating 98% of the time, against 90% for Strands Decider and 88% for Kev-4B. On the exact level they reach 46%, against 41% for Kev-4B. Ten of our 328 requests need a long answer. Clef flagged all ten and two others, and Kev-4B flagged seven and no other. Clef-flash also found the ten and flagged 13 others. Strands Decider found eight and flagged 58 others. ## Speed and memory on a GB10 We timed the three-question decision of the earlier articles on a new request each time, from a short request to a 4,000-token one. ![Median time of a three-question decision by prompt length on the GB10, on log scales, and memory held at its peak. Kev-0.8B: 23 to 171 ms, 4.0 GB. Kev-4B: 101 to 639 ms, 14.4 GB. Strands Decider: 108 to 820 ms, 4.0 GB, its longest prompt cut at its 4,096-token window. Clef-flash: 173 ms to 1.4 s, 18.4 GB. OpenJev Qwen3.5-4B: 192 ms to 1.1 s, 8.7 GB. Clef: 562 ms to 4.4 s, 52.1 GB. For reference, Kev-4B as served in production: 143 ms to 1 s.](https://carouter.ai/blog/clef-strands-decider-vs-kev/04-latency-memory-gb10.png) Kev-4B is the fastest of the larger models, at 101 milliseconds on a short request and 639 on a long one. Strands Decider, half its size, takes 108 and 820 milliseconds. Kev's serving code prepares its computations ahead of time, and Strands Decider's reference code does not. Clef-flash takes 173 milliseconds and 1.4 seconds. Clef takes 562 milliseconds and 4.4 seconds, five to seven times longer than Kev-4B. Cloudflare reports a median of 209 milliseconds for Clef and 39 for Clef-flash in its own tests. On the GB10, a one-question decision on the public prompts took a median of 428 and 141 milliseconds. A GB10 is a desktop machine, not a datacenter GPU. Memory matters more on this machine, because the processor and the GPU share one pool. Clef holds 52 GB, more than two fifths of it, and takes about six minutes to load. Clef-flash holds 18 GB, Kev-4B 14 GB and Strands Decider 4 GB, the same as Kev-0.8B. These timings are the best case for every model: alone on the machine, with no network in between. In production, Kev-4B shares its GB10 with other models and answers through an API, and the same decision takes 143 milliseconds and 1 second. ## The model nobody trained The previous article left a question open: how fast OpenJev's Qwen3.5-4B would be on the GPU that serves Kev. OpenJev trains nothing and asks an ordinary chat model to pick an option letter. We ran it again on the same machine, with the reworded questions. It is still the most accurate model on our requests: 97.0%, the same in English and in French, with every casual chat and 30 of the 32 agent steps recognised. With agent steps read from the request, it is at 97.6% and Kev-4B at 97.3%, one request apart. It is also still the most confident. It gave 292 answers at 90% confidence or more, and four were wrong. On the public prompts it is confidently wrong on 19% of the requests, the highest figure in the table. On the GB10's GPU, its three-question decision takes 192 milliseconds on a short request and 1.1 seconds on a long one, in 9 GB. That is about twice Kev-4B's time on a short request, and faster than Clef. ## What this does not prove - This is one run per model on two sets of requests. A kind holds 20 to 66 of our requests, and on the smallest, one mistake moves the figure by five points. - We wrote and labelled the 328 requests, and we tuned their questions on Kev-4B. Both benchmarks ask one thing: what kind of work a request is. Cloudflare's own tables cover some forty other tasks, from tool selection to contract reading, and put Clef ahead of Jev and Kev-9B on most of them. - Clef and Strands Decider ran through the code their authors published. Kev's serving code is tuned for speed, and so, presumably, is what Cloudflare runs in production. The time gaps say as much about the software as about the models. - We tested text only. Clef also reads images and video, and no other model here does. - Jev was not called again. Its figures come from the first benchmark, and it was never run on our requests. ## What this means for CARouter Kev-4B stays behind /v1/decisions. Clef is more accurate on our requests only where our platform already knows the answer. On a machine that serves other models, it would take 52 GB and five times the wait. Clef-flash is the more practical of the two. It leads on the public prompts in both languages and rates difficulty better than Kev, for 18 GB and under 200 milliseconds on a short request. It also reads half of our research requests as something else. Strands Decider is the one to watch at the small end. In the 4 GB that Kev-0.8B holds, it is seven points more accurate on our requests and eleven on the English public prompts, and it was never wrong while sure on ours. It is also almost five times slower than Kev-0.8B, and weaker in French. The open question is still the one of the last article. A Qwen3.5-4B that nobody trained reads our requests best, and we now know what it costs on the GPU: 192 milliseconds and 9 GB. Its confidence on other kinds of requests still needs checking, and the full selection test still has to be run with it. Every measurement behind this article is kept, and we will share it with anyone who asks. --- More articles: [CARouter Blog](https://carouter.ai/blog.md). What CARouter is and how to call it: [llms.txt](https://carouter.ai/llms.txt).