# Sparset Thinklite-27B model benchmark: speed and quality > Sparset's Thinklite-27B is Qwen3.8-27B trained to reason in fewer tokens. At 4 bits on an RTX PRO 6000 Blackwell, it wrote half as many tokens as Qwen3.8-27B on our five tests at the same speed per token, so a science answer took 83 seconds instead of 162. It scored as well on code, tool calls and long documents, and followed instructions a little less well, 91% against 94%. - Published: 2026-11-09 - Author: Flavien Berwick - Category: Research - Web page: https://carouter.ai/blog/sparset-thinklite-27b-model-benchmark-speed-and-quality - In French: https://carouter.ai/fr/blog/sparset-thinklite-27b-model-benchmark-speed-and-quality.md ![Research: Thinklite-27B, speed and quality. Tokens written across five tests: 50% fewer than Qwen3.8-27B. One request: 70 tokens a second. Seconds to a science answer: 83 against 162 for Qwen3.8-27B. Both at 4 bits with llama.cpp on an RTX PRO 6000 Blackwell.](https://carouter.ai/blog/sparset-thinklite-27b-model-benchmark-speed-and-quality/illustration.png) Sparset released Thinklite-27B on November 4. It is Qwen3.8-27B, trained on its own correct answers to reach them by a shorter route, so that the reasoning gets shorter while the answer keeps its length and format. Sparset publishes one build, at 4 bits, for llama.cpp. We ran it against Qwen3.8-27B in a 4-bit build of the same size, on the same engine and an RTX PRO 6000 Blackwell, a workstation card with 96 GB, and measured its speed and our five quality tests. [Why a dense 27B model is slow to answer: why we now serve Qwen3.6 35B-A3B instead of Qwen3.8 27B.](https://carouter.ai/blog/qwen3-6-35b-a3b-instead-of-qwen3-8-27b) ## The short version - **Tokens.** Across the five tests, Thinklite wrote 2.17 million tokens and Qwen3.8-27B 4.33 million, reasoning included: half as many. - **Speed.** Per token the two run alike, 70 tokens a second for one request against 73.5. Writing half the tokens, Thinklite finishes a science answer in 83 seconds on average, against 162. - **Quality.** Thinklite codes, calls tools and finds facts in long documents as well as Qwen. It follows instructions a little less well, 91% against 94%, and answers more science questions within our token limit, 87% against 81%. - **Memory.** The 4-bit weights fill 15.4 GiB, so one card holds the model with ample room for long contexts. ## Half the tokens ![Tokens written per answer, reasoning included, mean, Thinklite-27B then Qwen3.8-27B: coding 3 237 and 12 331, following instructions 1 331 and 2 087, calling tools 265 and 426, science questions 5 813 and 11 905, finding a fact in a long text 289 and 812.](https://carouter.ai/blog/sparset-thinklite-27b-model-benchmark-speed-and-quality/01-tokens-per-answer.png) A reasoning model writes its thinking before its answer, and on a hard question the thinking is most of the text. Thinklite shortens it on every test we ran: 74% fewer tokens on coding problems, 36% on instructions, 38% on tool calls, 51% on science questions and 64% on the long-text test. The cut is deepest where Qwen thinks longest. On the questions both models got right, Thinklite used 38% of Qwen's tokens for science and 25% for code. Sparset reports 48% fewer reasoning tokens on its own six tests. We count everything a model writes, answers included, and measured 50% fewer on ours. ## The same speed per token ![One request alone, seconds to the whole answer, mean, Thinklite-27B then Qwen3.8-27B: coding 46 and 168, following instructions 19 and 28, calling tools 3.8 and 5.8, science questions 83 and 162, finding a fact in a long text 4.1 and 11.](https://carouter.ai/blog/sparset-thinklite-27b-model-benchmark-speed-and-quality/03-seconds-per-answer.png) Thinklite keeps Qwen3.8-27B's architecture and size, so on the same engine it writes at the same pace: 70 tokens a second for one request against 73.5, and 574 against 605 in total for sixteen requests at once. The 5% gap most likely comes from the way each file was quantized. A dense model reads all of its weights from memory for every token it writes. A 16.5 GB file on a card that moves 1.8 TB a second tops out near 108 tokens a second for one request, and both models reach about two thirds of that. The time to a whole answer therefore follows the token count. One request waits 83 seconds for a science answer from Thinklite and 162 from Qwen, and 46 seconds for a coding answer against 168. The same holds with many requests at once. Twelve at a time on the same model of card, Thinklite went through the 198 science questions in 89 minutes and Qwen in 3 hours 36 minutes. Both read an 11 810-token prompt at 3 800 to 3 900 tokens a second, so the first token arrives after about 3 seconds. ## Quality ![Share of each test answered correctly, both models at 4 bits, Thinklite-27B then Qwen3.8-27B: coding 70% and 58%, following instructions 91% and 94%, calling tools 82% and 80%, science questions 87% and 81%, finding a fact in a long text 100% and 100%.](https://carouter.ai/blog/sparset-thinklite-27b-model-benchmark-speed-and-quality/02-five-tests.png) We ran our five tests on both models with the same prompts, and each answer could use up to 32 768 tokens, 8 192 for the long-text test. Science questions show what a shorter route buys within a limit. On GPQA Diamond, Thinklite answered 173 of 198 correctly and Qwen 160. Qwen's reasoning reached our limit on 29 questions, against 9 for Thinklite, and an answer cut off counts as wrong. On the 169 questions where neither model ran out, they scored 158 and 160, a tie. Instructions are where the training cost something. On IFEval's 541 prompts, Thinklite followed every instruction of 493 and Qwen of 508, a gap too wide for chance. Coding leans Thinklite's way: it solved 35 of 50 BigCodeBench problems and Qwen 29, though on 50 problems chance could still produce a gap of that size. Tool calls are level, 410 of 500 against 402, and both found a sentence hidden in documents of 16 000 to 100 000 tokens 12 times out of 12. ## How we measured Thinklite ran from Sparset's GGUF file, a 4-bit build with an importance matrix, 16.55 GB. Qwen3.8-27B ran from unsloth's UD-Q4_K_M build, 16.46 GB, the closest size to it. Our benchmark feature serves models with vLLM and does not take GGUF files, so we ran the same tests by hand on rented RTX PRO 6000 Blackwell cards, with llama.cpp v0.9.0 built for them. Speed comes from one card, each model alone. The quality tests ran on two cards, and a score does not depend on which. Both models reasoned at their default effort, the highest, with the sampling Sparset used for both in its own comparison: temperature 1.0, top-p 0.95 and top-k 20. The speed probe keeps 1 to 16 requests running for 40 seconds after a 20-second warm-up and counts every token written in that window. The one-request figure is the mean of three runs. The figures are also on our benchmarks page, beside every other model we measured. [See the benchmarks.](https://carouter.ai/benchmarks) ## What remains open - Lower efforts. Both models accept a lower reasoning effort, which also shortens the thinking, and we tested both at the highest. - A larger limit. With more than 32 768 tokens per answer, Qwen would finish more of its science answers. - Other builds. Sparset also publishes the full-precision model, which vLLM can serve, and we measured the 4-bit file on llama.cpp. - Agentic work. Our tests read single requests, and a long agentic session would compound both the savings and the gap on instructions. --- More articles: [CArouter Blog](https://carouter.ai/blog.md). What CArouter is and how to call it: [llms.txt](https://carouter.ai/llms.txt).