Qwen 3.8 Reasoning Turns 137 Seconds Into 21 Minutes

Qwen 3.8 27B spent 21 minutes generating a pelican SVG that took 137 seconds with reasoning disabled. Simon Willison’s hands-on tests expose the model’s central problem: not whether it can complete a task, but whether it can judge how much thought that task requires.

The official model card describes an Apache 2-licensed, 27-billion-parameter model with vision, code generation, tool use and a maximum context length of 262,144 tokens. Willison ran it from a 17GB quantized file and documented its default xhigh reasoning setting in his test report.

That default shifts a resource-allocation decision from the user to the model. The risk is not simply slower output. Routine work can consume the same deep-reasoning budget as tasks that require several dependent steps. It is like calling an architecture committee to choose a door handle.

Willison tested a 17GB Q4_K_M build through LM Studio on a 128GB M5 Max MacBook Pro and an NVIDIA DGX Spark, with additional llama-server runs on the Spark. His prompts covered SVG generation, image bounding boxes, browser-tool creation and a coding-agent workflow using Pi.

Pi and LM Studio served different roles in those tests. Pi drove the coding-agent workflow. LM Studio ran the local model. Keeping that distinction clear matters because agent behavior and model-serving performance are separate parts of the stack.

Serving configuration affected the results before output quality entered the picture. According to Willison’s account, LM Studio’s default 8,192-token context limit was exhausted by reasoning on simple requests. Raising the limit to 262,144 tokens prevented that failure, but did not remove the computation spent on reasoning.

More context therefore solved a capacity problem, not an efficiency problem. A larger token allowance lets the model continue thinking; it does not establish that the extra thinking is useful. Treating those failures as one issue would lead builders to tune the wrong control.

The clearest cost comparison came from the pelican-on-a-bicycle prompt. Willison recorded 22,276 reasoning tokens, 3,223 output tokens and a 21-minute runtime with default reasoning. Disabling reasoning produced 3,715 output tokens in 137 seconds. He judged the slower result his best locally generated pelican SVG, but not good enough to justify a wait roughly nine times longer.

The vision-interface task showed why reasoning cannot simply be disabled globally. In Willison’s test, the reasoning-enabled run correctly scaled 0–1000 image coordinates and rendered labelled bounding boxes, although it added an unrequested sample scene. The reasoning-disabled version nearly worked but placed the boxes incorrectly.

Those two prompts reveal the routing problem. Narrow generation benefited from speed, while multi-step implementation benefited from additional reasoning. One default cannot price both tasks efficiently because the cost of extra thought depends on what failure looks like for each request.

Willison also reported that Qwen completed a multi-file coding-agent loop in his Datasette project and created and tested a Python converter for a Pi JSONL transcript. Those runs matter because local agents require long context, code generation and dependable tool calls from one model. The benchmarks reported by Qwen do not establish that full workflow on their own.

The evidence has clear limits. As documented in Willison’s test report, this was one user’s prompt set across two machines, not a controlled comparison. The model was quantized, the reasoning configurations produced different artifacts and output quality was assessed from a small number of runs. Qwen’s published benchmark claims were not independently tested in this exercise.

That limits what the latency figures can prove. They show that configuration can dominate the user experience, but they do not produce a universal threshold for when reasoning becomes worthwhile. The practical lesson is to measure task success and delay together rather than treating either as a complete quality metric.

Serving choices add another layer. Willison measured about 15–30 tokens per second in LM Studio and reported that llama.cpp’s Multi-Token Prediction mode on the Spark was around 72% faster than the default LM Studio GGUF setup. His assessment partly linked the remaining limit to memory bandwidth because the model uses a dense architecture rather than a Mixture-of-Experts design.

Builders should therefore treat reasoning effort as a task-level control, not a universal quality switch. Lower settings fit formatting, extraction and simple generation. Deeper reasoning makes more sense for coding agents, tool orchestration and work where a failed first attempt costs more than added latency. Context limits and MTP serving should also be tested before poor speed is attributed solely to the model.

Qwen 3.8 27B puts vision, coding and tool use into a local, open-weight 17GB package. The specific question is whether runtimes can choose the right reasoning depth per task, preserving success on difficult work without turning a simple SVG into a 21-minute wait.