Back to Blog

Qwen4-Max on 175 Coding Tasks: 96% Solved, at the Same Cost as Claude Sonnet 5.5, Which Solved 100%

2026-11-16·3 min read·CodeRouter Team

Disclosure: we build CodeRouter, a router for coding agents. We called Qwen4-Max through OpenRouter, where Alibaba is its only provider. The other models' numbers are the ones already on our benchmark page, measured on November 3.

Qwen4-Max went live on OpenRouter on November 16 (UTC): $1.50 per million input tokens, $4.50 per million output, a 1M-token context, reasoning on by default, and no open weights. A cheaper frontier-class model is exactly what a router looks for, so we ran it the same day on the same tasks as every model on our benchmark page.

Qwen4-Max solved 96.0% of the tasks with one retry, 83.4% on the first try, for $1.47. That's what Claude Sonnet 5.5 cost, and Sonnet solved 100%.

The results

Same 175 single-file Python tasks (Exercism plus Aider's hard set). The model sees the instructions and the stub file and returns the whole file; we run the tests, and on a failure it gets one retry with the failing output.

Model First try With one retry Cost of all 175 tasks Median time per task
Claude Opus 5.5 98.3% 100% $2.61 4.8 s
Claude Sonnet 5.5 93.7% 100% $1.48 4.0 s
Qwen4-Max 83.4% 96.0% $1.47 11.3 s
Kimi K2.6 86.9% 94.9% $0.57 9.4 s
Claude Haiku 5.5 72.0% 93.7% $0.15 5.5 s

Costs are list prices applied to the tokens each request reported, retries included.

It missed 7 tasks: paasio and tree-building in both task sets, plus dot-dsl, go-counting and pov from Aider's hard set. It never hit our 32,768-token output limit and returned no errors.

Why a cheaper price list didn't mean a cheaper run

Per token, Qwen4-Max is cheaper than Sonnet 5.5: 25% less for input ($1.50 vs $2) and 55% less for output ($4.50 vs $10). But it wrote 272,000 output tokens on these tasks against Sonnet's 109,000, and 72% of them were reasoning. The extra output ate the whole discount.

The reasoning scales with the task. Half of its first answers reasoned for under 200 tokens, one in ten for over 2,600, and the longest for 14,200. It also makes Qwen4-Max slower: a median of 11.3 seconds per task against 4 to 5.5 for the Claude models.

What it means for routing

Against what CodeRouter already uses, Qwen4-Max doesn't earn a slot on this suite:

So we're not adding it to routing now. Single-file tasks aren't long agent sessions, and a model that reasons more can look different there; that would take an agent run to find out.

Limits

Per-task data: coderouter-qwen4max-175-2026-11.json. The other models' data is on the benchmark page.

Try phase-aware routing on your own workload

One base URL for Claude Code, Cursor or Aider. Free trial, no credit card.

Start free trial

Get weekly AI cost optimization tips

Join developers cutting their agent bills