Strands Decider vs Sonnet: Does a 98% Saving Hold Up?

The questions this answers: is the 98% saving real, why not just use Haiku 5.5, how many tickets before a GPU endpoint pays off, and did accuracy hold? Official facts, user reports and our opinion are kept apart.

Official facts

Checked on 11 November 2026. The post is from the Strands Agents blog (AWS), written with the two Amazon engineers, so it is a vendor case study about a vendor product.

Item	Fact	Source
What changed	Sonnet 4.6 (Bedrock, global endpoint) labelling tickets into 20 categories was replaced by Strands Decider E4B, a Gemma 4 build	Strands blog
What a decision model is	It cannot generate text: given a state and a question with options, it returns a probability for each option plus a confidence score	Strands blog, repository
Licence	Apache-2.0; the repository was created on 29 September 2026	Repository
Hardware used	SageMaker ml.g5.xlarge (one A10G, 24 GB), about 19 GiB used, about 640 ms per request	Strands blog
Context window	4,096 tokens: the old per-label descriptions were longer than that, so each of the 20 labels became a one or two sentence criterion, and long tickets drop their oldest comments	Strands blog
Claimed result	"Performed just as well at 98% less cost"; "save millions of dollars of token costs in a year"	Strands blog

What did Amazon measure?

Amazon's own table, from the same tickets:

Measure	Claude Sonnet 4.6	Strands Decider E4B
Average time per call	4,215 ms	897 ms
Input tokens per ticket	6,554	2,009
Output tokens per ticket	170	1
Cost per ticket	$0.0222	about $0.0005
Cost per million tickets	about $22,200	about $500

Prices used: Sonnet 4.6 at $3 input and $15 output per million tokens, and the instance at $1.408 per hour (us-east-1 on-demand). The Sonnet figure checks out: 6,554 tokens at $3 plus 170 at $15 is $0.0222 per ticket. The decider figure depends on how busy the instance is: at full use one instance handles about 5,600 tickets an hour, or $250 per million, and Amazon keeps headroom for bursts, which gives about $500. Amazon notes that Lambda, which now runs for less time, is left out.

The three-way switch is the part to copy

Amazon did not flip a switch. The labelling Lambda reads a setting from AWS AppConfig on every message: BEDROCK (Sonnet only), DUAL (both models run, Sonnet's label is applied, the two answers are compared) or DECIDER (decider only). The engineers say running both on real traffic and judging every disagreement "told us more than any offline eval would have", and that going back is "a 45-second config change".

They also report a trap: sending the same ticket with JSON keys in a different order moved the model's output probabilities by up to 0.19, so they now send sorted keys. If you serve any small model behind an API, check for this before you compare runs.

Is the 98% saving real?

The arithmetic is, the framing is not neutral. Three things to know:

1. The baseline is old and expensive. Sonnet 4.6 costs $3 and $15 per million tokens. Anthropic's price list now has Sonnet 5.5 at $2 and $10 and Haiku 5.5 at $0.10 and $0.50, so any comparison against 4.6 includes the price cut Amazon could have taken by upgrading.
2. The old prompt was long. 6,554 input tokens and 170 output tokens per ticket come from a long system prompt and a JSON answer. The decider reads 2,009 tokens and writes one. Part of the saving is a better prompt, which an LLM can use too.
3. The top line is a bigger number than the bottom line. "98% cheaper" is a ratio. In dollars it is about $21,700 saved per million tickets, so "millions of dollars a year" implies on the order of a hundred million tickets a year (our inference, Amazon does not give the volume).

Why not Haiku 5.5 instead?

This is the question Hacker News asked first, and Amazon's post does not answer it. Using Anthropic's API list prices and Amazon's own token counts per ticket, our sums are:

Option	Input / output price per 1M	Cost per ticket	Cost per million tickets
Sonnet 4.6 (Amazon's baseline)	$3 / $15	$0.0222	$22,200
Sonnet 5.5	$2 / $10	$0.0148	$14,800
Haiku 5.5	$0.10 / $0.50	$0.00074	$740
Haiku 5.5 with 30% more tokens	$0.10 / $0.50	$0.00094	$940
Strands Decider E4B (Amazon, about 50% busy)	$1.408 per hour	about $0.0005	$500

The 30% row is there because Anthropic says its newer tokenizer (Claude 4.7 and later) produces about 30% more tokens for the same text. Haiku 5.5 at the same prompt costs 1.5 to 1.9 times the decider, and 24 to 30 times less than Sonnet 4.6. With Anthropic's 50% Batch API discount it would be lower still, though a ticket system that labels in real time may not be able to batch. Two caveats: Amazon pays Bedrock prices, which may differ from the direct API list, and we do not know whether Haiku 5.5 would be accurate enough on these tickets. One commenter reports Haiku is "really not good at decision tasks" (see below), which is an opinion, not a measurement.

How many tickets before a GPU endpoint pays off?

An always-on ml.g5.xlarge costs $1.408 × 730 hours = about $1,028 a month, whatever the traffic. Divide by the per-ticket price of the alternative and you get the monthly volume at which one instance breaks even:

Alternative	Cost per ticket	Break-even per instance
Sonnet 4.6, long prompt	$0.0222	about 46,000 tickets a month
Sonnet 5.5, long prompt	$0.0148	about 69,000
Haiku 5.5, long prompt (+30% tokens)	$0.00094	about 1.1 million
Haiku 5.5, long prompt	$0.00074	about 1.4 million

One instance can handle roughly 4 million tickets a month at full use (5,600 an hour), so Amazon's $500 per million assumes about half of that. Below the break-even you pay for an idle GPU. We assume a single instance, one region and no redundancy; a production endpoint usually needs at least two, which doubles the floor, and Amazon's figure leaves out the engineers who built and watch the pipeline. The decider also has its own limits: autoscaling, model loading from S3 at start-up and an isolated network are all things you run.

Did accuracy stay the same?

Amazon says it "performed just as well" and that E4B beat production Sonnet on a separate set of its tickets, but the post gives no accuracy number, no ticket count and no error breakdown. The same gap was spotted on Hacker News. What the repository does publish is general: the Gemma 4 E4B decider scores 187 of 231 on JevBench, a third-party benchmark for this class of model, and the repository says answers at 0.9 confidence or more are right about 95% of the time on short tasks it has not seen. On the same JevBench board, the repository notes, the top systems are reasoning models (the best at 0.996), and rebuilt deciders on larger models score 0.83 to 0.93. So a small decider is a long way from the best general models, and Amazon's result holds for a closed choice from a known catalog, which is exactly where its authors say it works. Your own tickets are the only real test.

What does Strands Decider need to run?

Per the repository: a GPU in bf16 (the E2B about 9 GB, E4B about 15 GB, 12B about 23 GB, 26B-A4B about 51 GB for the weights), Apple silicon with MLX or CPU. It is installed with pip install strands-decider, queried with strands-decider ask or an HTTP server, and a SageMaker deployment guide is in the repo. An open issue, #113, reports that the Qwen3.5 build fails to start on a T4 (ml.g4dn.xlarge) because of a BF16 kernel, after a long start-up; pick an A10G or newer. The Gemma 4 builds are on the main branch and not yet in the PyPI package, according to the README.

What users report

The Hacker News thread had 701 points and over 50 comments on 11 November. These are individual claims, none verified by us. Reddit blocks our tools.

• "Just use Haiku 5.5." speedboat199 wrote that Amazon "could have just switched to Haiku 5.5 to get same savings", with no GPU orchestration. wenc disagreed from experience: "Haiku is really not good at decision tasks". HarHarVeryFunny argued a 2B decider is cheaper to run than any hosted Haiku-sized model.
• Where is the accuracy? troyvit noted: "I didn't see mention of accuracy in the article".
• Old tools still work. ryankrage77 built a ComplementNB ticket classifier on about 30,000 examples in under a minute on a laptop CPU, about 85% accurate. lofties said "a BERT model can do this even faster, with even less memory". tjungblut replaced LLM pipelines with a Gemma 4 embedding and a classifier head, using the LLM to label the data first.
• Cold-start argument. wodenokoto pointed out you cannot train an SVM without domain data, but an LLM works on day one and changing the category list needs no retraining, so the LLM is the starting point and a small model the later optimisation.
• Calibration. deepsquirrelnet said a general LLM like Sonnet gives an answer with no calibration, which is the main weakness for tagging.

Our take

Thesis: the story is not "a small model beats Claude". It is "a closed classification job should not run on a frontier model, and the cheapest fix is often the smallest change, not the most exotic one". The decider result is believable, and the 98% is only as good as the baseline. Amazon compared a nine-month-old Sonnet with a long JSON prompt against a purpose-built model. The sums above show that downgrading to Haiku 5.5 on the same tokens already removes about 96% to 97% of the cost at $740 per million tickets, with no GPU to run.

What I would do if I label tickets, alerts or logs with an LLM today:

1. Price the job per thousand items, not per token. Take your real input and output tokens per item and use the table above. If the number is under about $1,000 per million items, a self-hosted model will not repay its engineering time.
2. Shrink the prompt first. Amazon cut tokens by 70% by rewriting 20 labels as one-line criteria and asking for a single choice. That works on any model and costs nothing.
3. Use Amazon's dual-run pattern, whatever the model. Run old and new in parallel, apply the old answer, log every disagreement, review them, then flip. It is the best idea in the post.
4. Move to your own GPU only when volume is high and steady. Over about a million items a month per instance, with a closed label set and a few thousand tokens of context, a decider is worth a pilot. Below that, or if labels change weekly or need explanations, stay on an API model.
5. Do not skip the accuracy number. Ask for it before you copy anyone's saving, including ours.

Who should care: teams paying for an LLM to sort tickets, alerts or logs at six-figure token bills. Who can ignore it: anyone below a few hundred thousand items a month. For the related pricing arithmetic see our posts on Claude Haiku 5.5 API prices and Claude Fable 5.2 against Opus 5.5, which use the same cost-per-task method.

What we did not test

• We did not install Strands Decider, run any model or call the Claude API. Every price is from a vendor page, every token count is Amazon's, and the Haiku, Sonnet 5.5 and break-even figures are our arithmetic.
• We did not verify Amazon's accuracy claim, and it is not published. We have no evidence on how Haiku 5.5 or any other model would label these tickets.
• We did not check Bedrock prices; we used Anthropic's direct API list prices for Haiku 5.5 and Sonnet 5.5, and Amazon's figures for Sonnet 4.6 and the instance.
• We did not check the SageMaker price of ml.g5.xlarge independently; $1.408 per hour is Amazon's number.
• Every Hacker News statement is the commenter's claim.

FAQ

What is Strands Decider?

An open-weight (Apache-2.0) "decision model" from the Strands Agents team at AWS. It picks one of several options, answers yes or no, or scores on a scale, and returns a probability and a confidence instead of writing text. The repository was created on 29 September 2026 and has builds on Qwen3.5 and Gemma 4.

What is the difference between a decision model and an LLM?

An LLM generates text one token at a time. A decision model reads the input once and scores the options you give it, so one question costs one forward pass and one output token. It cannot explain its answer, and the label set must be fixed in the request.

Is an LLM the right tool for classification, or should I use BERT or an SVM?

If you have labelled data and a stable label set, classic models and encoders such as BERT are faster and far cheaper, as several Hacker News commenters say. An LLM is useful when you start with no labelled data or the categories change often. Many teams label with an LLM first and then train a small model.

Can Strands Decider run on Amazon Bedrock?

Not as a Bedrock model, according to the post: Amazon ran it on a SageMaker endpoint. The repository says its server answers the paths that SageMaker and Bedrock AgentCore Runtime call, and has a deployment guide for SageMaker.

Which tasks can a decision model not replace an LLM for?

Amazon's own list: tasks that need explanation text, label catalogs that change weekly, and jobs where the useful context does not fit in a few thousand tokens. The model has a 4,096-token window.

How much would Claude Haiku 5.5 cost for the same job?

On Anthropic's list price of $0.10 and $0.50 per million tokens, Amazon's 6,554 input and 170 output tokens per ticket would cost about $0.00074, or $740 per million tickets, before any tokenizer difference. Accuracy on these tickets is not known.

Related guides

• Claude Haiku 5.5 API: price, effort levels, migration for the cheap end of the Claude family.
• Claude Fable 5.2 price against Opus 5.5 for the other end, with the same cost-per-task method.
• Gemma 4 and Qwen3.6 by RAM: what fits for the model sizes behind small-model plans.
• Anthropic S-1 explained: what it means for Claude API users for Anthropic's own cost and margin picture.

Sources

• Strands Agents blog: Amazon saves millions by replacing Sonnet 4.6 with Strands Decider (10 Nov 2026).
• GitHub: strands-labs/strands-decider, JevBench notes, issue 113.
• Claude docs: pricing.
• Hacker News: thread 50436640.
• Google autocomplete suggestions for "strands decider", "decision model llm" and "llm for classification vs" (11 Nov 2026).

Update log

• 11 November 2026: first version, from Amazon's post, the repository, Anthropic's price list and the Hacker News thread. We will update it if Amazon or the Strands team publishes accuracy figures or the ticket volume, or if prices change.

Last verified: November 2026 (11 November 2026). Prices and project details change; we will update this page and the log above.

Header image: Image: DevOpsCompass.