Haiku 5.5 vs GPT-6 Luna: The Benchmarks Lied

On the benchmarks, Anthropic's new Haiku 5.5 beats OpenAI's GPT-6 Luna on nearly everything. In actual head-to-head testing, Luna finished every project, ran nearly 7x cheaper, and was 37% faster — while Haiku choked twice. I ran both small models through the same prompts, the same harness, and the same real builds. The benchmark story and the real-world story are not the same story.

Anthropic released Haiku 5.5 as their smallest, cheapest, and fastest model yet, and it instantly had people wondering if OpenAI's entire small-model lineup was dead. Up until this launch, Anthropic had nothing to put against GPT-6 Luna — they had big, powerful models, but nothing small and fast for cheap, repetitive, high-volume work. Now they do, and it sits at the exact same per-token price as Luna. So is OpenAI screwed? Not even close. Here's what the tests actually showed.

What Do the Haiku 5.5 Benchmarks Actually Say?

Start with Anthropic's own numbers, because they look brutal. The official launch benchmarks showed Haiku 5.5 crushing Luna across the board — terminal bench, computer use, hard coding problems, reading charts, and real-world tasks. In every category Anthropic showed, Haiku won.

The independent numbers backed that up, mostly. On Artificial Analysis, Haiku kept winning, and on the AA briefcase benchmark — which measures real-world tasks — Haiku actually tied GPT-6 Astra, OpenAI's flagship model. That's a small, cheap model matching a frontier model on practical work. On hallucination, Luna was worse too: GPT-6 Luna hallucinates roughly twice as much as Haiku.

But the clean sweep wasn't actually clean. On automation bench and knowledge accuracy, Luna beat Haiku. On long-context reasoning and scientific coding, the two basically tied. So even before cost enters the picture, the "Haiku wins everything" narrative already had cracks.

Why Does Haiku 5.5 Cost More Than Its Price Tag Suggests?

Here's the thing nobody puts on a benchmark chart: a score only matters next to what you paid to get it. And on cost, the gap between Haiku 5.5 and Luna gets ugly fast — even though per token, the two models are priced the same.

Look at the Artificial Analysis Intelligence Index. At max effort, Haiku scored 43 and Luna scored 38. A real win for Haiku. But that 43 cost about 21 cents per task, while Luna's 38 cost about 7 cents. That's 3x the cost for a five-point jump. You're paying triple to go from 38 to 43.

Most of that comes down to a price cliff baked into Haiku. What Haiku costs you depends on how many tokens it burns:

• Up to 100,000 tokens — 10 cents per million input, 50 cents per million output.
• Above 100,000 tokens — the rate jumps to 5x the cost.

Luna has a cliff too, but it's gentler in both directions. Luna only doubles in price instead of going 5x, and it doesn't hit that step until 272K tokens — versus Haiku's 100K. That difference completely changes the math on when you'd reach for one over the other.

And it bites in practice. On one max-effort task that came out to 21 cents, the model took 50 steps — and 31 of those 50 steps got billed at the 5x rate. On longer, agentic, multi-step jobs, that's exactly where Haiku quietly drains your budget.

How Did Haiku 5.5 and Luna Do on Real Builds?

Benchmarks are proxies. I wanted real outputs. So I ran both models inside Open Code, using the exact same harness and the exact same prompts, across a stack of tasks: front-end design, the infamous pelican-on-a-bicycle SVG, a needle-in-a-haystack long-context test, a 3JS game, and a run of HTML animation builds.

I gave both models no skills for any of it — raw reasoning and output only — with one exception: image generation, which I wired up through the Higgsfield CLI so both models could actually produce the images their websites needed.

Front-End Design: Luna Took It

On low effort, Haiku's hotel landing page looked pretty good for the price — nice hero background, some hover motion — but the "check availability" button did nothing when clicked, and there was a lot of awkward blank space. It cost 3.6 cents, took 6 minutes 31 seconds, and ran 13 steps. Luna's version was similar in vibe with a bit less going on, but its check-availability button actually worked — and it came in at 1 cent in 5 minutes 30 seconds. Roughly a third of the cost, a little faster, functional. Luna wins.

On max effort, the gap got absurd. Haiku took almost 27 minutes, cost $1.25, and actually failed the first time — I had to prompt it to try again. Luna did the same prompt in 9 minutes 19 seconds for 4.6 cents. Haiku's max output did look better in spots — a cleaner footer, more polish — but it leaned into that rounded-pill, glowing, "AI slop" aesthetic. Was the result 100x better to justify ~100x the cost? No. Luna again.

The 3JS Game: Haiku Flat-Out Failed

I asked both for a simple 3JS snowboarder game — dodge the poles, move left and right, tally a score. Simple, but long-running enough to stress the model. Both ran at extra-high effort.

Haiku completely failed. It kept trying to produce an output larger than its ceiling. Haiku has an output limit around 128,000 tokens, and when it tries to blow past that, it seizes up. It did this over and over, never smart enough to break the work into pieces. After 11 minutes 15 seconds, multiple runs, and 6.5 cents, I got nothing. Luna ran 13 minutes, took 26 steps, cost under 5 cents, and produced the game.

This is the pattern to watch: at high and max effort, Haiku starts behaving like Sonnet 5 — burning absurd amounts of reasoning tokens to pick a path, to the point where it literally chokes itself to death on its own output.

Long-Context Retrieval: Haiku's One Clear Win

Now the flip side. I ran the 8-needle test — hand each model a giant document stuffed with eight poems about snowboarding, then ask something like "what happens in the third line of the second snowboarding poem?" It's a standard long-context retrieval benchmark: find the needles, pick the right one, answer precisely.

Here Haiku crushed Luna. On medium effort, Haiku hit 96% accuracy, cost about $5, with a median time of 13 seconds. Across 32–64K, 128–256K, and 256–512K token ranges it ran 99%, dropping only slightly in the largest batch (9 out of 10). Luna managed 29% accuracy — slower, cheaper, and frankly unusable. Even in the small 32K–64K window, Luna only got 3 out of 10 right. If accuracy over long context matters, this is the one place Haiku is clearly the better tool.

The Rapid-Fire Builds

The rest of the tests kept favoring Luna on speed and cost:

• Lava lamp in a single HTML file (medium) — Haiku: 14 minutes, ~6 cents, and it looked rough. Luna: 2 minutes 34 seconds, under 1 cent. Twelve minutes faster for less than a penny.
• Animated night-train scene (parallax mountains, lit windows, star streams) — Haiku: 5 minutes 11 seconds, under 5 cents, decent. Luna: 2 minutes 50 seconds, just over 1 cent — arguably better-looking.
• Pelican on a bicycle in SVG — a rare miss for Luna, which produced an abomination and took longer than Haiku. Haiku win.
• Minesweeper — both fine; Haiku had the nicer UI and was noticeably faster at similar cost. Slight Haiku edge.
• Voxel sky island animation — on medium, Haiku looked good (4:15, 3 cents); Luna looked great at a third of the cost and a minute faster. On max, Haiku choked again on its own output and produced nothing, while Luna delivered.

Which Small Model Should You Actually Use?

Add it all up and the totals are lopsided. Luna was nearly 7x cheaper, 37% faster, and finished every single project. Haiku hit its output wall twice and produced nothing on those runs. The one decisive Haiku victory was long-context retrieval.

So this isn't "Haiku is bad." Haiku 5.5 is a genuinely strong model — it's just deceptively expensive at high effort levels, where the price cliff and the token-hungry reasoning stack up against you. Here's how I'd split them:

• Reach for Luna when you want quick builds, high-volume work, or you're seriously cost-conscious. For simple, fast, cheap jobs, Luna is probably the answer.
• Reach for Haiku when you need long-context recall and accuracy genuinely matters — but keep a tight leash on effort levels, because max and extra-high are where it bleeds money and sometimes fails outright.

And there's a catch even on Haiku's home turf: once accuracy on long context is the priority, the real question often becomes Haiku vs. Sonnet, not Haiku vs. Luna. If the task is simple and you want it fast and cheap, I keep landing back on Luna.

The benchmarks made this look obvious the day Haiku launched. It isn't. This is a real win for OpenAI — Luna has a cheap, fast sweet spot that Haiku can't touch on a lot of everyday tasks. But it's genuinely case-by-case, and now you've got the real numbers to decide with instead of a launch-day chart.

Frequently Asked Questions

Is Haiku 5.5 better than GPT-6 Luna?

On Anthropic's benchmarks, yes — Haiku 5.5 wins nearly across the board. In real-world head-to-head testing, no: Luna was nearly 7x cheaper, 37% faster, and completed every project, while Haiku failed twice by exceeding its output limit. The exception is long-context retrieval, where Haiku hit 96% accuracy versus Luna's 29%.

Why is Haiku 5.5 more expensive than Luna if they cost the same per token?

Because of the price cliff. Haiku's rate jumps 5x once a request passes 100,000 tokens, while Luna only doubles and not until 272K tokens. On agentic, multi-step tasks, a large share of steps end up billed at Haiku's 5x rate — in one test, 31 of 50 steps. Luna ends up far cheaper in practice despite identical base pricing.

What is Haiku 5.5's output limit?

Haiku 5.5 appears to cap output around 128,000 tokens. When a task pushes past that — like generating a full 3JS game in one shot — the model seizes up instead of breaking the work into smaller pieces, and the run fails. This showed up repeatedly at extra-high and max effort.

When should I use max effort with Haiku 5.5?

Rarely. In testing, higher effort levels drove huge cost increases for small quality gains, and max/extra-high is exactly where Haiku ran into its output wall and failed. For most tasks you set a small, fast model on, high effort or lower is plenty — and medium was the sweet spot in several tests.

How did you give the models image generation?

Neither model generates images from the raw API — even Luna calls out to a separate OpenAI image tool. I wired both up through the Higgsfield CLI, using GPT Images 2.5 with the flare option (not Sunburst) for cheap, solid outputs. Higgsfield now offers pay-as-you-go API access, so you don't need a monthly subscription to generate images piecemeal.

---

If you want to go deeper on picking the right model for the job, join the free Chase AI community for templates, prompts, and live breakdowns. And if you're serious about building with AI, check out the paid community, Chase AI+, for hands-on guidance on how to make money with AI.