
Gemini 4 Neon is still quite new, and there's been a lot of hype around it being the best Google model in a good while. I normally have GPT-6.2 Sol review everything my coding agents write, so I wanted to see how Neon holds up in the same job. For one night I ran both side by side on a project where Claude Opus agents were working through an Azure and .NET backlog.
In the morning OpenCode told me the Neon reviews alone had cost $115.78 at list price. That got me curious whether they were worth it, so I had agents go through every review and check what happened to each finding.
The short version
- The models are very comparable. Neither hallucinated, only 5 of their 809 findings were wrong, and both caught real bugs the other missed.
- Sol had the edge on the bugs that matter. It caught 34 high-severity bugs that Neon missed and that then got fixed. Neon caught 17 that Sol missed.
- Neon is slow. A median review took 11.4 minutes against Sol's 5.8, and cost about twice as much at list price.
- I'll keep defaulting to Sol. It's faster, and my subscriptions already cover it.
The setup
Both models got the same review job. Once the implementing agent finished a slice of work, each reviewer got the brief, the diff and the evidence, and reported findings with a file, a line and a severity. They get a rough scale for what high, medium and low mean, but in the end each reviewer decides the severity itself. The implementer then fixed or answered them, and a re-review checked just those fixes. Neon ran through GitHub Copilot in OpenCode and Sol through Codex, both at xhigh reasoning.
Sol runs on my ChatGPT plan, and falls back to the same model on Foundry when the plan hits its usage limit. That happened a dozen times during the night, but I've counted both as Sol here.
I also added one thing to my flow during the night. Some changes kept going in loops: each round fixed the last findings, the reviewers found new ones in the same place, and sometimes the fixes pulled the change in the wrong direction. So now, if a change still gets findings after three rounds, Claude Opus first gives a second opinion on whether the approach itself is the problem. That's about my workflow rather than the models, so its findings aren't counted here.
Head to head
All findings
| Measure | Gemini 4 Neon | GPT-6.2 Sol |
|---|---|---|
| Raised | 359 | 354 |
| Fixed | 315 | 281 |
| Fixed, only this one found it | 187 | 157 |
High severity
| Measure | Gemini 4 Neon | GPT-6.2 Sol |
|---|---|---|
| Raised | 49 | 87 |
| Fixed | 47 | 69 |
| Fixed, only this one found it | 17 | 34 |
On the rounds both reviewed, the raw counts are pretty much even, so the difference is in what got found. Severity is each reviewer's own label, and Sol is quicker to call something high, so I looked at the high-severity findings that actually got fixed and that the other model didn't catch. Looking at those, Sol found twice as many, 34 against Neon's 17.
Some of Sol's best catches:
- An inherited environment variable could redirect the model endpoint, which would have sent the agent's bearer token and the whole chat to another host.
- A cancelled activation could roll back a newer version, because a pointer was written after a lock was released.
- A repeated optional token claim made a valid request fail with a 500. Neon had called the same code "behaviorally equivalent".
Neon caught things Sol missed too:
- With a new token-cache policy, the auth library rethrew "refresh needed" errors, so users were never sent back to sign in.
- A default-port check could be bypassed, which let a sandboxed preview reach a private internal API.
- A publish span used a kind the exporter drops, so the whole operation disappeared from telemetry.
What happened to the findings
| Reviewer | Fixed | Declined or deferred | No change | Wrong | Total |
|---|---|---|---|---|---|
| Gemini 4 Neon | 315 | 17 | 24 | 3 | 359 |
| GPT-6.2 Sol | 356 | 51 | 41 | 2 | 450 |
Sol has more findings in total because it also reviewed rounds Neon didn't. Neon's fix rate looks better, 88% against 79%, but most of that is small stuff, which the implementer fixes because it's cheap to do. More of Sol's findings got declined as out of scope.
The five wrong findings were misreadings, not made-up code. Neon flagged a crash in a state the code can't reach, an invisible character that was already escaped and a test step that was repeated on purpose. Sol said cancelling could rerun a handler that had already started, which the design allows, and reported a formatting mismatch that didn't exist.
What I care about more are the approvals. Neon approved 11 of its 43 reviews and Sol 8 of 68, and three times Neon approved a change that still had a high-severity bug in it:
- A smoke-test script had a line PowerShell couldn't parse, so it stopped before any credential check ran.
- A failed reply could give a removed user their access back.
- Trace context leaked into child operations.
Sol caught all three, and none of its own approvals let a high-severity bug through.
Cost and speed
| Neon | Sol | |
|---|---|---|
| Total for the night | $115.78 | $82.62 |
| Per finished review | $2.69 | $1.22 |
| Per fixed high-severity finding | $2.46 | $0.94 |
| Median time per review | 11.4 min | 5.8 min |
| Reviewer | Kind | Minutes | Cost |
|---|---|---|---|
| GPT-6.2 Sol | first review | 4.1 | 0.45 |
| GPT-6.2 Sol | re-review | 2.6 | 0.35 |
| GPT-6.2 Sol | first review | 7.2 | 1.15 |
| GPT-6.2 Sol | re-review | 4 | 0.77 |
| GPT-6.2 Sol | first review | 11 | 1.92 |
| GPT-6.2 Sol | first review | 18.9 | 3.14 |
| GPT-6.2 Sol | first review | 9.5 | 1.77 |
| Gemini 4 Neon | first review | 16.3 | 3.61 |
| Gemini 4 Neon | first review | 14.7 | 3.87 |
| GPT-6.2 Sol | re-review | 14 | 2.26 |
| Gemini 4 Neon | re-review | 15.2 | 3.10 |
| GPT-6.2 Sol | re-review | 13.5 | 1.90 |
| Gemini 4 Neon | re-review | 9.1 | 2.36 |
| GPT-6.2 Sol | first review | 8.7 | 1.06 |
| Gemini 4 Neon | first review | 10.5 | 1.51 |
| GPT-6.2 Sol | re-review | 12.1 | 1.92 |
| Gemini 4 Neon | re-review | 11.6 | 2.02 |
| GPT-6.2 Sol | re-review | 11.1 | 1.65 |
| Gemini 4 Neon | re-review | 15 | 3.38 |
| GPT-6.2 Sol | first review | 7.7 | 1.31 |
| Gemini 4 Neon | first review | 16.2 | 3.20 |
| GPT-6.2 Sol | re-review | 10.4 | 1.58 |
| Gemini 4 Neon | first review | 12.2 | 2.35 |
| GPT-6.2 Sol | first review | 10.4 | 1.63 |
| GPT-6.2 Sol | re-review | 3.7 | 0.32 |
| Gemini 4 Neon | re-review | 9.3 | 1.53 |
| GPT-6.2 Sol | first review | 9.9 | 0.94 |
| Gemini 4 Neon | first review | 16.1 | 3.65 |
| GPT-6.2 Sol | first review | 7 | 0.85 |
| Gemini 4 Neon | first review | 14.2 | 4.24 |
| GPT-6.2 Sol | re-review | 4.5 | 0.52 |
| Gemini 4 Neon | re-review | 8.9 | 2.43 |
| GPT-6.2 Sol | re-review | 3 | 0.40 |
| Gemini 4 Neon | re-review | 9.9 | 2.18 |
| GPT-6.2 Sol | first review | 5.1 | 0.56 |
| Gemini 4 Neon | first review | 16 | 3.85 |
| GPT-6.2 Sol | first review | 6.9 | 0.57 |
| Gemini 4 Neon | first review | 10.8 | 1.89 |
| GPT-6.2 Sol | re-review | 4.2 | 0.46 |
| GPT-6.2 Sol | re-review | 2.9 | 0.31 |
| Gemini 4 Neon | re-review | 5.7 | 1.08 |
| GPT-6.2 Sol | first review | 2.9 | 0.44 |
| Gemini 4 Neon | first review | 11.4 | 2.98 |
| GPT-6.2 Sol | first review | 15.3 | 3.52 |
| Gemini 4 Neon | first review | 11.9 | 2.66 |
| GPT-6.2 Sol | first review | 18.1 | 3.70 |
| Gemini 4 Neon | first review | 16.1 | 4.11 |
| GPT-6.2 Sol | first review | 18.6 | 2.73 |
| Gemini 4 Neon | first review | 13.7 | 3.20 |
| GPT-6.2 Sol | first review | 14.5 | 2.51 |
| Gemini 4 Neon | re-review | 12.5 | 3.01 |
| GPT-6.2 Sol | re-review | 12.7 | 2.28 |
| GPT-6.2 Sol | re-review | 8.4 | 1.72 |
| Gemini 4 Neon | first review | 12 | 3.12 |
| GPT-6.2 Sol | first review | 11.7 | 2.37 |
| GPT-6.2 Sol | re-review | 18.8 | 3.21 |
| GPT-6.2 Sol | re-review | 13.2 | 2.30 |
| Gemini 4 Neon | re-review | 7.5 | 1.70 |
| GPT-6.2 Sol | re-review | 11.2 | 2.52 |
| Gemini 4 Neon | first review | 10.8 | 2.42 |
| GPT-6.2 Sol | first review | 3.8 | 0.98 |
| GPT-6.2 Sol | re-review | 8.3 | 1.45 |
| Gemini 4 Neon | re-review | 8.9 | 1.87 |
| Gemini 4 Neon | first review | 12 | 3.02 |
| GPT-6.2 Sol | first review | 13.4 | 2.54 |
| GPT-6.2 Sol | re-review | 6.3 | 1.25 |
| GPT-6.2 Sol | first review | 2.4 | 0.26 |
| GPT-6.2 Sol | first review | 3.5 | 0.27 |
| Gemini 4 Neon | first review | 11.6 | 2.47 |
| GPT-6.2 Sol | re-review | 2.8 | 0.26 |
| GPT-6.2 Sol | re-review | 2.7 | 0.20 |
| GPT-6.2 Sol | first review | 3.9 | 0.38 |
| Gemini 4 Neon | first review | 12.6 | 3.34 |
| GPT-6.2 Sol | re-review | 4.3 | 0.50 |
| GPT-6.2 Sol | re-review | 2.8 | 0.25 |
| GPT-6.2 Sol | first review | 2.9 | 0.30 |
| Gemini 4 Neon | first review | 8.1 | 1.62 |
| GPT-6.2 Sol | first review | 5.1 | 0.79 |
| Gemini 4 Neon | first review | 10.6 | 3.21 |
| GPT-6.2 Sol | re-review | 3 | 0.31 |
| Gemini 4 Neon | re-review | 4.5 | 1.04 |
| GPT-6.2 Sol | re-review | 5 | 0.64 |
| Gemini 4 Neon | re-review | 9.5 | 2.49 |
| GPT-6.2 Sol | re-review | 4.7 | 0.52 |
| Gemini 4 Neon | re-review | 8.7 | 2.09 |
| GPT-6.2 Sol | re-review | 2.5 | 0.28 |
| GPT-6.2 Sol | re-review | 2.7 | 0.30 |
| Gemini 4 Neon | first review | 9 | 2.50 |
| GPT-6.2 Sol | first review | 3.6 | 0.40 |
| Gemini 4 Neon | first review | 15.7 | 3.23 |
| GPT-6.2 Sol | first review | 8.2 | 0.90 |
| Gemini 4 Neon | first review | 12.7 | 4.32 |
| GPT-6.2 Sol | re-review | 2.7 | 0.29 |
| Gemini 4 Neon | re-review | 11.1 | 3.48 |
| GPT-6.2 Sol | re-review | 11.9 | 2.29 |
| GPT-6.2 Sol | first review | 9.5 | 1.51 |
| Gemini 4 Neon | first review | 10.6 | 2.50 |
| GPT-6.2 Sol | first review | 12.2 | 2.98 |
| GPT-6.2 Sol | re-review | 5.3 | 0.92 |
| GPT-6.2 Sol | re-review | 4.8 | 0.97 |
| Gemini 4 Neon | re-review | 7.8 | 1.69 |
| GPT-6.2 Sol | re-review | 4.9 | 0.93 |
| GPT-6.2 Sol | re-review | 19.5 | 2.03 |
| Gemini 4 Neon | re-review | 11.8 | 2.47 |
| GPT-6.2 Sol | re-review | 2.9 | 0.43 |
| Gemini 4 Neon | re-review | 8.6 | 2.18 |
| GPT-6.2 Sol | first review | 3.4 | 0.42 |
| Gemini 4 Neon | first review | 8.4 | 2.81 |
| GPT-6.2 Sol | re-review | 2.6 | 0.23 |
On speed and cost the gap is much clearer. A median Neon review took 11.4 minutes against Sol's 5.8, and cost about twice as much. Even the Sol reviews that fell back to Foundry, which were the bigger ones, came out cheaper than Neon for about the same time.
The wait is what I'd actually feel in regular use. Even with the agents running in the background, waiting twice as long for every review adds up quickly, especially in my flow where every small slice gets its own review and re-review.
These are list prices, not what I actually ended up paying. Copilot pricing depends on your plan, and the ChatGPT plan is a flat fee anyway.
So which one?
Neon is a good reviewer, and a lot closer to Sol than I expected. But it's slower, and Sol still had the edge on the bugs that matter. So I'll keep Sol as my default, also because my subscriptions already cover it and I don't have one for Neon.
Where I can't lean on subscriptions, I might next try Neon as the main implementing model instead, since it seems capable enough. I'm not convinced it would be the cheapest option though. Google models have had worse cache rates than the competition in my experience, and an implementer rereads its context constantly.
If you run a test like this yourself, don't stop at total findings or fix rates. The raw counts made these two look even and the fix rate made Neon look better. The serious bugs that actually got fixed told a different story.
Comments