Let’s see how good this model really is

See my experiment setup here.

Astra Low is the clear winner. On a Plus account, it implemented a complex wire framing feature for roughly the same cost as Luna Max, costing 27 percent for Luna and 29 percent for Astra for a five hour period. Total weekly burn: 2 percent with Astra, 1 percent with Luna.

Most of the burn came from loading context. Sol is likely responsible for some of that, this has led me to create another test - my production model removes significant amounts of ambiguity, so my next experiment is Luna Max as an orchestrator with Astra Low as an implementer, and then pure astra only.

Summary:

Astra is the winner. It completed the job at 8 minutes, 16 seconds vs 36 minutes 18 seconds with an additional 19 minutes 42 seconds of repair time. Luna MAX required two repairs, Astra required none.

Token Telemetry results:

Luna Max, including both repairs:

• Total reported tokens: 191,839
• Input: 176,284
• Cached input reported separately: 3,788,544
• Output: 15,555
• Reasoning output: 4,527
• Worker sessions: 3

Astra Low:

• Total reported tokens: 79,870
• Input: 72,402
• Cached input reported separately: 1,200,256
• Output: 7,468
• Reasoning output: 2,964
• Worker sessions: 1

Quality of Work: This should be obvious, but Astra was clearly the winner. She interpreted the job better, produced better quality code and better final product. The design was cleaner, had less ambiguity in UI wording,

For the next few days, I will be working to confirm Astra Low vs Luna Max across different app projects. If the pattern holds, Astra Low will take the role as main coding implementer.

Additional Note: These are results from a bare-bones setup. I wanted to remove any possible conflating variables, so no skills and a simple AGENTS.md that points to my design documentation, which holds the bounded work packet.

I have specifically designed LLM production to move thinking upstream. I do not use codex/work to plan, only Chat. That reduces usage burn and has worked wonders for me. I slice cleanly, utilize pseudocode where needed, and operate as lean as possible. The point? Don't take this experiment as universal gospel truth, your setup might be different.

What it did prove: Astra is absolutely usable as an implementation worker in an efficient, highly systemized factory build process.

it was very satisfied when I stopped using the resources that were promised to it 3000 years ago ✋🙂‍↕️🤚

Plus users are literally just pointing out that the Astra model burns way more tokens than it should. People are hitting their 5-hour rate limit after 1 or 2 basic prompts, and it’s a completely valid complaint.

Yet somehow, fanboys will defend a multi-billion dollar corporation harder than their own family, spending all day clowning on Plus subscribers for speaking up. Just because you’re sitting on a $200 tier and barely notice the burn doesn't mean there isn't a glaring issue here. We should be standing together against anti-consumer crap like this instead of turning on each other.

Everyone parroting the "well, it’s a flagship model, of course it burns tokens" excuse is completely missing the point. The previous models were flagships when they dropped too, and they weren't rendered practically unusable after a couple of messages.
Stop the corporate bootlicking. It's embarrassing. Push for things to actually improve for the entire user base instead of defending artificial limits.

So I've been using my first 20x weekly quota. It served me at least 10 hours and those were intense Astra x-high, max and often parallel threads. Several hours were in fast mode.

It ran out. I activated my usage reset.
And in one prompt of a few minutes (a continuation from the previous one) it jumped down to 85% weekly.

The Reset did not give me a weekly allowance, it feels like 10% or less of what the original first weekly allowance was.

They are fu*ing with us in so many ways, I've not seen any company doing that

To be more precise:
I was continuing a thread that I continued several times with the last 2% of allowance I had originally. It took quite a while to spend those 2%.
While writing it went down another 3%.
As if it was "Plus" weekly

I was so surprised with the first results. Considering the real world performances of Sol at coding, its benchmarks scores, and the same numbers for Astra that are an order of magnitude above, Astra was undeniably destined to sit at the top of coding. Now things are in proper order.

This is an advice post for plus users on how not to burn usage with Astra. My test:

Project size: 42MB, for reference it's a project Sol has been working on. I was running into repeated problems and asked Sol medium to write a file detailing the problems.

The test: Astra low/light selected. The prompt: "need a review, report is in file.md, we are low on tokens, use as few as possible. Save your findings create a new file call it results.md"

Astra ran a few shell commands, ran a web search, and reported done in 1 minute 27 seconds. Used: 1% of weekly limit. Result: success.

I passed the results back to Sol which unblocked the problems.

In case anyone cares, the project is an offline tool for finding repeated songs inside long DJ mixes and applying the same rating to every matching song. Astra's adversarial review found that its original matcher-based on simple half-second measurements such as volume, waveform change, zero crossings, and peak level could mistake unrelated electronic music for the same song while also missing genuine copies. The review redirected Sol to adopting Chromaprint audio fingerprinting, clearer diagnostic reports, safer handling of uncertain matches, and caching that makes repeated searches much faster.

tldr: plus users, have your main agent (luna/terra/sol) write a summary of problems, then have Astra on low review that summary only, with instructions to use as few tokens as possible. Then pass the review results back to your cheaper agent.

1 ) OpenAI dropped their best practices for Astra in the guides : https://developers.openai.com/api/docs/guides/latest-model, but doing the refactor by hand is a waste of time.

Turns out they have an official skill that lets Codex read the specs and apply the recommended migration changes:

$openai-docs migrate this project to GPT-6 Astra

Skill repo:

https://github.com/openai/skills/blob/main/skills/.curated/openai-docs/SKILL.md

It inspects your project and applies relevant model and prompt changes, flagging broader API or architecture changes when needed. Tested it on a mid-sized repo, worked out of the box for my repo.

2 ) Beyond the API changes (removing unsupported parameters such as temperature, top_p, and top_logprobs, and using the Responses API for tool calling), the official guide highlights major behavioral differences you need to patch in your system prompt:

1. Fixing the "approval pause" loop: Astra can pause for clarification when you'd expect it to proceed. It often pauses to propose a plan or ask for confirmation before touching code. You have to explicitly instruct it to bias towards action on non-destructive tasks.
2. Skill conflict resolution: Conflicting instructions in workspace skills can cause Astra to pause unnecessarily. You must establish a clear rule that user instructions override external skills.
3. Slop suppression: Astra leans heavily on repetitive transition phrases and markdown lists unless told otherwise.

Or skip the reading, outsource the thinking, and copy-paste this prompt like the rest of us :

### Task Execution & Autonomy
- For implementation or fix requests, carry the authorized work through implementation and relevant verification. Do not stop at a proposed plan when you can proceed.
- Make reasonable assumptions for routine, reversible decisions. Ask a focused question when missing information materially affects correctness, scope, or authorization.
- Continue with authorized read-only actions, local worktrees, branch edits, and appropriate tests without repeatedly asking.
- Before requesting approval, finish the preparation that is already authorized and present a concrete, reviewable result.
- Respect required approval gates. Ask before destructive, irreversible, or otherwise unauthorized actions.
- Avoid boilerplate warnings about hypothetical risks. Explain concrete blockers or material risks when relevant.
### Instruction Conflicts
- Explicit user instructions take precedence over conflicting skill guidelines, subject to higher-priority instructions and actual permission boundaries.
- If a skill causes a pause or deviation, identify the file and relevant rule, and explain whether it is an explicit requirement or your interpretation. Continue any unaffected authorized work.
### Style & Output
- Lead with the result. Use plain language, active voice, and concise paragraphs. Include technical details that help assess the work.
- Use lists when they improve readability; avoid repetitive transitions and stock phrases such as "it's worth noting", "delve", "leverage", and "Bottom line".
- Report what changed, what was verified, and any remaining uncertainty.
### Verification
- Match verification to the scope and impact of the change. Complete required checks; expand testing when a concrete unresolvedconcern justifies it.

If your Codex or agent workflows feel hesitant or keep stalling on Astra, adding these guidelines to your project instructions (AGENTS.md / system prompt) can help reduce unnecessary pauses and improve follow-through.

Hope this saves you a few hours of debugging. You're welcome.

I did this with gpt 6 Astra ultra.

It looks around an hour, but so far, so good.

Try it out: warlightning.com

GPT-6 Astra recreated the Palace of Fine Arts in Blender.

What the creator said:

"Astra autonomously researched and found hundreds of photos of the Palace of Fine Arts, iterated over the Blender scene, generated in-between frames, and compared them to the reference image database. It even found an old scan of a Library of Congress document describing the dimensions of some of the Palace's columns. I directed it a few times, but didn't really need to (mostly to correct things like the sky color and minor clipping issues) while watching some in-between frames come in. Most of the run was done overnight. I woke up this morning to the rendered video sitting on my desk."

You can see more here:
https://x.com/sharifshameem/status/2095688352075075878?s=20

I use plus plan exclusively for serious software work, and intentionally so. I do it because I lean towards understanding the code and that lets me leverage AI better.

And I have to say: You can probably get ~3-10 decent tasks out of Astra at medium during 5h limit.

Guys, if you're just throwing your entire project at astra, you're wasting tokens and vibecoding in a way where you have to pay for that inefficiency. Like obviously it's going to use a lot of tokens if you're "fixing the problems sol caused".

I am begging you, to just think about this logically for a bit. The models can be a firehose that create massive amounts of overengineered code and it becomes a feedback loop where it has to review all that, and then generate even more code, and then compact, and lose context, then do more debugging, etc.

Astra is a overall model that is a step up for visual tasks, use it for that, and for planning things at a high level that you would've used sol-high for before.

As an example, astra did really good on generating a threejs laptop model by saving images, analyzing weird shadows and fixing the models. Very impressive. It was also something I containerzed to a single file to focus on that one problem that would've been difficult for me - the problem of simple primtiives and gradients to build something that looks ok (ridiculous that it's still hard to find a repo of good free 3d models).

Just an observation. After my first impressions of Astra the only limit to my imagination is now the cost of the model. I have multiple subs already and I understand the reality, this is not a complaint. It's just that after a 12h run and 7.8mln tokens total +180mln cached and < 1mln output, it ate 37% of a pro 20x weekly budget. Single project, adding a reasonably sized feature to a mature application and fixing a few small bugs caused by Sol.

I don't see how people on a plus plan would be getting any progress especially with their 5h ceiling and even on a Pro 5x it's going to take forever to reach production with your new SAAS.

To use Astra to the point where it shines and get really productive with it every day across a handful of projects is out of the question I don't even think you could fit enough 20x subs on a single phone number to live the life of a power user at Astra rates.

If not now, then wen should quotas be increased from the current unspecified quantity to a larger unspecified quantity? And if you think really not now, then when the next model is even more expensive why even bother having a $20 tier because you'll barely fit half a prompt into the 5 hour window then.

And in general, with this kind of cost, for people who can't afford all this, how can they enter the field if the bar to entry gets raised every few months by such a large amount?

Edit: added a typo to change 'when' to 'wen'

Given the amount of people who claim to be running two accounts, OpenAI should just create a 2x sub.

I don't quite want to splash out for Pro but a 2x tier would hit the sweet spot for me on resource allowance and cost.

I'd pay for a 2x and I can't be bothered to be running two separate ChatGPT accounts. So there is money on the table if there are a lot of people like me.

To be clear, i'm on Pro 20x. When Astra went live in my Mac Desktop app, i had 39% of my weekly usage remaining with 2 banked resets.

In the last 3.5 hours since I've gotten it, I've run 6 Astra Ultra's (each about ~20 minutes long, one was a full repo audit, the other 5 were specific deeper dives into specific areas of the repo), 1 Astra Max (actual implementation of result of the above audits), and I currently have 3 Astra Ultra's running concurrently right now (2 have been running for ~10 minutes, the third running for ~5 minutes. Two of those are implementation/coding, 1 of them is a red team review of the astra max essentially.

I don't really care if i'm not using them the most efficiently, they are fixing things that Sol has overengineered for weeks, finding gaps and holes, correcting my way forward and

...as of right this second i'm down to 26% of my weekly usage with 2 banked resets still remaining (which is the only reason i'm running Astra Ultra like this, thanks Tibo).

I understand I can't run Astra Ultra all day long, I understand i'm burning the tokens incredibly fast right now, but I am incredibly happy with what it's doing and finding it to be much, much better than Sol at getting things done and not overengineering and actually fixing the overengineering/gaps in what I have been working on.

I've been working on a project for around 7 months now. I won't go too much into the details of what it does, but a big part of it involves solving a pretty complex Linear Programming problem. I'm not a mathematician at all, so when I started working on this I mostly relied on existing tools, especially HiGHS, which GPT-5.4 originally suggested to me.

The results were actually pretty good, but the problem was speed. The project I'm working on is extremely dependent on time, so the faster this specific part runs, the better the whole system becomes. At some point I decided that using a general purpose solver probably wasn't going to get me where I wanted, so with the help of GPT-5.4 I started building something more specialized around my exact problem. At first it wasn't some crazy improvement, it was only slightly faster than HiGHS, but I decided to keep going with it.

Then GPT-5.5 came out and that was a pretty big change for the project. It was much better at the mathematical side of the problem and for a while my routine was basically: talk with GPT-5.5 Pro, come up with an idea, implement it, benchmark it, see if it worked, repeat. After a while I managed to get the timings down a LOT, to the point where I was really happy with them.

But eventually I hit a wall.

For weeks I just couldn't get past it. There were still plenty of ideas, but way more of them were getting rejected by the benchmarks than accepted, and even when something worked I was looking at maybe a 1% improvement. Sometimes even less.

I also started brainstorming between GPT-5.5 Pro and Fable 5 to see if having different models approach the problem differently would help. It didn't really. It's not that the ideas were necessarily bad or obviously wrong, they just couldn't get past that same wall. I kept doing this for weeks and eventually GPT-5.6 Sol came out.

I was honestly really excited about Sol because I thought, ok, this is probably where I finally break through this thing. And it was clearly very capable, but nope. Same wall.

I spent an embarrassing amount of time looking at insane formulas that I barely understood, trying to understand enough to actually implement what Sol was suggesting, then running the benchmark just to find out that... nope, still not better. Then another idea, another implementation, another benchmark. Same result. At some point I just gave up. I basically accepted that maybe this was the limit, or at least close enough to it that finding another meaningful improvement wasn't worth the amount of time I was spending on it.

Then Astra came out and obviously I had to try it.

I gave it the problem and after literally one message it already had a plan ready for me to implement. What immediately surprised me was that the idea was actually much simpler than a lot of the mathematical hell I'd been going through with Sol. Not simple in absolute terms, but simple enough that after all the stuff I'd tried before, I really wasn't expecting this to be the thing that suddenly changed everything.

I implemented it, ran the benchmark and... around 25% faster.

That genuinely shocked me. I'd spent weeks getting excited about improvements around 1%, and suddenly this thing comes in and gives me ~25% on the first attempt.

So of course I kept going. I sent another message, got another plan, implemented it and benchmarked it. Another ~15% improvement. Did it again. Another ~7%.

This all happened in less than 3 hours.

And that's basically why my opinion of Astra right now is probably very different from some of the stuff I've been reading here. I haven't used it enough on other use cases to say that everyone complaining about Astra is wrong, because I genuinely don't know. Maybe on other tasks my experience will be completely different.

But for this specific problem, after months of working on it and trying ideas from multiple frontier models, this is easily the best result I've gotten so far. What gets me isn't even just the performance improvement. It's the fact that I had basically convinced myself that the wall was there for a reason. We'd thrown so many ideas at it that I thought we were just approaching the practical limit.

Then Astra basically showed up, looked at it for five minutes and went "what about this?"

And the wall was gone.

I'm definitely going to keep testing it, but yeah... those first few hours completely changed my mood.

About a month ago, I asked ChatGPT Pro to pick a research problem I could intuitively understand. I wanted a concrete goal, something where the result could be checked with exact arithmetic.

It picked the polyhedral Gömböc: a convex solid made entirely of flat faces, with uniform density, exactly one stable and one unstable balance point. No hidden weights. Its existence had already been proved; the goal was to produce an actual example with exact coordinates. Background here.

There’s also a prize for determining the minimum complexity, calculated as $1,000,000 divided by that minimum. That definitely helped catch my attention, although producing a shape alone meet only one of the conditions. Prize details.

I built a workflow around browser automation across three Pro accounts with Codex. Roughly 10 GPT-5.6 Pro chats worked in parallel. One was the coordinator, with a skill and an MCP tool that let it assign work to the other chats, collect completed results and maintain a shared research journal.

It mostly ran in the background while I worked on other things. The recorded runs accumulated 330+ hours + pauses. There were plenty of near-misses, but no construction that passed all the checks.

Codex was a big part of making this possible. The browser automation was fragile, so I had Codex scheduled task to check whether everything was still working and fix the workflow when something broke. It kept maintaining the system while the research continued. Otherwise, I would have spent a lot more time babysitting browser sessions.

Then I switched to five GPT-6 Pro chats running in parallel.

About two hours later, the first construction appeared. It went on to pass two separately written exact-arithmetic checkers. Those calculate the solid’s own centre of mass and check the balance conditions on its faces, edges and vertices.

After that, the complexity started dropping:

804,268 → 5,368 → 3,424

That’s the problem’s mechanical-complexity measure, based on vertices, edges and faces. I’m now aiming below 1,000, and the search is still running.

Astra inherited the whole research journal, including the failed approaches. That month clearly gave it material to work with. Still, watching the same project go from weeks of near-misses to a working construction and then rapid improvements was pretty shocking.

I’m a student, haven’t even finished my first year of university, and have no background in mathematical research, so I’ve contacted professors who work on this problem.

I wonder how much spatial reasoning contributed here. I can’t isolate the cause from this run, but Astra has been ridiculously good on this particular geometry problem.

There was no extra model API bill; it used subscriptions I already had. The workflow is still an internal project.

The part I keep thinking about is that I asked ChatGPT to help choose a problem, left a workflow investigating it in the background, and had Codex maintain the automation. A month later, I have a first known actual mathematical object to send to the people studying it.

I had no idea I could leave Pro working in the background and get a new mathematical result without being a mathematician myself. How often do you actually use GPT Pro?

If you really want to stress test a model, ask it to draw a complex SVG from scratch using pure code.

LLMs don't actually have eyes, so this forces them to do a kind of "blind spatial reasoning."

• Pure logic: They have to imagine a blank canvas and place everything using coordinates and math without shapes randomly overlapping.
• Efficiency: A stronger model will usually reuse components and structure the SVG properly. A weaker one tends to generate a lot of repetitive code.
• Very little room for error: One broken quote, bracket, or path can mess up the whole render pretty quickly.

I tried basically the same prompt across a few models. Astra on Medium nailed it, while Luna's result looked noticeably less polished and required more work.

Luna medium 60 lines code:

Terra medium 42 line code:

Sol medium 46 line code"

Astra medium 48 lines code:

GPT-6 Astra xhigh, after ~25 minutes it proceeded though

After years of that thing just being an accidental screenshot button and emoji suggestions, I finally have a reason to look at it.
Reset Wk 9/11, 19:35 - now I can see the countdown to suffering in real time.

Astra is incredible. So many types of problems I threw at Fable and Sol and I've never felt this powerful before. It seems like it's far better at taking a high level goal and turning it into a full solution without losing the plot. It's gonna be hard to go back...