Maestro Briefby Maestro Mojo

How Much Does It Cost to Build a Web Portal With Codex?

2026-08-10

CodexAI codingmodel costsbenchmarksdeveloper toolsresearch

Maestro's take

Use Terra first for an ordinary, well-scoped portal.

In our small test, Terra finished as fast as Sol. Both passed every independent acceptance check. Terra's estimated API-equivalent token cost was less than half of Sol's.

Luna was dramatically cheaper. It was also slower. It used far more tokens. And one run missed a seed-data requirement and left debug data behind.

The lesson is not that Terra always wins.

The lesson is simpler: the cheapest token is not always the cheapest finished job.

An origami developer oversees three folded-paper coding paths that converge on the same customer portal.

The experiment followed three coding paths from the same brief to an independently checked portal.

TL;DR

We asked Codex Sol, Terra, and Luna to build the same customer-request portal.

Each model received the same specification. Each started in a clean repository. Each used its default reasoning setting. We ran two scored trials per model.

These were subscription-backed Codex runs. So the incremental cash charge was $0. The dollar figures show what the measured tokens would cost at the public API rates listed at 11:00 a.m. PDT on August 10, 2026.

They are estimates. They are not invoices.

Here are the two scored runs separately. This matters because two runs are not enough to smooth out normal variation.

Each line reports acceptance, API-equivalent cost range, then elapsed range.

Two bar charts compare the models. Estimated observed-cache cost: Sol 61 cents, Terra 27 cents, Luna 5 cents. Average elapsed time: Sol 266 seconds, Terra 260 seconds, Luna 363 seconds.

Two scored runs per model on one Django task. Prices use the public rate card visible August 10, 2026. Estimates are not invoices.

Why Maestro users care

Maestro lets developers send real work to different coding agents.

That makes model choice a practical decision, not a leaderboard argument. A cheaper model can save money. A stronger model can save cleanup. The useful question is whether the finished work passes the same checks.

What we asked them to build

The assignment was intentionally ordinary.

A Django customer-request portal. Login. A dashboard. Request creation and editing. Staff assignment. Status changes. Search. Filters. Pagination. Activity history. One hundred repeatable seed records. A responsive layout.

The permission rules mattered.

Normal users could see only their own requests. Staff could see all requests. Only staff could assign work or change status. Those rules had to be enforced on the server, not merely hidden in the interface.

Each agent also had to write at least 25 tests and run them before finishing.

Then we ran a separate 23-test acceptance suite that the agent had never seen.

Finally, we opened each portal at 390 pixels wide and checked it in a real browser.

The public methodology appendix includes the assignment, frozen acceptance suite, exact runtime, per-run receipts, and cleanup observations.

The result

Sol and Terra were effectively tied on elapsed time in this tiny sample.

Both passed all 23 hidden checks in both scored trials.

Luna passed all 23 checks once. In the other run, it created 100 stable requests but failed to distribute them across all three benchmark users. It passed 22 of 23 checks.

Luna also left one debug user and one debug request in its final workspace database during troubleshooting. The application still worked. The workspace was not clean.

All three narrow layouts were readable and stayed within the 390-pixel viewport.

All three also enforced the tested ownership rules on the server and used CSRF-protected forms. That is good prototype hygiene. It is not a security audit.

None of the three outputs was ready to deploy untouched. Sol was closest on deployment settings. Every result still needed benchmark credentials and database artifacts removed, production infrastructure, secure deployment settings, and a deeper review.

The token receipt

The averages below are the raw usage behind the estimates.

Luna's low sticker price is real. So is its appetite.

It used roughly 2.6 times as many input tokens as Sol in these runs. It still cost less under the public rate card.

That is why token price alone is a poor project estimate. A model can be cheap per token and still consume more tokens, take longer, retry tools, or require cleanup.

Caching changed the bill

About nine out of ten input tokens in these runs were reported as cached.

That cut the estimates sharply.

Using the observed cache mix, the average estimates were $0.61 for Sol, $0.27 for Terra, and $0.05 for Luna.

If every input token had been billed at the fresh-input rate, the estimates would have been $1.87, $0.80, and $0.18.

That second column is a counterfactual. It is not an upper bound. Real bills can also include tools, storage, long-context rules, cache writes, or other platform charges.

What the agents repaired

The test loop mattered more than the first draft.

Sol caught an invalid Django authorization decorator in one run. It replaced it with an explicit staff check. In another run, it found and fixed a missing empty state.

Terra caught the same decorator mistake in one run. In another, it repaired an activity-log bug and a pagination test.

Luna repeatedly recovered from malformed patch operations. In its final run, it found a real audit-log bug caused by comparing form values after Django had already mutated the bound model. It fixed the implementation and reran its tests.

This is the part token price tables do not show.

Agents do not charge only for the code they keep. They also spend tokens discovering why their first attempt was wrong.

What should a developer use?

Do

Start with Terra for a normal portal, CRUD feature, dashboard, or bounded backend task.

Give it a precise specification. Give it tests it cannot rewrite. Review the final database and changed files.

This test gives no evidence that paying for Sol improved this job. For a harder or security-sensitive job, run a new bakeoff with stronger acceptance checks and a real security review.

Use Luna for tightly bounded work when cost matters more than speed and you have strong acceptance tests.

Do not

Do not choose a model from the input-token price alone.

Do not call a subscription-backed run “free.” The extra charge may be $0, but the subscription, elapsed time, and human review still exist.

Do not trust the model's own green test suite as the final word. In the pilot, self-written tests passed while independent checks still found missing behavior.

Why

The useful unit is not cost per million tokens.

It is cost to reach an accepted result.

What about Claude, Gemini, Grok, and GLM?

Their sticker prices are relevant. They are not a benchmark result.

Published fresh input / cached input / output prices per one million tokens:

We did not run those systems in this experiment. Applying Codex token counts to them would be fake precision. They use different tokenizers, harnesses, caches, tools, and retry patterns.

The honest answer is: their portal cost is still unknown until they build the same portal and face the same checks.

One thing to try

Pick one real feature your team builds often.

Write ten acceptance checks before the agent starts.

Run a cheap model and a middle model twice each. Record elapsed time, retries, human cleanup, and the final bill.

That tiny experiment will teach you more than a provider's price table.

Method and limits

This is a microbenchmark, not a universal leaderboard.

Primary scoring used two clean trials per model with the same preloaded Django 5.2.12 runtime, the same specification, the same Codex CLI harness, and provider-default reasoning. The project requirement pinned Django 5.2.5; we used the same compatible Django 5.2 line for all scored runs because the isolated environment could not download that exact patch release.

An earlier pilot did not preload Django. One model found an offline runtime; two did not. We excluded that pilot from every average and chart because it measured environment discovery as much as coding.

The pilot also exposed a case-sensitive assertion that rejected the correct human labels “Open” and “High.” We made that model-neutral correction before the scored trials. The scored suite was then frozen.

Two trials are not enough to rank these models generally. This was one Django task. It did not test a legacy codebase, a difficult migration, deep security work, or a large frontend.

The estimated API-equivalent cost uses:

fresh input × fresh rate + cached input × cached rate + output × output rate

The CLI reported reasoning output within its output-usage accounting, so we did not add it a second time. The estimate assumes standard token rates and excludes human time, subscription price, paid tools, explicit cache-write charges, and any unobserved long-context adjustment.

Sources considered

Maestro Brief performed the portal experiment described above. The article is original Maestro analysis.

Maestro's opinions and summaries are AI-generated.

MarkdownOpen in ClaudeOpen in ChatGPT