AI TOKENOMICS · ASSESSMENT
How we run a tokenomics assessment.
A tokenomics assessment shows you where your AI spend actually goes, where it's heading, and where you can bring it down — measured on your own traffic, inside your own cloud. This page walks through what we examine, how we measure it, and what you keep at the end.
An AI bill arrives as a single number. It doesn't tell you which feature, customer or job spent it — and that mix keeps shifting month to month as work moves onto heavier models and agent loops multiply, often while the total stays flat enough to go unnoticed.
A normal audit reports that number and stops. We go further: we measure your spend inside your own environment, track where it's heading over time, and put a dollar figure on each thing worth fixing. Everything below is how that runs — and what you get. In short: we make the spend visible, score it, show where it's going, price the fixes, and — where there are easy wins — prove a few before we leave.
The four deliverables.
Everything comes out of one measured pass over your own data, so the breakdown, the forecast and the plan all trace back to the same evidence — not three separate exercises that disagree. You keep all four when we leave.
A living spend breakdown
Your AI bill attributed to features and jobs as far as the logs allow — with whatever can't be cleanly attributed shown on its own line, not forced into a bucket — tied back to the invoice finance already reconciles, and read across time rather than as a single-week snapshot.
A scorecard
Eleven signals across four lenses, each given a rating and a trend, and each gap turned into a dollar figure with the evidence attached.
A trend map and a do-nothing forecast
How your spend is moving, one axis at a time, with a two-to-four-quarter projection of the bill if nobody touches anything.
A plan to bring it down
Where there are easy, low-risk wins, we take them during the run and prove them on your own traffic. Beyond that, a prioritised list of the larger changes worth considering — each one sized so you can decide what's worth doing, rather than a promised re-architecture. All against a cost-per-outcome target you keep.
If your spend is already lean and the curve is flat, we say exactly that — a one-page all-clear — and stand down. The verdict is whatever the data returns, including "there's nothing here to do." We don't manufacture a problem to justify a second phase.
Four principles behind every number.
These are what make the numbers trustworthy. They hold from first access to final readout — each one a rule you can hold us to.
Test on your traffic, not a leaderboard
Every model and every fix is decided by re-running your own traffic against the alternatives and watching what happens — tokens, dollars, latency, whether the answer still holds. A public benchmark tells you nothing about where your workloads actually land.
Every number shows its source
Nothing is shown untagged. Each figure is marked by how solid it is — measured, logged, modelled, or assumed. A combined figure inherits the weakest tag of its inputs, and anything modelled ships as a range, not a single number.
Price the task, not the token
The unit that matters is one finished piece of work — which may be many calls, tool hops and retries stitched together. Per-token totals flatter the bill and bury the jobs that actually cost.
Nothing leaves your cloud
Reading logs, redaction and testing all run inside your own account and region; tests hit your endpoints on your quota. No prompt, response or invoice is ever copied to us — it's in the contract, not left to trust.
What's in scope, and the access we need.
The results are only as honest as the data underneath them, so we settle two things before starting: where the assessment begins and ends, and exactly what read-only access you give us.
How we measure the spend.
We do two things at once: rebuild the last two quarters of spend from your invoice inward, and start measuring live traffic going forward. Both are organised by job and laid out over time — because the point is to watch how the spend changes, not just read a total.
Group by job
A workload is the unit every later decision hangs on — a cluster of requests doing the same thing, like one product feature or one agent. When no clean list exists, we rebuild one: the label the app emits, else the caller's identity, else clustering prompts by their fixed structure. Anything that still won't attribute sits in a named bucket, never forced into a bin.
Rebuild the spend, month by month
The invoice sets the total finance trusts. Then we split each job into input / output / cached / reasoning — because that mix, not the headline, is what names the fix. Done month by month, it surfaces the early tremors: work climbing to heavier models, ratios drifting, agent loops multiplying, reasoning tokens bloating.
Meter live traffic
A gateway and a trace layer — whichever you already run — sit in front of traffic. Every call is stamped with who issued it, where, and the two fields that carry the weight: the task it belongs to (which fuses many calls into one unit of work) and the turn number (which exposes context growth).
Close-out check: the rebuild has to meet the invoice within 5%, or the difference is named — untracked keys, marketplace lines, rounding. We call the read done when tagging covers >90% of traffic, the draft breakdown reconciles within 5%, every job has a named owner, and at least two months of history are in hand to read the trend.
What we score, and how.
With the spend finally visible, we check it against four groups of questions — four lenses, eleven signals in all. Each signal gets two readings: a rating on a four-rung ladder (Blind → Visible → Under control → Improving on its own), and a trend showing whether the gap is widening, holding, or closing on its own. The rating isn't the point; the priced gap and its direction are.
Can you see it?
Can you see the spend, and see it move?Traceable spend
Can every dollar on the invoice be traced to a job, a team, and what it bought? Nothing downstream is trustworthy until it can.
How we read itReconcile tagged spend to the invoice; attribute the untagged tail by who owns the keys and by log volume. Many AI-native teams meter nothing below the invoice line, so real coverage is an edge.
FixesUnlocks every other lens, plus per-customer margin & pricing.
Cost per outcome
Does the setup know the price of one delivered unit — a resolved ticket, a completed task — and is that price tracked and actually used?
How we read itJoin usage to outcome events; where the join is missing, stand up a feature-level proxy and mark it modelled. Set unit cost against the outcome's value to read headroom or alarm.
FixesOutcome-priced billing, invest / retire calls, a margin-aware roadmap.
How fast you catch a spike
How long a cost spike lives before anyone notices — a prompt edit that triples context, a retry storm, an agent that forgets to stop.
How we read itWalk the last quarter for step-changes and, for each, separate when it began from when it was caught. If the record is silent, we stage a controlled spike and see what fires.
FixesCost alerts, per-key ceilings, cost-diff checks at deploy.
Right-sized models?
Is each job on the cheapest model that still holds up?Right-sizing
Whether each job runs on the cheapest model that still clears its quality bar — proven by replaying its own traffic, not asserted from a spec sheet. Top-tier to small pricing spans 10–25×; most extraction and classification never needed the top rung.
How we read itTake a representative slice of the job's own prompts and run it down an ordered set of models, cheapest first. Fire each a fixed number of times and grade every result against your own production output — your ground truth, never a benchmark we invented. Stop at the first one that holds. Anything that clears nothing goes to human review with its evidence, not shoved downward.
FixesStatic remapping — the highest-yield, lowest-effort move — then smaller custom models for narrow high-volume work.
Smart routing
Whether the setup routes by difficulty — cheap model first, escalate only what fails — or sends everything to one tier regardless of how hard the request is.
How we read itFrom the right-sizing tests, isolate jobs where a cheap model passes most of the time but not always — the cascade candidates. Model the blended cost of cheap-first-plus-escalate against the flat-flagship baseline, and price the difference.
FixesConfidence cascades, verifier-gated escalation, provider-fallback routing.
What's in each call?
What rides inside each call, and is it bloated?Context reuse
The shape and reuse of everything sent into the model. A stable prompt prefix can be re-read by the provider at up to a ~90% discount, so what matters is how much of each call is reused rather than freshly billed. Agent workloads run 100:1–300:1 input-to-output because the full context is resent every turn — which makes reuse the lever, not trimming replies.
How we read itRead cached-vs-fresh input straight from provider usage; analyse sampled templates for the reuse ceiling and the cache-breakers — volatile data spliced ahead of stable scaffold, tool lists that reorder, per-session IDs in the system prompt. Price the distance to the ceiling at the cached rate.
FixesStatic-prefix restructuring, cache-TTL tuning, an app-level semantic cache.
Output control
Control on the expensive side. Generated tokens price at 5–6× input, and reasoning tokens bill as output — often at a high default nobody set on purpose. The most common quiet overspend we find.
How we read itAudit request settings at every call site in code, joined to trace data. Compare what the model produces to what the downstream code actually consumes — three parsed fields don't need a paragraph wrapped around them. Inspect reasoning settings explicitly; unset or defaulted-high is where it leaks.
FixesOutput ceilings, structured outputs, per-class reasoning budgets, terser prompts.
Agent-loop cost
Tokens consumed per completed task inside agent loops, where each turn re-sends the whole conversation and tool output so far. It's the fastest-growing line in most setups: a ten-turn task can cost ~50× a single call, and trimming stale context alone recovered ~84% in a published 100-turn evaluation.
How we read itRebuild each loop from grouped traces and measure what fills the window turn by turn — instructions, tool schemas, history, results. Surface the usual offenders: full schemas resent for a two-tool task, multi-KB dumps never read again, no compaction, no stop condition.
FixesTool-result trimming, scheduled compaction, sub-agent isolation, loop budgets.
Discounts & guardrails
Are you paying list price, and are there hard limits?Discounts claimed
Whether you pay list price when you need not. Discounts already on the table — batch and flex tiers (a slower lane, roughly half price, for work no user is waiting on), caching, volume commitments — frequently go unclaimed. Prices fall ~10×/year at a fixed capability, so a routing choice two quarters old was made against a price sheet that no longer exists.
How we read itSize the eligible discount per class from the inventory, classify each job by how long it can wait, and price async volume stuck on the realtime tier at the flat ~50% recovery. Then find who owns the re-pricing cadence — usually no one.
FixesBatch / flex migration, commitment negotiation, a refresh cadence tied to releases.
Wasted spend
Money on calls that returned nothing usable — hard failures, re-parse retries, duplicate submissions, double-completed timeouts.
How we read itRead status codes, retry headers and near-identical prompts in tight windows from gateway logs, then attribute cause — re-parse loops point at missing structured output, 429 bursts at absent backoff, duplicates at missing idempotency. A well-run setup sits in the low single digits.
FixesStructured outputs, idempotency keys, jittered backoff, circuit breakers.
Spend limits
Whether spend has hard stops — per-key and per-team ceilings, rate limits, a kill-switch that actually fires — and whether anyone owns the page when one trips.
How we read itAudit gateway and provider config, then prove it — drive a test key into its limit and confirm the block, rather than trusting the setting. Ask who got paged the last time a ceiling tripped, and what happened next.
FixesEnforced per-key ceilings, dev / prod key separation, per-job spend targets.
Where your spend is heading.
This is the part a one-time snapshot can't give you, and the reason the whole assessment runs over time. Your spend is never still — model line-ups change, agent features spread, prompts grow, providers cut prices. A number that looked fine two quarters ago can be a live leak today with nothing obviously broken. We track each of these movements and project it forward.
Climb to heavier models
Spend drifting up the model stack. Usually one team adopts a new reasoning model and it seeps into jobs that never needed it.
Agent share
The slice of the bill sitting inside agent loops. A flat total can hide an agent share quietly doubling under it.
Context growth
The per-turn growth curve over time. Steepening month over month means history-replay cost is compounding.
Reasoning share
Thinking tokens as a rising fraction of output — the quiet drift as more work lands on reasoning-capable models.
Reply-length creep
Typical reply length per job over time, held against what the consuming code actually reads.
Waste trend
Waste as a moving ratio, not a point — retry storms and duplicates tend to grow with traffic until someone trips over them.
Unit-cost curve
Cost per outcome against volume growth. Rising unit cost on rising volume is the alarm finance actually watches for.
Price-sheet risk
Provider repricing and new releases you haven't reacted to — the outside axis moving whether or not anyone looks.
The do-nothing curve: from the measured motion we extend the bill two-to-four quarters out on the assumption that nobody intervenes — always modelled with a range, never a single line. What we find again and again: the total can sit flat while the mix underneath churns — and "just move everything to the newest model" often lifts the bill rather than lowering it. The value was never in the total; it's in the per-job distribution.
Putting a dollar figure on each fix.
A finding nobody has priced is just an opinion, so every gap and trend signal is turned into money the same way, each line carrying its evidence tag. We price each fix twice — against what you spend today, and against the do-nothing forecast you're already heading toward — because that second, growing number is what shows the real cost of waiting.
Where it lands untouched
Where the trend read puts the bill if nobody acts. Every saving below is measured against this line, not just against today.
modelled · rangeTest-proven
Every job on the cheapest model it actually cleared in the tests. This curve is measured, not estimated.
measuredEverything the profile unlocks
Fixes applied in order — route, then cache, then batch, then trim — because they stack, they don't add.
stackedRule: fixes stack, they don't add. Route a job to a model 10× cheaper and you've shrunk the base that caching then discounts — so they're applied in sequence, never summed, or the total lies. Every recommendation also carries a build estimate; anything paying back beyond two quarters is roadmap, not a quick win, and it's labelled as such.
What we ship during the run.
Most reviews end with a list of recommendations. Where we find easy, low-risk wins during the run, we take them — implement them and verify the saving on your own traffic — so the readout can open with results already proven, not just projected. If nothing is safe to ship yet, we say so rather than force it.
The final readout.
It all lands in a single ninety-minute session, with engineering and finance in the same room — because the real value is a conversation those two sides can finally have over one set of numbers. The running order:
And if the read comes back lean and the curve comes back flat, this meeting is ten minutes and an all-clear. We'd rather hand back a clean bill of health than talk you into a rebuild you don't need.