Tim O’Brien Warns $500 AI Spend Metrics Mislead as Tokenomics Fuels Hangover

1 articles · Updated · O'Reilly Media · Aug 28

$500 spent on an hour of AI work may be impossible to judge from token counts alone, Tim O’Brien argues, because the bill shows usage and model calls but not whether the work created value.
Tokens fail as a common currency, he says, since a million tokens can mean very different things across models, tasks and architectures, and a cheap model can cost more overall if it triggers retries or bad output.
Layered AI stacks make that mismatch worse: companies may mix frontier, mid-tier, local and specialist models billed by token, minute, seat or request, with some workloads running on owned hardware.
O’Brien says AI financial management should tie consumption to workflow, quality, architecture and business outcomes rather than rely on vendors promising they can 'just optimize' token spend.