Estimating what an AI feature will cost before you build it

The unit economics of an AI feature are knowable before a line of code exists, because the bill is arithmetic: tokens in times the input price, plus tokens out times the output price, times calls. The estimates that go wrong go wrong in the same three places.

Miss one: the prompt you don't see. Every call re-sends everything the model needs — system instructions, retrieved context, conversation history, tool definitions. A feature whose visible input is a one-line user query routinely sends thousands of input tokens per call, and for most production features input dominates the bill, even though output is several times pricier per token. Measure a realistic call, not the query.

Miss two: one feature, many calls. Anything agentic — tool use, retries, multi-step chains — multiplies the arithmetic by the number of steps, and each step often re-reads the context that came before it. A "single" answer built from five model calls costs like five calls. Count the calls in your worst realistic path, not your diagram.

Miss three: pricing the wrong model. The spread between models at similar quality is enormous — that is the whole story of our price index — and the same open model is often served at very different prices by different hosts. Price your actual candidate on its actual host, and note which levers your provider offers: prompt caching (a large discount on the repeated part of your prompt, where offered), batch endpoints for non-urgent work, and per-key spend caps as the backstop.

The method, in one pass. Write the real system prompt. Run twenty realistic requests through your candidate model and read the token counts the API returns — every provider reports them. Take the median tokens-in and tokens-out, multiply by the live prices, multiply by expected daily calls, then double it for the misses above. If the number only works at the cheapest host, check who that host is before betting the feature on it.

One structural choice changes the slope. For narrow, repetitive work — classification, extraction, tagging — a small model is often enough, and the cost difference against a frontier model is not a discount, it is orders of magnitude off the bill.

Where next: What is a token · Compare prices · Using a small model as a classifier