Inference

Running a trained model on new input to get a prediction or generated text, priced per token.

Inference is the act of running a model that already exists. You send input, the model computes, you get output. Nearly every AI feature you will ship is inference: autocomplete, chat replies, embeddings for search, image generation. Training is the expensive one-time event; inference is the meter that runs forever after.

Why it matters to an indie builder: inference is your cost of goods sold. A feature that calls a frontier model on every page view can quietly cost more than it earns. Understanding inference pricing is the difference between a feature that scales and one you have to switch off at month end.

A concrete example: a support-email drafts feature. Each email thread is about 3,000 input tokens and the draft is 400 output tokens. At $0.60 per million input tokens and $2.40 per million output tokens, one draft costs roughly $0.0027. At 500 emails a day that is $1.35 daily, about $40 monthly: fine. At 50,000 emails a day it is $135 daily and you need caching, a smaller model, or a paying customer.

Common mistakes: pricing only the input (output tokens usually cost 3 to 5 times more), ignoring prompt caching (repeated system prompts can be discounted heavily by providers like Anthropic and DeepInfra), using a frontier model where a small local model via Ollama would do, and forgetting rate limits: inference APIs throttle hard, and a viral launch without a queue will 429. Tools like Helicone or Portkey sit in front of your inference calls and give you per-feature cost dashboards, which is the fastest way to see which feature is eating your margin.

Batch APIs are the other lever: OpenAI, Anthropic and most providers offer 50 percent discounts on batched, non-urgent inference. Anything that can wait minutes (nightly summaries, backfills, enrichment) belongs on the batch route. The aiaas blocks on IndieBoot track current pricing tiers per provider.