| Step (cumulative) | Monthly cost | Saved so far |
|---|
How AI APIs bill tokens, and how to cut the bill
Your bill is not "messages × price". Every API call is stateless, so the provider reads the full prompt again each time. Three things set the size of that prompt: the system prompt you send every time, the history of the conversation, and the length of each answer, which is also the most expensive part per token.
Where the money goes
Output tokens usually cost several times more than input tokens. In a long chat the input grows with each turn, so a conversation of ten turns can cost many times more than two single questions. Use the "Which lever saves what" table above to see which change pays off most for your usage.
Free or nearly free savings (no quality loss)
- Prompt caching. Put stable text first (system prompt, tool definitions, reference documents) and the changing text last. Anthropic needs you to mark the cached block; other providers may do it automatically for long prompts. Check the docs for the minimum length and how long the cache lives.
- Shorter system prompt. Remove duplicate rules and examples you do not really need. A 1,500-token prompt sent 3,000 times a month is 4.5 million tokens before any user writes anything.
- Ask for short answers and cap the length. Use the provider's max-tokens setting, and tell the model "answer in under 100 words" or "return only the JSON".
- Batch non-urgent work. Reports, tagging and bulk rewriting do not need a live answer. Many providers offer a batch API at a discount.
- Cache repeated questions in your own app. If the same question comes in often, store the answer and skip the call.
Trade-offs: measure before you apply them
- Trimming history. Sending only the last few turns cuts cost, but the model forgets earlier facts. A short running summary of the conversation keeps the key facts for a few tokens.
- Routing to a cheaper model. Simple classification, extraction and formatting often work well on small models. Test on 30 to 50 of your real examples, compare quality, and route by task type, not at random.
- Retrieval instead of whole documents. Sending only the relevant passages of a document is cheaper than pasting the whole file into every request. It adds engineering work and can miss context.
Apps, plugins and tools
- API gateways with cost tracking. Tools such as OpenRouter, Portkey and LiteLLM record token use and estimated cost per request, and can switch models from one place.
- Provider dashboards and budget alerts. Set a monthly spending limit and an alert at 50% and 80%. Most of the surprises come from one runaway script, not from normal use.
- Coding assistants. Clear or compact the context between unrelated tasks, keep the project memory file short, and attach the file you need rather than the whole folder.
Prompt templates that cut tokens
Answer in at most 80 words. No introduction, no summary at the end.
Return only valid JSON with the keys: title, category, priority. No other text.
Use the conversation summary below instead of the full history: [summary]
Suggested order
1) Turn on caching and shorten the system prompt. 2) Cap output length. 3) Only then trim history or route to a cheaper model, and check quality on real examples each time.
Frequently asked questions
How do AI APIs charge for tokens?
Most providers bill input tokens (everything you send: system prompt, history, user message) and output tokens (the answer) separately, per million tokens. Output usually costs several times more than input.
Why does a chat get more expensive as it goes on?
Chat APIs are stateless. Each new message re-sends the whole conversation so far, so input tokens grow with every turn. A 10-turn chat sends far more than ten times the tokens of a one-off question.
What is prompt caching?
Providers can reuse the processing of a repeated prefix (usually the system prompt and fixed documents) and charge a lower rate for those cached input tokens. Caching only helps when the start of the prompt is identical between requests, so keep stable text first.
How accurate is this calculator?
It is an estimate. Prices in the presets are examples, so enter your provider's current prices. Token counts are your own guesses about message length; real tokenizers differ, and non-English text (including Russian) often takes more tokens per word.