What drives AI token cost up and down

Written byAnna Katharina Bollé

August 27, 2026

AI & Intelligence7 minutes

Your total AI token cost can rise while headcount and software contracts stay unchanged. Tokens are consumption units, so every prompt, response, document, automated call, and retry can add to the bill.

Track your AI-token spend in one place

AI-token spend can originate across employees, automated workflows and multiple providers, while Finance often only gains visibility after the money has been spent. That is why we built Moss AI Token Cost Tracker: a free dashboard anyone can run locally to bring all AI spend into one view tailored to Finance and trace costs to the provider, model, team, person, API key or workflow behind them.

Try it yourself by following the video tutorial and step-by-step instructions for your system below.

Clone and set up the moss-ai-token-cost-tracker tool for me. Act as my setup assistant using your file system and terminal access:
1. First check whether the current folder (or a `moss-ai-token-cost-tracker` subfolder) already contains this tool (look for server.mjs and index.html). If it's already there, run `git pull` inside that folder to get the latest changes, then skip straight to step 3.
2. Otherwise, clone the repo:
   git clone https://github.com/getmoss/moss-ai-token-cost-tracker
   then work from inside that folder, and check whether Git, nvm, and the Node.js version this repo's README requires are installed, installing or switching to whatever is missing.
3. Ask me whether I want to run it in production mode (connecting my real accounts with API keys) or demo mode (mocked data, so I can just see how it looks without any API keys). Wait for my answer before continuing.
4. Double check that everything needed is set up correctly, and then run `npm start` for production mode, or `npm run demo` for demo mode, based on my answer.
5. Watch the terminal output for errors. If something fails, diagnose the root cause, explain it to me in plain language, and either fix it or tell me exactly what to run.
6. Tell me the URL to open in my browser once the server is running.
7. Later, if I ask you to stop or restart the tool, do that for me.

Important: never read, print, or send the contents of my .env file, and never share any API keys or company data outside this terminal session.

A quick explanation of tokens

To understand how much do AI tokens cost, we must look at how they are measured. A token is a small unit of text processed by an AI model, which may be a word, part of a word, punctuation, or a number. The exact count varies by model, language, and content.

Input tokens cover prompts, documents, system instructions, and conversation history. Output tokens cover the model's response and are commonly priced at a higher rate. To manage this, teams often use an AI token calculator. Finance therefore needs more than the total bill; it needs to know which type of usage changed and why.

The following eight drivers explain most movements in token spend.

1. Agentic workflows and hidden calls

Spend goes up when one employee request triggers an agent to plan, research, use tools, check its work, and retry. The employee starts one task, but the agent may create many model calls in the background. A loop or repeated correction step increases the cost further.

To bring the cost down, remove unnecessary steps, limit retries, and set a ceiling for calls or tokens per task.

Finance signal: spend jumps after an automated workflow launches. Calls and costs per completed task are significantly higher than during the guided trial, suggesting that the agent has gone off track and needs tighter guardrails.

2. Movement to more expensive models

A team can switch to a premium model without buying a new tool or changing the number of requests. A provider or software product may also change its default, directly impacting your openai api cost. The result is a higher rate for the same volume of work.

Teams can reduce the cost by testing routine tasks on a lighter model and reserving premium tiers for work that needs the extra capability. The comparison should use the cost of a successful task, since repeated failures on an unsuitable model remove the saving.

Finance signal: cost rises while token volume and request count stay broadly stable. The model line items or premium-model share increase.

3. More providers, models, and subscriptions

Different departments may add separate AI tools for coding, research, customer support, or legal work. Each local purchase can look manageable, but complex api pricing structures mean the combined company cost grows and finance starts to lose oversight. Overlapping subscriptions and untracked API accounts make the category harder to control.

The practical response is to define approved tools and models by use case, remove duplicates, and keep a central inventory.

Finance signal: more AI vendors, subscriptions, models, or invoice lines appear than the central inventory records. The spend is spread across cards, provider invoices, and cloud accounts.

4. More requests and usage events

Spend rises when more employees, customers, products, or automated workflows call AI models. A launch, campaign, contract-review sprint, new team, or new agent can create a sharp increase even if the rate per token falls.

Removing unnecessary calls brings total spend down. Forecasting planned events and setting alerts will not remove valid usage, but it makes the cost more predictable.

Finance signal: total tokens and total spend rise together. Cost per request and effective rate remain fairly stable, pointing to volume rather than a pricing change.

5. Weak or falling cache use

For API workflows that repeatedly use the same system instructions or reference material, caching can meaningfully reduce input costs. Stable content may be served at a lower cached-token rate instead of being processed at the standard input price each time. Changes near the start of a prompt can reduce how much content qualifies for caching.

This saving is mainly controlled by engineering through the way the API workflow is structured, rather than by employees using standard AI applications. Common context should remain stable and consistently positioned, while cache performance should be monitored across similar workflows.

Finance signal: for the same model and a similar volume of work, costs rise as fewer input tokens are served from cache and more are processed at the standard input rate.

6. Longer context and conversation history

Large system prompts, full documents, redundant extracts, and long chat histories all add input tokens. Earlier messages may be processed again as a conversation continues. The request count can stay flat while each request becomes larger.

Employees can start fresh chats for new tasks, while workflows pass only the context required for the current job.

Finance signal: input tokens per request rise. Total requests remain stable, but the input share of the bill grows.

7. Longer model responses

Reports, code, explanations, and multiple regenerated versions can create more output than the task requires. Since output tokens commonly carry the higher rate, a change in response length can have a visible effect on cost.

Users can reduce unnecessary output by specifying the required format and length, such as five bullets or a 150-word summary. The limit should still leave enough room for a useful answer.

Finance signal: output tokens per request rise, along with the output share of total cost. Input volume and request count may remain steady.

8. Faster or premium processing

Some workloads use fast or priority processing even when an immediate response is unnecessary. Non-urgent reporting and bulk document work may also run in real time when a lower-cost batch option is available.

Standard processing can be the default, with batch processing for suitable work and premium speed reserved for time-critical tasks.

Finance signal: the effective rate rises without a matching change in model, token volume, or request count. Processing-mode line items or service settings change.

A quick diagnostic for finance

• If cost and tokens rise together, investigate request volume and planned usage events.
• If cost rises while tokens stay stable, inspect model tier, processing mode, and effective rate.
• If input per request rises, check conversation history, system prompts, and document context.
• If output per request rises, check requested response length and regenerations.
• If the effective rate rises on the same model, check cached usage and contract or pricing changes.
• If calls and costs per task rise after automation, inspect agent steps, retries, and loops.
• If vendor and model counts rise, look for overlapping tools and needed capabilities.

The pattern matters because each one needs a different response. Asking employees to write shorter prompts will not fix a premium-model migration. Renegotiating a provider rate will not fix an agent caught in a loop. Finance needs the driver before it can send the problem to the right owner.