AI Tokens and Context Windows: What Sets Your Bill
By Alexandre Saint-Jean
Audio version
Audio version produced by text-to-speech from the article. Our AI charter
A token is the unit behind every AI model's bill: not a word, but a fragment of text. Every model charges separately for what it reads and what it writes, at different rates. Understanding this mechanic explains a line item that shows up, month after month, on an AI usage statement.
What is a token, exactly?
A token is a fragment of text, not a full word. A common word often costs a single token, but a rare word, a technical term, or a word with unusual spelling frequently splits into several fragments. There is no fixed, universal ratio between word count and token count: it depends on the model and the language used.
The order of magnitude is enough to read a bill sensibly: the longer a text, and the more unusual vocabulary or accented characters it carries, the more tokens it consumes, regardless of the word count shown on screen. Two texts of similar visible length can cost noticeably different volumes of tokens.
Why are input and output priced differently?
This is the detail almost nobody checks before running a model in production: a model bills what you send it (input) and what it produces (output) separately, at different rates, with output consistently priced several times higher than input.
The practical consequence: pasting a ten-page document into every question, to get a three-line answer, charges the whole document on the input side, every time. Multiplied across a day's worth of questions, that is not a rounding error, it is often the line that explains a bill higher than expected.
A dated example (checked 17 August 2026): Gemini 3.7 Flash charges $0.75 per million input tokens and $3.75 per million output tokens, an introductory rate running until 31 December 2026 ($1.50 and $7.50 afterwards). At OpenAI, standard rates range from roughly $0.20 / $1.20 per million tokens for the lightest model to $5 / $30 for the most capable one, input then output. These rates move fast: check the official pricing page at the time of use.
A workflow built on long questions and short answers stays relatively cheap. A workflow that pushes a model to draft long documents costs noticeably more at a comparable token volume, because output is priced higher. Choosing the right model for a given task goes beyond this one point: that is the cost, latency and quality decision framework.
What is a context window?
The context window is the total number of tokens a model can hold at once, question, attached documents and answer included. Beyond that limit, text has to be split, summarised, or reduced to only the passages relevant to the question at hand.
That is exactly the logic behind RAG, covered against fine-tuning in this article: the context window is finite, and filling it in full on every question costs money for no benefit.
A dated example: Gemini 3.7 Flash offers a context window of up to 1 million tokens, with output capped at 64,000 tokens (source: official Google DeepMind model card, checked 17 August 2026). A large window lets a model process a substantial document in one pass, but it does not change the separate input/output billing logic covered above.
Is a large context window the same as memory?
No. The context window empties between conversations: nothing said in one exchange stays available in the next, unless it is resent manually or the tool in use carries it forward behind the scenes.
A second point worth correcting: filling the window to the brim does not automatically improve an answer. A context cluttered with irrelevant material dilutes the useful information and often lowers quality rather than raising it. The better habit is not sending everything available, but sending what actually serves the question.
This distinction connects to a broader confusion about what an AI model actually is: its context window defines what it can read at once, not what it retains from one conversation to the next.
Frequently asked questions
- Is a token the same as a word?
- No. A token is a fragment of text, usually shorter than a full word. A common English word often costs one token, but a rare word, a technical term, or a word in another language frequently splits into several fragments. There is no fixed ratio between word count and token count: it depends on the model and the language.
- Are the prices quoted in this article still current?
- No, they move quickly. The figures here were checked on 17 August 2026, and some already have a known end date: Gemini 3.7 Flash's introductory rate changes on 1 January 2027. Check the model's official pricing page at the time of use rather than trusting a figure recorded months earlier.
- Why can a short answer still be expensive?
- Because input is billed too. Pasting a long document into every question charges that whole document on the input side, every single time, even when the answer is two lines long. That line item, more than answer length, usually explains a higher-than-expected usage bill.
- Does a bigger context window always mean better answers?
- No. A larger window lets a model process more text at once, but a context cluttered with irrelevant material dilutes the useful information and often lowers answer quality rather than improving it. Sending what actually serves the question, not everything available, remains the better habit.