It's measured in tokens.

Anthropic's context window documentation calls it the model's "working memory", separate from the data it was trained on. A desk is a fair comparison: everything you're working on has to fit on the surface at the same time, and once it's full, something has to come off before anything else goes on.

What fills the context window?

Far more than the message you just typed. The product's system prompt, the earlier conversation, any files or documents, notes loaded from memory, the results of any tools the agent used, and the reply itself all count towards the total. In an agent working through a task with several steps, tool results can take up a large share.

How big is a context window?

It depends on the model. Several current models accept around a million tokens, roughly 750,000 English words, and many others take a few hundred thousand.

A bigger window helps with long documents, but filling it has a cost. Anthropic's documentation notes that accuracy and recall degrade as the token count grows, which it calls "context rot", and details buried in the middle of a long input are the easiest to miss. If an answer ignores something you know was in the document, this is a likely reason.

Why does an agent forget what you said earlier?

In a long conversation, the earliest messages are the first to go once the window fills. Depending on the product, they're dropped or replaced with a summary that keeps the gist and loses the detail. An instruction you gave three weeks ago may no longer be in front of the model at all, unless the product saved it separately as a memory.

What helps is giving the model less, more precisely:

  • Start a fresh thread when you change topic.
  • Ask for a summary of what matters before carrying on with a long piece of work.
  • Put standing preferences somewhere the agent stores them, instead of repeating them in chat.

An example

You want a view on one clause in a 60-page shareholder agreement. Pasting all 60 pages buries that clause among everything else, and on a smaller window it can crowd out your instructions. Pasting the two pages around the clause, with one line on what's worrying you, gets a sharper answer for a fraction of the cost.

Context window vs AI memory

Whatever sits in the window is gone once the conversation ends or is cleared. Memory is a separate store that the product writes to and reads from, inserting the relevant facts into the window when a new conversation begins. A good memory system means less has to sit in the window, because the agent brings in only what's relevant to the task in hand.

Common questions

Does the context window reset when I start a new chat?

Yes. A new conversation starts with an empty window apart from the system prompt and anything the product loads from memory. That's why switching to a fresh chat can fix an assistant that has started mixing up details from earlier.

Do images and PDFs use up the context window?

Yes. Images and documents are converted into tokens and count towards the window just as text does. A long scanned PDF can use considerably more of it than the same words pasted in as plain text.

Does a model with a bigger window cost more to use?

Only when you fill it. You pay for the tokens you send, so a short message costs the same on a large-window model as on a small one at the same rate. Some providers charge a higher rate once a prompt passes a certain size, so very long inputs can cost more per token as well.