What is the context window and what does it do?

Alright, let me explain the “context window” in a way that’s clear and easy to understand—like I’m sitting across from you in a cosy London café.

The context window is essentially the chatbot’s memory during a single conversation. Imagine you’re chatting with someone who has a photographic memory but can only hold a set number of snapshots in their mind at any one time. That’s the context window. It’s the amount of text—whether that’s your messages, the bot’s replies, or instructions—it can remember and refer to as the conversation progresses.

For example, ChatGPT or Claude might have a context window of, say, 8,000 tokens. Tokens are like tiny chunks of words or characters—so “Hello, world!” might count as three tokens. Once you reach the limit of the context window, it’s like sliding a window along a train of words: the oldest bits of the conversation drop off, and the bot can no longer “see” them.

This has a big impact on the chatbot’s ability to follow a long conversation. If you’re discussing a complex topic over many messages, older details might get lost if the context window is too small. Bigger windows mean better continuity and depth in conversations, but they also need more computational power.

In the future, as Ai advances, we’ll likely see context windows grow larger, making these bots even more like chatting with a well-informed friend who remembers every detail.

Crafted for clarity and powered by innovation