Community · Announcements & Feedback
Context Window Recommendations & Tips for Saving Credits
Yumina · Staff ·
Before you touch this setting, it helps to know what gets sent to the AI every time you press send. Here is a stripped-down version of it:
` // —— World setup —— [lore 1: AI Instruction] You're the narrator of this survival horror game. Cold and restrained in tone, you never decide the player's actions for them.
[lore 2: World Settings] The long-abandoned Matsuzaki Sanatorium, sealed off in the winter of 1987.
// —— Conversation history —— [user] I push the door open
[assistant] A hand grabs your wrist — the skin unnaturally cold...
[user] I drink the potion `
The world setup is the lorebook. The author writes it: who the AI is playing, what the world is, the rules it has to follow. It has a fixed amount of information.
The conversation history is everything you and the AI have said so far. It grows every single turn.
Both get sent again, in full, every time you ask for a reply. AI re-reads the whole thing from scratch every time.
Why a long session costs more
AI providers charge by how much you send them. Your lorebook stays the same size, but your history keeps growing, so every turn costs a little more than the one before it. That is why the mushie cost per reply creeps up the longer you play.
Why we cannot send everything
In a perfect world the AI would see every word of your story forever and never forget a thing. Two things stop that. The cost above, and the model's own context limit, which is a hard ceiling on how much it can read at once.
So the line has to be drawn somewhere. That is what the where our current memory setting does (defaulted to 64k) It caps how much of the story gets sent, and it keeps the full lorebook no matter what. The world the author built is never the thing that gets cut.
What you give up
Once a story gets long, the AI starts losing the very beginning.
In practice this matters less than it sounds. To write the next good paragraph, the AI needs the lorebook and what just happened. It rarely needs chapter one.
When it does matter, that is what Session Memory & Story Summary is for. Instead of dropping old scenes, it summarizes them and keeps the summary. You lose the exact wording, not the events.
The mistake almost everyone makes
The obvious move is: I can afford it, so I will set it as high as it goes. That is usually the wrong call, and money is not the reason.
Too much history distracts the model. Buried in twenty thousand words of old chat, it drifts away from the lorebook and from what you just said. Smaller models suffer from this badly, but it happens to the big ones too. A tighter window is often a sharper AI.
So we recommend turning it down. Everyone, on every plan, whatever your budget.
What we actually recommend
Currently, the medium size for worlds is only about 7500 tokens, this means that with your default been set to 64000, you have more than 50000 tokens for context, which is extremely costly and can be unnecessary. With our new improved UI, we made a feature so that you can just focus on choosing the amount of conversation history you want to preserve. We recommend setting story memory somewhere between 12,000 and 24,000, and install Session Memory plugin.
Here is what that looks like on a normal world. A medium lorebook on Yumina is around 7,500 tokens:
| | Tokens | |---|---| | Lorebook | 7,500 | | Story memory | 15,000 | | Sent per turn | 22,500 |
All 15,000 of that is story, and all of it is recent. On a bigger world the lorebook half goes up and the total goes up with it, but your story memory stays exactly where you put it. That is why our new UI split it into two variables for you to edit.