Every time I opened a new session, the context I’d built up the day before was just gone. This was while building my stock app. The architecture direction we’d settled on, the alternatives we’d ruled out — the new session knew none of it. At first I coped by pasting: “this project is such-and-such, here’s what we’ve done so far, yesterday we decided this.” As that repeated, the background explanations started piling up like a tax.
It wasn’t just coding projects either. In GPT I split things into projects, split them further into sessions — and it would still drag in content from other projects, and still make me re-explain the background in every new chat. I ended up not using GPT projects at all.
This post is the story of following that problem until it arrived at context engineering. It sounds grand, but the question is single: what do you ship, every time, to an LLM that has no memory?
LLMs keep no memory
The starting point is a fundamental property: when a session ends, the LLM retains nothing. Yesterday’s conversation isn’t stored anywhere in the model. The next session’s Claude is not the previous Claude. It starts from a blank slate.
I’d heard “it has no memory” when I first learned about LLMs. Back then I nodded and moved on — it took living through the scenes above to understand, bodily, what that sentence means.
Then how does a chat remember what I said earlier?
There’s an odd wrinkle here. No memory, you say — but within the same chat room, it remembers what was said minutes ago just fine?
The trick is simple. It doesn’t remember; the entire conversation gets re-sent from the top, every time. Each time I send a message, the chat app hands the model the whole history so far, and the model reads it fresh, every single time. It only looks like remembering.
Once you see this structure, two things follow immediately: the longer the conversation, the more gets sent each time — and all of it is token cost. I ran into this concretely in Claude Code: I tried to continue a new topic inside a session that had gone deep on one subject, and all those earlier search results kept riding along with every message of the new topic, leaking tokens and diluting attention. Hence the advice: move new topics to a new chat.
What about project information, then?
Conversation history the chat app re-sends for you. But project information? “What this project is, how it’s structured, what decisions we’ve made” — that lives nowhere in the conversation history. That’s exactly the part I’d been laying down by hand at the start of every session.
Claude Code has a designated place for this. Put a CLAUDE.md file at the project root and it’s read automatically when a session starts — standing context that rides along with every conversation. Put your project information there, and a fresh session’s LLM starts having already read it.
At first I only put in rules — coding conventions and the like. Then I started adding other things: “here’s the current state of this project,” “we made this decision for this reason,” “we tried this and abandoned it because of that.” And suddenly the LLM became someone who knew me. I could open a new session and pick up right where we left off, no background briefing.
I put everything in — and got ignored
So just put all the project information in CLAUDE.md? That’s where I went too. Figuring more is better, the file kept growing — and at some point, convention violations started appearing. Rules plainly written in the file, not being followed. The “present but ignored” state.
Digging into it, the cause was CLAUDE.md overload. Two problems compound.
One is cost. CLAUDE.md is re-read with every message. Every character in it rides on every request for the whole session. The longer the file, the more that tax recurs through the entire conversation.
The other is compliance. As context grows, the model’s attention spreads across all of it, and a specific rule buried in a distant standing block loses priority. The longer your CLAUDE.md, the lower the compliance on each individual rule. Always present turned out not to mean always followed. “Too much context makes it dumber” isn’t just a saying.
So now I carry only the table of contents
Out of this came one judgment criterion: “Is this worth carrying in every single message?”
If yes, it stays in CLAUDE.md. If not, it moves out. The actual project content goes into documents under docs/, and CLAUDE.md keeps only a table of contents pointing at them — one-line pointers like “when working on P&L, read docs/pnl.md first.” The heavy content stays out of context in normal times, and when the relevant task arrives, the LLM reads that file into just that session.
The minimal skeleton that survived in my projects is three files: index.md (what state are we in), todo.md (what’s next), log.md (what’s been done). Of these, index.md is the heart — it’s the handover memo you give the new session’s LLM, so it can locate the project within a minute.
One condition attaches to this structure: the documents the table of contents points to must be current. If index.md is frozen at two months ago, the LLM works from a two-month-old map. So the habit of updating docs whenever the code changes is half of this whole structure. Documentation is density, not volume — only what the LLM can read now has value.
What my CLAUDE.md looks like
Talk only goes so far, so here’s a skeleton example — a simplified version of what I actually use.
# trading_mvp
Stock portfolio app. Working branch is dev.
## Standing rules (short and universal only)
- Run tests before committing.
- Write documents in Korean.
## Document index (read when needed)
- Current state → docs/index.md
- To do → docs/todo.md
- Done + decision history → docs/log.md
- Working on P&L logic? → read docs/pnl.md firstTwo points. Standing rules: only what applies to every task, kept short. Everything else: pointers. When this file starts running past one screen, it’s time to move something out.
Start with a single paragraph of index.md. “What state is this project in right now,” three to five lines. It becomes the first briefing you hand the LLM the next time you open the project.