I Built a Second Brain Because I Hated Watching My AI Conversations Disappear
Second Brain Series — EP1·EP2: Building It
One day I spent over an hour with Claude tearing apart and rebuilding a project structure. We landed on something pretty good, I thought — but the next day when I tried to pull up that conversation, I scrolled and scrolled and couldn’t find it. I gave up and started the whole thing over from scratch.
One thought hit me right then. “This is just vanishing.”
Conversations I’d put real time into, decisions I’d agonized over for days, lessons I’d earned through trial and error — all of it buried somewhere in a chat list, evaporating without a trace. AI doesn’t remember what I said yesterday. I know that’s how it’s designed, but it still felt like a waste. If things were piling up anyway, I wanted them to leave some kind of shape.
That’s why I started building a second brain.
The first thing I decided: split raw and wiki
At first I thought “just throw it all in one folder.” Just collect everything, originals and summaries together, right? But once I actually used it, problems showed up. When an LLM merged several conversations into a summary, I’d later struggle to tell: “did I actually say this, or did the AI add its own spin while summarizing?” Because I had no idea where the original was.
So I set exactly one rule. Raw is mine. Wiki belongs to the LLM.
raw/ is the original conversations, unprocessed notes, sources with nothing touched. Names unchanged, content unedited. It’s the source of truth. wiki/ is the synthesized output the LLM produces after reading. If something’s wrong, I fix the wiki — but raw stays exactly as it was.
This separation means even if the LLM synthesizes something incorrectly, the original is safe. Knowing I have somewhere to go back to when I wonder “was that actually right?” — that peace of mind was bigger than I expected.
What to keep and what to throw away
At first I tried to write up every single conversation. Naturally, I couldn’t keep up. I needed a rule.
The default for keeping something is: “an AI conversation means there was intent to solve something.” If it wasn’t small talk, there’s probably something worth keeping. I throw away three things: pure small talk, how-tos that a Google search would answer in seconds, and one-off calculations I’ll never need again.
There’s one thing to be careful about here. Even if something is “findable on Google,” I keep it if it’s know-how I personally figured out in my own context. Even for the same technical topic, why I made this particular choice in this particular situation — that’s not on Google. General knowledge is a commodity. My experience is a moat. That distinction set the direction for the whole wiki.
How far should the automation go
I wanted to automate collection. I already knew that if I had to manually sort through every Claude Code session that piled up each day, it wouldn’t last. I wrote a script to automatically load everything into raw/ every evening at 10pm. That worked well.
The problem came with digesting. I thought “might as well automate this too” — so I added a cron job on the Hermes server to run automatic digests. The result: wiki quality actually went down. With the LLM running without judgment, the noise increased. I ended up killing the cron and went back to only running digests when I explicitly asked for one.
Digesting is work that requires judgment. What matters, what connects to which existing page, whether to keep or discard something — that can’t be run mechanically. Collect automatically, digest manually. That’s the approach that’s stuck. How this automation setup actually runs is written up separately.
For it to last, it has to be a system
Before I built the second brain, I thought “I’ll just push through it by willpower.” Summarize the conversation when it ends, write down decisions somewhere. It worked once or twice, but it never became a habit.
After collection got automated, it was different. With no friction requiring me to do something, things started accumulating. And those accumulated things became assets I could actually pull out and use later. Making the system run instead of relying on willpower — that was the real difference.
Starting with raw/wiki separation is enough. Don’t touch the originals, keep the summaries separate. Start there, and the rest builds on itself.
Second Brain Series — EP1·EP2 Building It (this post) · EP3 Integration · EP4 Discord · EP5 Using It · EP6 The App
Background: LLM meets you for the first time every new session · Method: Second Brain automation structure