After building my second brain, I ran into a strange problem. I had a digest skill that was supposed to read my raw notes and turn them into wiki pages. But no matter how many times I ran it, barely anything came out.
I figured the skill needed to be smarter. So I added signal-filtering logic, a pin feature to mark high-priority notes, and tighter rules for what counts as worth saving. None of it helped. I had 1,861 raw files piled up, and my wiki was barely growing.
What I got wrong
My assumption was that the bottleneck was quality. If digest could just read and synthesize better, the wiki would fill up. More refined criteria, better signal detection.
That wasn’t it. digest just couldn’t find anything.
When you have 1,861 files, an LLM doesn’t read them all. In practice, it looks at filenames to decide what to open. And most of my filenames were date-plus-session-id strings — things like 2026-06-23__local-command-caveat Caveat The messages below wer__a48bda7b.md. There’s nothing in that name that tells you there’s useful wiki material inside.
The signal-filtering logic and pin feature I’d built were fine — but they only applied to new files going forward. The hundreds of files already in the pile were sitting there with nothing but a filename. I’d built features for the top of the funnel while the bulk of the archive sat completely opaque.
digest wasn’t stupid. It just had nothing to work with except file names.
So I built a summary layer
What I needed was an index that an LLM could actually read — not a filename, but a compressed version of the content that an LLM could scan to decide “yes, open this one.”
The format was simple. Read each file in full. Every time the topic shifts, write a topic sentence. Collect those into a topics + summary block per file:
path: raw/...
topics: [keyword1, keyword2, ...]
summary: 2–3 sentences on what happened and what was discussed.The work was slow — every file had to be read entirely. Excluding the projects folder, there were 341 unprocessed files. The size distribution was huge (biggest: 810KB), so I split them into 93 batches and ran them in parallel. The result was 635 cache entries.
Then I ran digest again
Wiki pages poured out. One run: 17 new pages created, 70 updated. 87 pages touched in a single session.
The digest logic itself hadn’t changed at all. The same skill that barely produced anything before now generated a flood. The only difference: before searching through raw files, digest now reads the cache first, narrows down to a handful of candidates, then reads just those. Instead of navigating by filename, it could navigate by content.
Where I was wrong
I’d been looking for the problem in the wrong place — trying to make the digestion smarter, when digestion was never the issue. The system couldn’t find what to digest.
I hadn’t thought about the fact that LLMs navigate file systems by filename. Once I understood that, the whole pattern was obvious: bulk-imported files have nothing but a name. New files coming in are cached as they arrive, so they’re fine. But the existing pile was just sitting there, invisible.
A summary index isn’t just for humans. It’s for the LLM. The way a reader skims a table of contents to decide which chapter to open — that’s how an LLM needs to navigate a large archive. Without that layer, it doesn’t matter how many files you have.
What I took from this
I called it a “second-level index,” but it’s not complicated. Read the files. Write a topic sentence when the subject shifts. That’s it. That’s what changed the productivity of my entire wiki.