Tracking Your AI Stack: A Quarterly Ritual for Builders

If you tried five new AI tools last month, you'd probably struggle to name three today. The AI ecosystem moves faster than memory can keep up — and most builders end up re-evaluating the same tools twice, six months apart, with nothing written down between attempts. This is a quarterly ritual for fixing that: a structured snapshot of what you tried, what you shipped, what you learned, and what stuck. Twenty minutes, every twelve weeks. The receipts of your craft, kept on purpose.

Why your AI stack disappears from memory

The forgetting curve is real and it applies to tool evaluations the same way it applies to anything else. The names of the agents you tried last March, the prompts that worked, the API quirks you patched around, the model that surprised you on a Tuesday — they fade. Without an external record, most of the work of evaluating your stack ends up redone every six months, with no compounding learning across the attempts.

Tool fatigue makes the problem worse. The release cadence of AI tooling has outpaced human cognitive bandwidth for category-tracking. New models ship weekly. New frameworks every other month. The pace isn't slowing. The only way to retain signal from your own experimentation is to write the signal down at the time you have it.

The five-section snapshot

An AI Stack Snapshot is the solo-builder version of a brag doc. The format is intentionally simple — five sections, twenty minutes to fill if you have the raw material, fifty if you're reconstructing. Done every twelve weeks, it becomes a quarterly ritual that captures the shape of your work in a way no other artifact does.

1. Tools tried this quarter (kept / dropped / maybe)

Every new tool, model, or service you evaluated. The verdict — kept, dropped, or maybe-revisit — and one sentence on why. This is the section future you will rely on most. The dropped tools matter more than the kept ones; specifically naming why you dropped something is what stops you re-evaluating it six months later.

• For each tool: name + version, the use case you tested it against, the call (kept / dropped / maybe), and the one-sentence reason.
• Cost per use or per month, with the date. Pricing changes fast; the date matters for future-you.
• Be specific about scope. 'Tried Claude' is useless. 'Tried Claude Sonnet 4.5 for code review on the walima-backend repo, 3 weeks, 40 PRs' is the entry you'll want.

2. Projects shipped or moved forward

What you built, launched, or substantially advanced this quarter. Lead each bullet with the outcome, then the work. This section is the closest thing solo builders have to a performance-review entry — the shipped work, the scope, the impact, in a form that holds up to scrutiny three quarters later.

• What did you ship publicly? Who's using it? How many?
• What did you build for clients or internal use? What did it replace?
• What did you advance significantly but not yet ship — and what would 'shipped' look like next quarter?
• Numbers where you have them: installs, revenue, users, time saved, errors prevented.

3. Skills developed

The capabilities you didn't have last quarter that you have now. The technical pattern you mastered, the workflow you adopted, the judgment you sharpened. This is the section that, year over year, tells you whether you're growing or treading water. Most builders avoid writing this section because it forces an honest answer to a hard question.

• What can you do this quarter that you couldn't do last quarter?
• Which technical pattern, framework, or service did you go from 'reading docs' to 'shipping with'?
• Where did your judgment improve — bugs you now catch earlier, tradeoffs you now see more clearly?
• What's the next skill you want to compound on?

4. Lessons learned

What surprised you. What didn't work that you expected to. The moment you realized you were wrong about a tool, a framework, or your own approach. These are the entries that make the snapshot worth re-reading; the kept-tool list ages quickly, but lessons compound.

• Which assumption you carried into the quarter did you have to revise?
• Which bet paid off in a way you didn't predict?
• Which bet didn't pay off, and what did you learn from the loss?
• What would you tell yourself at the start of this quarter?

5. The stack as of today

The tools you actually reach for, not the ones you wish you used. A snapshot of your production-grade stack — coding, writing, research, design, deploys, ops — as it stands on the day you wrote this document. Twelve weeks from now you'll write a new one and the diff is the story.

• Daily-use stack: coding, writing, research, ops, design, deploys.
• Which tools graduated from 'experimenting' to 'permanent' this quarter?
• Which tools left the stack since last quarter? Why?
• What's still on the watchlist for next quarter?

A worked example

What does a real entry look like? The strong version of each section is concrete, dated, and specific. The weak version is vague enough that it's worse than nothing — it's a placeholder that fooled you into thinking you did the work.

Weak entry: 'Tried a few AI coding tools.' Strong entry: 'Tried Cursor for three weeks against my existing VS Code + Copilot setup. Switched permanently — the Composer mode for multi-file refactors saves ~2 hours per week on the walima-frontend repo. Kept Copilot for inline suggestions inside Cursor. Cost: $20/mo for Cursor on top of $10/mo for Copilot. The combo is worth it; either alone wasn't.'

The difference is specificity. The strong version compounds because three quarters later, when you're evaluating yet another coding tool, you can read this entry and immediately know what bar you're measuring against.

When to write the snapshot (and when not to)

The right cadence for an AI Stack Snapshot is every twelve weeks. Faster than that and there isn't enough delta — you'll repeat yourself and lose the ritual. Slower than that and the memory you're trying to capture has already started fading.

• Quarterly: write the full snapshot, all five sections.
• Monthly: just-in-time scratch notes on tools you're evaluating, kept somewhere you can pull from at quarter-end.
• When you finish a major project: a project-specific recap that feeds the next quarterly snapshot.
• Never: don't write a snapshot in the same week you're doing your year-in-review. Save the snapshot for a regular quarter and pull from it for the recap.

Mistakes to avoid

Curating for an audience you don't have

The snapshot is private. The minute you start writing for an audience — a future investor, a client, a Twitter thread — you'll soften the parts that matter most. The dropped tools become 'still evaluating.' The failed bets become 'pivoted from.' That edit kills the document's value. Write it as if no one will read it but you.

Skipping the 'dropped' column

The instinct is to focus on what worked. The 'dropped' column is the higher-information section. A list of tools you specifically chose not to keep, with reasons, is the asset that stops you re-evaluating the same things six months later. Don't skip it.

Treating it as a shipping log

A shipping log lists what you shipped. A snapshot includes what you tried, what you learned, what you almost shipped, and what you decided not to build. The full picture is more useful than the shipping subset — and it's the part that doesn't show up anywhere else.

Make the ritual stick

The hard part isn't writing the snapshot once. It's writing it four times a year for the next three years. Three things make the difference: a calendar reminder, a place to drop notes throughout the quarter, and a low enough capture friction that you don't avoid the running notes between snapshots.

• Calendar block: 'AI Stack Snapshot — Q1' for thirty minutes at the start of each new quarter. Recurring.
• Running notes file: somewhere you drop one-liners on tools you're evaluating, lessons you noticed, projects you advanced. Doesn't need structure — the snapshot is where structure happens.
• Voice capture: the lowest-friction running-notes substrate. Talk for thirty seconds about the tool you just tried, the bug you fixed, the thing you learned. Capture is the hard part; structure is the easy part.

Take twenty minutes this quarter. Write the snapshot. Save the markdown file. In a year you'll have four documents that read like a story of how your craft evolved — useful when a client asks 'what have you been working on,' more useful when you ask yourself whether you've gotten better. Most builders won't do this. The ones who do compound, because they can see the curve.