Model Evaluation

I Had No Documentation. Then Everything Crashed.

When I started building TuXo, my autonomous referral agency, I was deep in the code every single day. Five specialized agents, a shared LLM pipeline, an.

The false confidence of "I'll remember this"

Here's a lie every solo founder tells themselves: I'll remember this.

You won't. I promise you, you won't.

When I started building TuXo, my autonomous referral agency, I was deep in the code every single day. Five specialized agents, a shared LLM pipeline, an email outreach system that runs 24/7, a dashboard that tracks everything from lead scoring to pain signal quality. This is exactly the kind of personal AI agent system that needs documentation. I knew every file, every function, every weird workaround I'd put in because I was shipping fast and "I'd clean it up later."

I didn't write anything down. Why would I? I built it.

It's in my head. Writing docs is for teams, for big companies with onboarding processes and knowledge bases nobody reads. I'm a solo founder.

I don't need that overhead.

That feeling lasted about three weeks.

Three weeks of building, then a weekend off, then Monday morning and I'm staring at a module I wrote fourteen days ago like it was written by a stranger. What does this function do? Why did I name this variable that way?

What connects to what? Why does the Hunter agent reference a config field that doesn't exist in the schema?

I spent two hours that Monday morning just reading my own code, trying to rebuild context I'd already had. Two hours. That's not engineering.

That's archaeology, and it's your own codebase you're digging through.

The moment everything broke

The real wake up call wasn't Monday morning confusion. It was a crash.

I was refactoring the agent pipeline. The Sender agent, the one that composes and sends cold outreach emails, was getting tangled. I wanted to separate the personalization logic from the sending logic.

Clean separation of concerns, the kind of thing you read about in architecture books and feel good about doing.

So I started moving things around. I moved the personalizer into its own module. Updated the imports.

Ran the tests. Some passed, some failed, but the failures looked like test data issues, not real problems. I fixed the test data, everything passed, and I deployed.

Then the emails started going out wrong.

The personalization was working, technically. The LLM was generating hooks, injecting variables, all of that was fine. But the template engine, which I had also moved, was now pointing to a different config path.

A config path that didn't have the right email templates loaded. So the system was sending personalized intros with the wrong body copy.

I didn't catch it for six hours because the dashboard showed green across the board. Emails were being sent. The pipeline was healthy.

The metrics looked fine. But the content was garbage.

I spent an entire day tracing the problem. Not because the bug was complex, but because I had no documentation of the data flow. I didn't know which config file fed into which module.

I didn't know that the template engine had a fallback behavior that silently loaded default templates when the specified path didn't exist. I didn't know any of this because I had never written it down.

The fix took twenty minutes. Finding the fix took eight hours. That's the tax you pay when you don't document.

What a single source of truth means

A single source of truth doesn't mean a wiki. It doesn't mean Confluence pages that nobody updates. It doesn't mean a README that's six months out of date.

It means one place, one format, always current.

For me, that's a set of markdown files that live in the same repository as the code. Not separate. Not in a different tool.

Not behind a login. Right there, next to the thing they describe.

The structure is simple:

  • MEMORY.md, the index. A topic map with pointers to details. Think of it as the table of contents for everything I know about the system.
  • BOOT.md, the operational context.

    What agents are running, what campaigns are active, what the current tech stack looks like, what the last known state is.
  • Daily logs, raw notes from each day. What I did, what broke, what I learned. Messy, honest, and searchable.
  • Architecture docs, the actual system design.

    Data flows, module relationships, config structures. The stuff I wish I'd had during that crash.

The rule is brutal and simple: if it's not written down, it doesn't exist. Period.

No "I'll add it later." No "I know this already." Write it now, or lose it forever.

It's better to waste money on missed projects because you were too busy documenting than waste millions when you're ready to scale.

This quote hangs in my head every time I think about skipping documentation to ship faster. Shipping faster without docs is like running faster without shoes. You'll get somewhere, and then your feet will bleed.

Why you should never delegate documentation

I've seen this advice floating around: hire someone to write your docs. Get a technical writer. Outsource it.

No. Absolutely not. Not when you're a solo founder.

Documentation is not a separate activity from building. Documentation IS building. When I write down how the pain signal quality scoring works, I'm not taking time away from development.

I'm forcing myself to understand my own system at a deeper level. I'm catching design flaws before they become production bugs. I'm creating a feedback loop that makes the next build cycle faster.

The person who builds the thing is the only person who understands the tradeoffs. Why did I choose SQLite over PostgreSQL for the lead database? Because the query volume doesn't justify a full database server, and the backup story is simpler.

Why is the classifier using a different model than the outreach composer? Because classification needs speed, not creativity, and deepseek-v3.2 handles structured output better than the alternatives at that price point.

A technical writer won't know that. They'll document the what, not the why. And the why is what saves you eight hours during a crash.

If you're a team of one, you are the documentation team. There's no escape hatch. Write it yourself, or lose the context yourself.

The real cost of not documenting

Let me put some numbers on this.

Before I started documenting properly, I was spending roughly 20 percent of my development time rebuilding context. Not writing new features, not fixing bugs, just figuring out what I'd already done. On a 50 hour week, that's 10 hours.

Ten hours of archaeology on my own codebase.

After I started my documentation system, that dropped to maybe 3 percent. A few minutes at the start of a session reading BOOT.md and today's daily log, and I'm back in the flow. The math is stupidly simple:

  • Without docs: 10 hours per week lost to context rebuilding. That's 520 hours per year, or about 13 full work weeks.
  • With docs: 1.5 hours per week on documentation. That's 78 hours per year, or about 2 work weeks.
  • Net savings: 442 hours per year.

    That's 11 weeks of recovered productivity.

I'm not even counting the crash scenario. Eight hours tracing a bug that took twenty minutes to fix. Multiply that by the number of times it happens per month, and the numbers get ugly fast.

But the thing that really matters. The cost of not documenting isn't just time. It's decisions you can't reverse. See also why logging everything matters for a complementary approach to operational discipline.

When I moved the personalizer module and broke the template engine, I had no record of why the config path was set up that way. No record of the fallback behavior. No record of the dependency chain.

If that system had been serving paying customers, the blast radius would have been massive.

Documentation is insurance. Cheap insurance that you buy with time instead of money. And like all insurance, you're grateful for it exactly when you need it most.

How I built my documentation system

The actual system I use, without the philosophy. Concrete files, concrete rules.

MEMORY.md, the brain

This is the index file. It lives at the root of the project and gets loaded at session start. It's capped at about 4 KB, which forces me to be concise.

Each entry is a topic with a pointer to where the details live. Think of it as a search engine index, not an encyclopedia.

Example entry: "Lead scoring v2, components: pain signal quality, profile strength, contactability, content relevance. Details in shared/lead_scorer.py and memory/2026-03-21.md"

That's it. Enough to know what exists and where to look. When I need depth, I follow the pointer.

BOOT.md, the context loader

This one is about operational state. What agents are running. What campaigns are active.

What model each agent uses and why. Current infrastructure details. Server credentials (encrypted or referenced, never plaintext in the doc itself).

BOOT.md is the file I read first every morning. It tells me where I left off yesterday, what's in progress, and what needs attention. It's the bridge between sessions.

Daily logs, the raw truth

Every day gets a memory/YYYY-MM-DD.md file. These are messy, unstructured, and honest. I log decisions, bugs, wins, things I learned, things that frustrated me.

No formatting rules, no structure requirements. Just write. I wrote more about why this practice is non-negotiable in LOG EVERYTHING.

I'm Begging You.

The daily files overflow at 20K characters. When they do, I split them by topic. The rule is simple, capture everything now, distill later.

Architecture docs, the system map

These are the formal documents. Data flow diagrams (I use Mermaid, it's fast and renders in GitHub). Module dependency maps.

Config schemas. API endpoint documentation. Deployment procedures.

I update these whenever I change the architecture. Not after. Not during a sprint review.

When I change it, I update the doc. The code and the docs move together or not at all.

The rules

  • Write it now, not later. Later never comes.
  • One source per topic. If information exists in two places, one of them is wrong.
  • Read your docs at session start.

    Every time. No exceptions.
  • Update when things change. Stale docs are worse than no docs.
  • Keep MEMORY.md under 5 KB.

    If it grows, prune and archive.
  • Write in your own voice. Corporate docs die because nobody relates to them.

Better to lose a project than lose everything

There's a version of this story where I didn't learn my lesson. Where I kept running without docs, kept losing hours to context rebuilding, kept getting lucky that the crashes weren't worse.

That version ends badly. It ends with a system too complex to maintain, too undocumented to hand off, and too fragile to scale. It ends with burnout, not because the work is hard, but because the overhead of remembering everything is exhausting.

I chose differently. I chose to be obsessive about documentation. To treat it as non-negotiable, not as overhead.

To accept that I'll sometimes lose a few hours documenting instead of building, and to trust that those hours come back tenfold.

If you're a solo founder building something complex, something with moving parts and dependencies and config files and agent pipelines and systems that talk to other systems, write it down. Write it down today. Not tomorrow.

Not when the feature is done. Now. If you want to see how I structured memory for my AI agents specifically, check out My AI Agents Have Memory Files.

Your future self will thank you. Probably Monday morning, probably with coffee in hand, probably wondering what that config path does. And then they'll open BOOT.md, find the answer in thirty seconds, and get back to building.

That's the whole point. That's why you document. Not because it's best practice.

Not because some blog post told you to. Because it's the difference between spending your time building and spending your time remembering. If your system is already fragile, get a workflow diagnosis. See our case studies for real-world examples.

01 CAPTURE Input Signal Structured payload 02 VALIDATE Rules + Schema Deduplicate & check 03 EXECUTE Agent / API Scoped action loop 04 VERIFY Audit & Gate Human checkpoint
Figure 1. Production execution spine for I Had No Documentation. Then Everything Crashed. (Operations).
Operational layerFragile defaultProduction architecture
Process designAutomating a broken manual habit as-isMapping the data contract and failure modes before writing code
Human oversightEither 100% manual or unguarded autopilotAutonomous execution on routine cases with approval gates on exceptions
Data custodyOperations locked inside third-party SaaSCode, prompts, logs, and API keys owned inside your accounts
Failure recoverySilent drops when an upstream API changesSchema validation, retries, and operator alerts on every break

FAQ

What is the first step to implementing i had no documentation. then everything crashed.?

Start by mapping the single workflow that loses the most hours each week, defining the exact input and output data contract, and placing human approval gates before writing code.

How do you measure whether an AI workflow is working in production?

Track time-to-completion, exception queue rate, duplicate or failed writes, and the hours returned to operators each week compared to the manual baseline.

Can our team run the system after the engagement ends?

Yes. Every build includes architecture documentation, alerting, and live operator training so your team owns and runs the system without an ongoing retainer.

Send the broken workflow.

If your CRM, intake, document pipeline, API bridge, Zapier chain, Make scenario, GHL workflow or agentic system is leaking time or money, send me the broken path.

Open AI Workflow Repair Intake Browse the store

Stay in the loop.

Technical breakdowns, new system builds, and lessons from deploying autonomous AI. One email a week, direct to your inbox.

One email a week. Leave with one click.