Build Your Own AI CLI: The Terminal Agent I Run

TL;DR. You can build your own AI CLI in a weekend if you pick one workflow you repeat every week, wrap it in typed tools that sit on plain code you can test without a model, and ship it as a single binary with a one line installer. I did exactly that for my own AI enablement work, and the thing now scouts clients, writes the outreach, holds an approval gate before anything sends, and keeps the records in a database I open myself. This is the honest version of that build, with the mistakes in it.

Some numbers from the machine I use every day, before you decide whether to keep reading. It ships as one binary with 26 tools, it has 24 unit tests around the parts that touch data, it builds for four platforms on every release, and installing it takes one command that finishes with a browser tab open on the CRM.

Why your agent needs a home of its own

An agent living inside a chat window borrows everything from that window. Context fades between sessions, files end up wherever the conversation left them, and two days later you are rereading a transcript to work out what happened. I got tired of that somewhere around week two.

A CLI gives the agent somewhere to live. It trusts one working directory and keeps a session store that survives restarts. I wrote the permission policy myself, and the tools do the specific jobs my business runs on. I used to do all of it by hand every week, from scouting a business to writing the handover document at the end of the job, and my practice sells Build your system to other people, so it felt strange to run mine out of a browser tab.

Low code automation tools got my first workflows out the door and I still recommend them for a first pass. I kept the automation the week the work turned into client facing delivery with real invoices attached, and moved it onto a machine I own end to end.

Fork a base or build from zero

You have two honest options and the choice shapes everything after it.

Building from zero is right when the tool stays small, say a few flags, one API and one output format. You own every line of it. You also own argument parsing, config loading, update mechanics, and the day a second surface wants access to the same state.

Forking an agent runtime is what I did. Mine is a fork of an open source terminal agent that I reworked for one job. The codebase splits the terminal from the browser and the desktop shell, keeps the session engine and the model layer underneath them, and exposes a plugin API where I wanted to make changes. I swapped her identity and defaults, replaced the tool surface entirely, and rebuilt the release machinery while keeping the engine underneath. Those pieces carry years of work in streaming, tool calls, permission gates and session recovery, and I would rather inherit all of it than write it again.

I would fork again, and I wrote up why in more depth in my notes on building a custom agent shell. The short version is that your product is what sits on top of the engine, and the engine is the boring part you want somebody else to have maintained.

Python, Rust or TypeScript, and what each one costs you

Python gets a prototype running by lunch and then hands you a packaging story you will fight for the rest of the year. Rust hands you one static binary and a compiler that argues with you all week. I chose TypeScript on Bun because one language covers the whole thing and one test runner goes with it, which matters more to me than raw startup time.

Whichever one you pick, settle the packaging story on day one. A tool people install with curl and bash needs a real binary and a real installer, while a tool people install with pipx needs a wheel you can rebuild at 2am with cold hands. I have done both and the second one taught me to decide early.

One server, and every surface speaks to it

My rule, which I have kept through every version, is that one process owns sessions, tools, models and configuration. Every surface talks to that one process, so a record written in one place shows up in the others and there is no sync code anywhere. It works the same from the terminal and from the CRM dashboard in a browser.

That decision is also how chat gets into a browser later without a rewrite, because the engine already streams. You send one input and listen to the event stream.

POST /api/session/:id/prompt # admit one message, schedule the run
GET /api/event # server sent events, live output

Authentication stays deliberately boring. One password covers local use, and a signed session token from a pairing link lets a browser stay signed in for thirty days. If you want the full shape of that, I wrote it up in AI agent tool calling architecture.

Typed tools over plain modules

The architecture in one sentence is this. Tools stay thin, modules stay plain, and the model calls a tool that calls a module which has no idea a model exists anywhere nearby.

// plain TypeScript: testable, no model in sight
export function addClient(input: ClientInput) {
 const existing = findByWebsite(input.website)?? findByName(input.name, input.address)
 if (existing) return { client: existing, created: false }
 return { client: insert(input), created: true }
}

The tool wrapper around that function is twenty lines of JSON schema and a passthrough. That split is why I can test the CRM, the outreach cadence and the delivery loop without spending a token, and it is why the browser dashboard stays cheap to build. It calls the same module the terminal calls.

The tool set has grown to 26 and covers the work in four groups. The memory group stores and recalls what she learns, the health group checks and repairs her own state, and the signals group reads the local market radar. An operator group does the business work from the first scout through to the handover document, so it keeps CRM records current, enriches them from the client's own website, drafts the outreach and drives the delivery. I keep that boundary hard because of a lesson from running five autonomous agents at once, where every shortcut in the tool layer turned into an incident later.

State you can read with sqlite3

Two SQLite files, on purpose. One holds memory, where whatever she learns is typed up and decays on a schedule, whatever shape it arrived in. The other holds clients, interactions, assets and the outreach pipeline. Client records stay out of the memory index entirely, which means I can back one file up and wipe the other without thinking about what connects them.

Plain files beat a hidden brain for one reason. When something goes strange I can open the state and read it. Personas and playbooks are markdown and JSON in a folder any editor opens, the agent reads them on every turn, and she writes her own skills back into the same folder. If you want the full layout I wrote it up in AI agent memory architecture, and the file discipline in AGENTS.md as memory.

The gates that keep me honest

I run three checks before anything leaves the machine and they run in a fixed order. The first is lint and types through a git hook, so a push cannot even start with a red check. The second is unit tests against real modules, twenty four of them in the brain package, running without any model calls. The third is a smoke script that boots the plugin inside a throwaway home directory and walks the entire loop, covering memory, scouting, CRM, outreach, the follow up cadence, delivery and the handover document.

That smoke script is the one that earns its place. Last week it caught a defect my type checker waved through with no complaint at all. An insert statement was listing 29 columns against 28 placeholders, so every attempt to add a client on the live machine came back with 28 values for 29 columns while the unit tests stayed green, because the SQL lives in a template string and the parameter array sits next to it where the type system cannot see the relationship.

const columns = ["name", "vertical", "status", /* … */, "created_at", "updated_at"]
db.run(
 `INSERT INTO clients (${columns.join(", ")}) VALUES (${columns.map(() => "?").join(", ")})`,
 values,
)

I build by that rule now. Generate the placeholders from the column list so the two cannot drift apart, and keep one smoke path that exercises the write you care about. If your CLI touches a database, that smoke run belongs in your release checklist.

Ship it like software

A CLI earns trust when it updates itself cleanly, so treat it like any other piece of shipped software. One tag triggers the builds, and every platform publishes its own artifact.

on:
 push:
 tags: ["v*"]
jobs:
 build:
 strategy:
 matrix:
 settings:
 - { name: linux-x64, host: ubuntu-latest }
 - { name: linux-arm64, host: ubuntu-latest }
 - { name: darwin-x64, host: macos-latest }
 - { name: darwin-arm64, host: macos-latest }

Two habits worth stealing here. Publish each target to the release the moment it compiles, because the macOS runners sit in a small pool and a single publish step at the end makes the whole release wait for the slowest build. And write the changelog section before you tag, while the commits still make sense in your head.

The installer is one line, and it replaces its own install directory every time it runs. That is the design, and it sets a bar for repo hygiene because whatever you commit and release is what survives the next upgrade. Mine lives at elitza.life with the source in the public repo, and the deployment habits carry to whatever host you pick. I run agents on Hetzner VPS boxes when they need to stay up all week.

Grow a UI when the loop earns one

Months in, the approval queue deserved a screen of its own. I added a dashboard that the plugin itself serves on 127.0.0.1:8787. The pipeline shows up as a board, the approval queue gets its own view, and a client record carries clickable contact details along with every social profile. It boots when the engine boots and stops when the engine stops.

One rule kept the cost of that down. The dashboard calls the same modules the terminal calls through a small JSON API, so there is one writer for every record and the surfaces cannot drift apart. That principle shows up in every build I hand to a client, from onboarding systems to CRM rebuilds.

What mine runs today

One loop, end to end, on real businesses in my town. It scouts, and a town and a trade turn into a list of businesses with names, phone numbers, websites and ratings. It qualifies what it finds by scoring each business from 0 to 100, deduplicating against what I already have, and reading the business's own site for a public email address.

Then it enriches the good ones, pulling contacts and social profiles off their website and writing a diagnosis and a scoped proposal over my playbooks. Outreach goes out as a draft file with the compliance footer already in it, waits for me to approve a batch, and only then sends through my provider, after which the 3, 7 and 14 day cadence keeps the thread warm. When a client says yes, the same machine moves them through proposal, call, payment and a handover document generated from the work.

That is the agentic AI implementation I sell, running as the product. If you want to start smaller, AI lead qualification automation is the piece most people begin with, and the operator view of the economics sits in building a business with zero employees. For where all of this is heading, see every serious operator will have a personal AI agent, and for the CRM side of the same loop, the automated lead routing build.

The release checklist I run every time

Print this one. It is the exact order I use before a version goes out and every line earned its place the hard way.

1. Keep types and lint green, and let the git hook enforce it before anything leaves the machine.
2. Run the unit tests against real modules, which in my case is twenty four of them in the brain package.
3. Run the smoke script against a throwaway home directory so the whole loop gets exercised in isolation.
4. Write the changelog section while the commits still make sense in your head.
5. Push the tag and let every platform publish its own artifact as soon as it compiles.
6. Open the release tarball before anybody installs it, and confirm the files you changed are inside.
7. Run the installer once on a clean machine and check that the uninstall path works too.
8. Complete one real workflow end to end with a live client, including the approval click and the money.

Where to go next

When you want the parts without building them, I packaged the same machine into a few things. The business AI agent build kit carries the scoping, the build plan and the launch checklist for putting an agent to work inside a real business

Two free kits are the cheapest way to see whether any of this pays for you. The workflow audit kit maps a messy process in one sitting, and the lead routing kit covers the routing rules and the handover contract for leads that arrive faster than you can answer. The routing contract builder writes that contract for you in the browser, and the full store has repair kits and spec packs for operators building on top of all this.

Questions I get asked about building your own CLI