A Terminal Agent That Kept Falling Over
This is a build log, not a launch announcement. I started Agen as a single Python script that let me talk to Gemini from my terminal and have it run shell commands on my behalf. It worked, in the sense that it occasionally did what I asked. Most of the time it either locked up the terminal or got rate-limited into oblivion. This post walks through what actually broke in that first version, what I changed to fix it, and what I'm still not entirely happy with.
Where It All Began:
The Limits of V1
The first prototype of Agen was a single monolithic Python script. It
proved a terminal-based AI assistant was worth building, but it was
fundamentally fragile. It relied on synchronous polling and raw shell
commands — when I asked it to parse a directory, it would literally
shell out to cat and grep and sit there
blocking on the output.
When you're bound by a 4-request-per-minute free-tier limit, synchronous polling is a death sentence.
I was running on Gemini's free tier, which meant strict rate limits.
V1 would enter a loop, repeatedly querying the API to check on a
long-running terminal command, and within a minute I'd get slammed
with a 429 Too Many Requests that crashed the whole
session. Worse, the UI was fully blocking — while the agent was
"thinking," I couldn't type anything else into the terminal. I lost a
few evenings just watching it hang.
The ReAct Engine
To fix V1's brittleness, I tore down the basic chat loop and rebuilt it around a LangChain ReAct (Reason + Act) loop. The agent stopped just generating text and started reasoning about the problem, forming a plan, and executing actual Python tools on my machine instead of shelling out blindly.
🧠 How ReAct Changes Things
Instead of asking the LLM to write code and hoping for the best, the
ReAct loop forces the model to output a specific JSON schema
indicating a tool call. The Python backend
intercepts this, runs the matching native function (like
read_file or search_web), and feeds the
exact result back into the prompt. The model then reasons about that
output before deciding on its next action.
"Fix this bug"
Observes & Plans
Reads files, Greps
Writes fixed code
⚠️ Where it got annoying
The ReAct loop is only as reliable as the model's ability to stick to the JSON schema. Early on, Gemini would occasionally wrap its tool call in explanatory prose, and my parser would choke on it. I ended up writing defensive parsing around every tool-call response before this felt trustworthy enough to build on.
The Swarm Architecture
The part of V2 I'm most proud of is the multi-agent swarm. As projects got bigger, a single agent couldn't hold the context of a frontend component, a backend server, and a database schema all at once — it would either forget earlier context or mix concerns across files.
Using Python's asyncio.Task, I gave Agen an
invoke_subagent tool. The main agent acts as a manager:
when it hits a complex task, it spawns an independent, asynchronous
clone of itself, hands it a specific instruction, and lets it run in
the background.
Because subagents run on the async event loop rather than a while-loop poller, they don't block the UI. That's what makes it possible to have one agent writing tests while another watches server logs, both feeding results back to the manager.
Agent Manager
Oversees the swarm, delegating tasks and merging asynchronous results back into the main terminal conversation.
Async Tasks
Background workers that handle heavy work like semantic RAG chunking or web scraping without freezing the terminal UI.
SSE Streams
Server-Sent Events push tokens and tool events back to the client as they happen, instead of polling for status.
Local LLM Support
Run entirely offline and keep your codebase private by switching to local hardware inference via Ollama (e.g., Llama 3, Gemma).
⚠️ Where it got annoying
Subagents sharing state through a message queue sounds clean until two of them try to write to the same file at once. I don't have a real locking mechanism yet — right now I just try to scope subagent instructions narrowly enough that collisions are unlikely, which is a workaround, not a fix. It's the part of the swarm I trust least.
MCP & Semantic RAG
An agent that can't see your code isn't very useful. V1 relied on crude string matching. In V2, I integrated a local ChromaDB vector database instead.
Running /index makes Agen recursively parse the current
directory, generate embeddings locally, and store them in ChromaDB.
When I ask a question, it runs a similarity search and pulls back the
exact relevant lines across hundreds of files, without trying to stuff
the whole codebase into the context window.
🔌 The Model Context Protocol (MCP)
Rather than hardcoding every possible tool — a web scraper, a GitHub client, and so on — I adopted the open MCP standard. Agen V2 can connect to external MCP servers at runtime: if the agent decides it needs to search the web, it connects to a local Tavily MCP server, fetches the tool schema, and calls it on the fly.
The PyInstaller Crucible
Writing the software was only half the work. Packaging a modern AI stack — FastAPI, LangChain, Uvicorn, ChromaDB, Rich, and Prompt Toolkit — into a single Windows executable turned out to be its own project.
PyInstaller struggles with the dynamic imports AI libraries love to
use. I had to manually trace hidden imports for
pylatexenc, add hook paths for
langchain-community, and explicitly exclude frameworks
like Tkinter to keep the binary size down.
> pyinstaller --name "agen" --onefile client/cli.py
INFO: Analyzing hidden imports...
INFO: Building PKG (CArchive) agen.pkg completed successfully.
INFO: Bootloader Windows-64bit-intel/run.exe
INFO: Build complete! The results are available in: dist/agen.exe
170MB. One file. No Python installation required on the host machine.
The end result is a standalone agen.exe. You can drop it
onto a fresh Windows machine with no Python installed, double-click
it, and get a swarm-capable terminal agent running immediately. The
macOS and Linux paths still go through the installer script rather
than a packaged binary — that's next on the list.
⚠️ Where it got annoying
The hidden-import chase took longer than writing the swarm code did. There's no clean way to know which import PyInstaller missed until the packaged binary crashes on a machine that isn't yours — the feedback loop is slow and mostly guesswork.
Join the Swarm:
Contributing to V2 & V3
Agen isn't a walled garden. It's an open experiment in making terminal agents actually useful, and I'd love for you to get your hands dirty with the codebase. Whether you're squashing bugs in documentation, building out new features, or just testing the absolute limits of the system — there's a place for you here.
Our current prime directive? Battle-testing V2 across every platform we can find. If you have a weird Linux distro or an M-series Mac, I want to know exactly how spectacularly it breaks (or succeeds) so we can bulletproof the installation scripts.
Platform Testing
Run V2 on macOS, Linux, and Windows edge cases. Find the friction points and help smooth out the installation scripts.
Feature Engineering
Have an idea for a new MCP tool? A better way to resolve file conflicts between subagents? Fork it, build it, and open a PR.
The V3 Roadmap
V2 was about stability. V3 is about scale. Jump into the discussions, share your wildest ideas, and help architect the next generation of the swarm.
Autonomous Onboarding
Using an AI agent to contribute? Point it at our comprehensive .agent/ directory. We've equipped the repo with 14 strict skills covering V3 architecture, UI design, and testing.
The Bottom Line
Terminal agents are no longer just a gimmick — they are the next evolution of developer velocity. V2 fixed the problems that made V1 unusable — the blocking UI, the rate-limit crashes, and the reliance on raw shell commands. But it's not a finished corporate product. The swarm still needs conflict resolution when subagents fight over files, and the ReAct parser is still more defensive than I'd like.
If any of this is useful to you, the repo is open for issues and PRs. If you build something amazing on top of it, or if you hit a wall I didn't mention here, I'd genuinely like to hear about it.
Reach out to discuss new ideas for V3:
aman@mail.com
The code is out there. The question is whether you're ready to start delegating to the swarm.