My Agent Phase, and Why I Got Out

I've built a few LLM agent systems now and watched plenty more fail in real time. Here's where I landed.

LLMs might be the closest thing we have to a general intelligence, and they're probably the most important technology since the internet. But the products that actually work right now are ChatGPT and Copilot. A chatbot and an autocomplete. Neither can carry out a complex task. You hit the context wall fast. An LLM is like a tireless genius with the working memory of a goldfish. It forgets everything after about ten minutes.

So agents. String together a bunch of LLM calls with some orchestration logic, and suddenly the goldfish-genius can use external tools, remember things, plan ahead, execute across multiple turns. It's a reasonable idea. The architecture everyone reaches for looks a lot like the cognitive architectures from 80s AI research. Soar, ACT-R. An agenda of tasks, a long-term memory (vector database), a retrieval layer. Programmers reinvented cognitive science from scratch, often without knowing the field existed.

And yet.

There isn't a single software development agent you can download, point at a repo, describe a change to, and come back to an hour later for a pull request you'd actually merge. Codex, GitHub's Copilot coding agent, Devin and Jules will all hand you a PR now. AutoGPT, babyagi and gpt-engineer were doing the same trick worse a couple of years earlier. I keep trying these things and they never do anything actually impressive on a real task. The demos are always cherry-picked. The failure modes are predictable and punishing.

This stuff matters. Copilot makes me faster, but it's a faster horse. I still write the code, I just write it faster. An agent that could write code on my behalf would let me move up the value chain completely. So if agents can work in principle, why they don't work yet matters a lot.

Who Actually Builds Agents

There's an asymmetry I don't see people talk about enough.

If you have the resources to pretrain your own LLM, you aren't building an agent with tool loops and RAG pipelines. You're doing something like Yann LeCun's differentiable neural architectures. The model architecture is the agent. Or at minimum you're designing training data so the model knows it will be part of a larger system.

If you're a regular developer with an API key, you can't innovate at the model layer. You innovate at the API layer. You wrap the LLM in orchestration code, build a cognitive architecture around it, and hope prompt engineering bridges the gap. The people most qualified to solve the agent problem are not the ones building agents the way most of us build them.

The Generality Trap

A lot of agent builders make the same mistake.

You use ChatGPT and see that a single completion can do almost anything. Write poetry, write code, explain quantum mechanics, extract LaTeX from a photo. Incredibly general. So you think: I want to build an agent that's equally general. I should build a general architecture.

And you do. You draw the diagram from every cognitive science paper. A central executive, long-term memory, short-term memory, sensors, effectors (function calls). Very general, very abstract. Underconstrained in every direction.

These architectures never work.

Compare Copilot. There's a huge software layer that converts file context and repo context into a structured prompt. It doesn't have a general-purpose vector database for RAG, or a tree of recursively broken-down subtasks with priority weighting. It's specific. It's overconstrained. It cannot write a sonnet. But it makes me maybe 40% more productive at writing code. Because it does the one thing.

Maybe LLMs work best as what I call "magic function calls." Performing a narrow, concrete, supervised task. Not writing a program. Writing a single function.

Three Ways They Fail

Watch the transcripts of any LLM agent failing at a task. It's almost always one of three patterns:

1. The loop. The agent goes in circles, revisiting the same goals, never making forward progress. It successfully retrieves the same information and arrives at the same conclusion four times before running out of steps.

2. The procrastination recursion. The agent gets "lazy." It takes a high-level goal, breaks it into subgoals, and one of the subgoals is just "do the work." When it gets to that subgoal, it breaks it down again. "Do the work" spawns "actually do the work." Infinite recursion of avoidance.

3. The wanderer. The agent doesn't loop. It just drifts. Meanders, talks to itself, accomplishes nothing specific. Looks busy. Nothing converges.

I kept coming back to this and realized it maps almost perfectly onto Wolfram's complexity classes of cellular automata. Within a single completion, LLMs give you Class IV behavior. Complex, coherent output. But string them into an agent and most systems degenerate into Class I (nothing), Class II (looping), or Class III (chaotic). Something that should be sustained and structured collapses.

The Entropy Argument

There's another angle I find compelling, even if it's hand-wavy.

LLM-generated text has measurably lower entropy than human-written text. The model just produces less varied outputs than a human would. So if you point a thousand low-entropy outputs at a single goal, can they ever reach criticality?

Here's the analogy I heard: if you have a lamp running at 90 degrees, you can point a thousand of them at an object and it will never reach 100 degrees. The individual outputs don't compound the way you need them to.

I don't know if this is a real physical limit or just a useful metaphor. But the pattern keeps showing up. The more steps you string together, the more the system regresses toward the mean, and the mean is not good enough.

When Agents Do Work

Narrow use cases, with massive software scaffolding.

RAG pipelines that retrieve from a known corpus and answer questions about it. Those work, because the task is constrained. Code generation with RepoCoder-style iterative retrieval. Works, because the feedback is concrete. Does it compile? Does it pass tests? Customer support bots with strict escalation rules. Work, because the failure modes are bounded.

The common thread: the LLM is a component in a larger system where most of the intelligence comes from the classical software. It never has to act as a general agent. It fills in specific gaps. The borders are tight.

Five Explanations (Pick Your Favorite)

I don't know the real answer. But here's what I think might be going on.

1. Fundamental impossibility. LLM agents are like perpetual motion machines. There's some yet-undiscovered theorem in comparative psychology or information theory that says you can't get sustained complex behavior by chaining statistical language models. This feels plausible but I can't prove it.

2. Models aren't good enough yet. The current generation just can't sustain the coherence needed for multi-step work. GPT-5 or whatever comes next might cross the threshold. Optimistic take.

3. General architectures are the wrong target. The generality trap I described above. Agents should be narrow, specific, overconstrained. More like Copilot, less like a cognitive science diagram. The architecture should do the heavy lifting. The LLM shouldn't have to.

4. We need way more complexity. The simple architectures (a handful of modules, prompt templates, vector database) are too simple. Real complex behavior needs messy, biology-level architecture. Specialized sub-systems, redundant pathways, error correction at every layer. Less elegant, more alive.

5. The LLM-classical boundary is the problem. The seam between what the LLM does and what the surrounding software does is where things break. The solution is to train the whole thing end-to-end. Make the agent itself differentiable. Current agent architectures have a brick wall between "model reasoning" and "tool execution." Information gets lost crossing it.

What I'm Actually Doing About It

I've stopped trying to build general-purpose agents. I use LLMs as components in conventional software. Narrow pipes with tight contracts. If I need a system to do something complex, I decompose it into small, concrete steps where each step is a simple LLM call with strict supervision, and I put most of the intelligence in the orchestration layer.

The dream of "point it at a repo and come back to a PR worth merging" is not dead. But I think we'll get there through thousands of narrow, boring integrations rather than one elegant cognitive architecture. The diagrams will look less like neuroscience and more like plumbing.

And that's honestly fine.

References

1. Park et al., "Generative Agents: Interactive Simulacra of Human Behavior." arXiv:2304.03442.
2. RepoCoder: Repository-Level Code Completion Through Iterative Retrieval and Generation. arXiv:2303.12570.
3. Langton, C. (1986). "Studying Artificial Life with Cellular Automata."
4. "Integrated Models of Cognitive Systems." Oxford University Press.
5. Weng, L. (2023). "LLM Powered Autonomous Agents."
6. Wolfram, S. "Universality and Complexity in Cellular Automata."