The AI-pilled Software Factory

AI-pilled is not a vibe. It’s a factory.

The most AI-pilled companies aren’t simply adopting AI or making their existing employees more productive. They are building software factories.

I have spent the past nine months studying over 30 companies and 50 individuals, from three-person startups to large public companies, trying to understand what separates individual AI use from a company that has actually reorganized itself around AI. That work originally led me to an L0–L5 maturity model based on four questions: What can AI see? What can it do? Who can extend it? And how has the organization changed?

But after spending more time with the companies at the frontier, I think there is a bigger idea hiding inside the maturity model.

The mistake is to confuse an AI-enabled workforce with an AI-native company. Giving everyone access to Claude, ChatGPT, or Copilot can make employees dramatically more productive. It does not, by itself, change what the organization is capable of.

The engineering organizations have gone far beyond coding faster with more agents. These companies are creating the infrastructure that allows the entire company to continuously build software. Marketing can build. Sales can build. Finance can build. A product manager can identify a problem, create an agent to solve it, and propagate that capability to every other product manager.

That changes the role of engineering. In a traditional software company, the engineering organization largely built the software product. Increasingly, we are seeing engineering teams build the harness that gives the rest of the organization the context, permissions, evaluations, model routing, and guardrails they need to safely create software themselves.

Ramp Glass and Ramp Inspect are good examples of what this looks like in practice. Glass functions as the context layer and everyday agent: employees across the company can talk to it, it understands the context of their work, and they can use it to create agents of their own. Inspect is the execution layer: a background coding agent equipped with Ramp’s development environment, tools, and company-specific standards so that what employees want to build can become production-quality software.

What does the factory actually unlock?

The L0–L5 model tells me something important about the architecture of a company: how much context AI has, how much agency it has, who can extend it, and how deeply the organization has changed around it. But it doesn’t necessarily tell me how much value that architecture is producing. A company can have agents everywhere and still accomplish very little.

This is where I think many companies are measuring the wrong thing. AI adoption is relatively easy to measure: seats, tokens, agents, AI-generated code. What matters is whether any of that has created a capability the organization did not have before.

That has led me to a different set of questions. The Software Factory Test comes down to five questions:

See:  What becomes visible?

Build: Who can create software?

Propagate: Does what they build spread?

Remember: Does experience accumulate?

Notice: What does the machine catch on its own?

These are the capabilities the factory is supposed to produce.

See

One product manager we spoke to described giving her AI access to her product documentation, usage data, future roadmap, strategy, and the company’s customer conversations in Gong. Previously, she depended on the sales organization to tell her what customers were saying about her product. Inevitably, that meant getting anecdotes: a salesperson would hear something important and remember to pass it along.

Once the AI could listen to every call, find every reference to her product, and connect those conversations with the rest of her context, she could get a complete picture of what customers were actually saying. That picture was sufficiently different from the one she had before that in some cases it changed her product roadmap and in others influenced her product line strategy.

This is why the ability to see is much more powerful than simply “better search.” A human product manager cannot listen to thousands of Gong calls, remember every customer comment, connect those comments to every piece of product telemetry, and simultaneously hold the company’s strategy in her head. The machine can. The employee is not merely faster at gathering information. She has access to a picture of the business that she literally could not assemble before.

Build

One salesperson I spoke with showed me a simple example. When she contacted an account, other salespeople were not supposed to contact that account for thirty days. Historically, she could have complained about violations or convinced someone in engineering or sales operations to build a solution.

Instead, she built an agent herself. The agent detects when another salesperson contacts one of her protected accounts, alerts her, and gives her a way to alert the other salesperson.

The breakthrough isn’t that AI allowed an engineer to write that software ten times faster. It is that the person who understood the problem could now create the software to solve it.

This changes the economics of software inside a company. When software was expensive to create, organizations needed mechanisms for deciding which problems deserved scarce engineering resources. A salesperson might recognize a small problem every day, but almost none of those problems could justify an engineering ticket. A marketer might have an idea for a useful internal tool but never even articulate it because she knew it would not be built.

When the cost of creating software collapses and the person closest to the problem can build against company context directly, thousands of previously uneconomic pieces of software become viable. The bottleneck migrates from the ability to build and toward the ability to recognize which problems are worth solving.

Vercel’s internal data system, d0, provides another window into this transition. d0 allows anyone at Vercel to ask questions of the company's data warehouse in natural language. Employees now use it to ask tens of thousands of questions each month.

The more interesting story is what happened next. Once employees experienced how easy it was to interrogate company data themselves, asking questions began to turn into building. What started as a way to make data visible became part of a broader proliferation of internal agents and applications across the company including finance and product.

The ability to see created a desire to build. And a community building created more builders.

Propagate

Earlier, I described the product manager who had given her AI enough context to develop a much richer picture of her customers. She then took the system she had created for herself and packaged it into a reusable skill so that other PMs could have the same capability without reconstructing everything she had done.

Building turns an employee into a software creator but propagation turns what one employee creates into an organizational capability.

Companies have always tried to spread the skills of their best people. The exceptional salesperson teaches the rest of the team how she prepares for meetings. The best product manager writes a playbook. Managers coach people. Companies create training programs. But all of these are lossy ways of transferring tacit knowledge.

In a software factory, some portion of what the exceptional person does can become executable. The best product manager’s research process becomes a skill. The best salesperson’s account preparation becomes an agent. The best marketer’s analysis becomes a workflow. Instead of merely documenting excellence, you can begin to distribute it.

The gap between the best person on a team and the median person is often enormous. Companies have spent decades trying to close that gap through hiring, training, management, and process. AI gives them another mechanism: encode some portion of what the best people do and make it available at the moment everyone else needs it.

Companies have always been able to accumulate knowledge. A software factory gives them a new way to accumulate capability and potentially compound it.

Remember

Remember is easy to confuse with See, but the distinction is important. See is about making what a company already knows accessible. Remember is about whether experience changes what the company knows next.

Companies are surprisingly bad at this. A team tests something and discovers it doesn’t work. Six months later, another team tries essentially the same thing. A strategy changes, but the reasoning behind the change lives in the heads of the three people who were in the room. An agent takes a bad path, a human corrects it, and the next time the agent encounters the same situation it makes essentially the same mistake.

The most sophisticated system I have seen does something different. It records what it got wrong, what it believed and then believed less, what evidence caused it to change its mind, and which approaches failed. That record isn’t simply an audit trail. It becomes an input into the next attempt.

This distinction matters enormously. You can give an AI access to every Slack message, Gong call, document, and database in the company and still have a system that does not develop judgment. It may be able to retrieve the postmortem from six months ago, but that is different from having incorporated the lesson from that postmortem into how it behaves today.

See gives the machine access to the company’s memory. Remember allows experience to change that memory, enabling the company to become smarter.

Notice

The final capability feels most different from how companies have traditionally operated because the human no longer has to initiate the work.

One company connected its AI systems to Snowflake, Postgres, meeting transcripts, town halls, and other sources of company context. During one town hall, a product manager made a passing comment that perhaps shorter job shifts would be accepted more frequently within the marketplace they operate. He did not ask the AI to investigate it.

The system noticed the hypothesis, went into the company’s data, tested whether it was true, concluded that it was, built the product, and wrote a memo explaining what it had found and what it had done. Only then did it bring the entire package back to the team for approval.

That is a qualitatively different capability. Much of management is not actually making decisions; it is noticing what deserves a decision. Someone has to see that a metric moved, remember a comment from last week’s meeting, realize that the two might be connected, assign someone to investigate it, and remember to check back later.

When the machine can notice, some of that burden disappears. It can continuously watch the company, surface anomalies and hypotheses, investigate them, and bring back work that deserves human judgment. Humans still decide. Every consequential action in the most advanced system I observed requires explicit human approval. But humans no longer need to personally notice every potentially important thing that deserves a decision.

Taken together, these five capabilities attack five very old constraints on organizations: no person can know everything the company knows; excellence is hard to transfer; software creation is bottlenecked by technical talent; organizations repeatedly forget what they learn; and humans cannot continuously pay attention to everything.

But there is an important trap here. Building more sophisticated AI infrastructure does not necessarily mean you have unlocked more value. A company can have proactive agents everywhere and still be surprisingly bad at seeing, propagating, building, remembering, and noticing.

ROI rot

I have started to think of that problem as ROI rot.

The company has agents everywhere. It consumes an enormous number of tokens. Employees generate analyses constantly. Automations are firing. AI usage statistics look fantastic. Perhaps there are even demos that make the company look extraordinarily advanced.

But very little changes in the actual business. The analysis doesn’t alter a decision. The automation doesn’t remove meaningful work. The agent creates output nobody uses.

Many companies are likely accumulating AI debt: agents nobody uses, workflows nobody evaluates, generated software nobody maintains, and token consumption disconnected from business outcomes.

AI is particularly dangerous because the marginal cost of producing more activity has collapsed. We have never had a technology that makes it this easy to produce sophisticated-looking work nobody needs. The fact that something would have been expensive to produce three years ago does not make it valuable today.

A software factory optimized for output can produce waste at unprecedented scale. AI can make a poorly designed organization worse faster.

This gives us a simple way to think about ROI rot: if AI usage is exploding but your organization is not materially better at seeing, propagating, building, remembering, or noticing, what exactly are all those tokens buying?

A business is fundamentally a loop: observe something about the world, decide what to do, act, and then learn from the result. AI can collapse enormous portions of that loop. Observation that once required a data request can become instantaneous. Building something that once took months can take hours. A loop that once took a quarter can happen in days.

But making the loop spin faster does not guarantee that the company learns. If nothing records what worked, what failed, what the organization changed its mind about and why, every trip around the loop begins from roughly the same place. Consumption goes up, but capability does not compound.

A great factory is not measured by how fast its machines run. It is measured by whether it reliably produces something valuable, gets better at producing it, and stops producing things that don’t work.

Otherwise you have built a factory optimized for production rather than outcomes.

The Software Factory Test

So rather than asking a CEO how many agents the company has, how many tokens it is consuming, or how much code is AI-generated, I would ask:

See: What can your employees see today that they couldn’t see a year ago?

Build: Can the person closest to the problem build the solution?

Propagate: When one person discovers a better way to work, does everyone inherit it?

Remember: When the company learns something, does the system learn it too?

Notice: Can the machine notice something important nobody thought to ask?

The defining asset isn’t access to the best model. Everyone will have access to roughly the same frontier models. It isn’t the sheer number of agents either, because agents themselves are becoming cheap to create.

The durable asset is the factory: the context that makes the company legible, the infrastructure that lets employees build against that context, the mechanisms that turn individual excellence into shared capability, the memory that prevents the organization from repeatedly relearning the same lessons, and the evaluation and governance systems that determine which machine-generated work deserves to propagate.

We have spent the last fifty years building software companies. The next generation of great companies will increasingly be software factories: organizations that continuously turn their own context, judgment, and experience into software. The companies that learn to do this will not merely operate more efficiently. They will learn faster than their competitors and increasingly encode what they learn into the company itself.

If you are building the harness layer (context, permissions, evaluations, or tools that let non-engineers create software safely) or your company has the shape of a software factory, I want to hear from you.