Why We Didn't Use LangChain in Production (And What We Built Instead)

One of the questions I get asked most about PHHM is surprisingly consistent.

"Why didn't you build it with LangChain?"

Sometimes the question is about CrewAI. Sometimes it's AutoGen. Sometimes another orchestration framework. The assumption is always the same. If you're building a multi-agent AI platform, surely you start with an orchestration framework. We didn't. Not because those frameworks are bad. Quite the opposite.

They're impressive pieces of engineering. They solve real problems. We evaluated them carefully before writing a single line of orchestration code. And then we made what initially felt like the harder decision. We built our own orchestration layer using FastAPI, asyncio, and YAML-driven configuration. Looking back, I think that decision had more impact on PHHM's long-term architecture than almost any model or prompt we chose. Not because our solution was more sophisticated. Because it gave us ownership of the parts of the platform that mattered most.

This Was Never About Choosing a Winner

Before discussing architecture, it's worth making one thing clear. This wasn't a competition. We weren't trying to prove one framework was better than another. Every framework we evaluated solves a different problem. That's exactly why engineering decisions should start with requirements—not preferences. For PHHM, the requirements were unusually specific. We needed:

• deterministic workflow routing
• explicit orchestration
• YAML-configured agents
• strict validation
• workflow state ownership
• structured observability
• prompt version tracking
• role-based permissions
• predictable deployment behaviour

Those requirements became the benchmark. Not popularity. Not GitHub stars. Not social media discussions.

The Frameworks We Evaluated

Early in the project we spent time evaluating several orchestration approaches. Broadly, they fell into three categories.

Notice something important. None of these tools are "wrong." They're optimized for different engineering goals. The question wasn't:

"Which framework is best?"

It was:

"Which approach best matches the platform we're building?"

Prototypes and Platforms Have Different Constraints

One realization kept resurfacing during our evaluation. The requirements for a prototype are almost the opposite of the requirements for a production platform. A prototype optimizes for:

• speed
• experimentation
• iteration
• abstraction
• convenience

A production platform optimizes for:

• predictability
• observability
• debugging
• deployment
• operational control

Those priorities don't always point toward the same architecture. And that's where our decision started to take shape.

The First Architectural Question

Instead of asking:

"How can we orchestrate multiple agents?"

We started asking:

"Who owns orchestration?"

That single question influenced everything that followed. If orchestration lived inside a framework, many of our operational decisions would also live there. If orchestration lived inside PHHM, we could shape it around our workflows instead of adapting our workflows to someone else's abstractions. That distinction became increasingly important as the platform grew.

Building Versus Owning

This wasn't really a decision between using a library and writing code. It was a decision about ownership. Who owns:

• routing?
• retries?
• workflow state?
• validation?
• logging?
• versioning?
• permissions?
• deployment behaviour?

For PHHM, those weren't implementation details. They were core platform capabilities. Once we recognized that, the path became much clearer.

The Principle That Guided Everything

Looking back, one principle summarizes the entire evaluation.

Use frameworks for capabilities. Own the architecture that differentiates your platform.

That idea influenced every decision that followed.

One thing surprised me during our evaluation. Every framework worked. LangChain. CrewAI. AutoGen. Within a few hours, we could build a working multi-agent prototype using any of them. Requests flowed between agents. The models produced useful outputs.

Basic orchestration worked exactly as advertised. If our goal had been demonstrating an idea, we could have stopped there. But PHHM wasn't being built as a demo. It was being built as a platform. And platforms ask different questions.

Prototypes Optimize for Speed

Frameworks are excellent at reducing the amount of code you write. Instead of implementing orchestration yourself, you compose higher-level building blocks.

Application
↓
Framework
↓
Model
↓
Response

That dramatically shortens the path from idea to working software. For experimentation, that's exactly what you want.

Production Optimizes for Control

As PHHM grew, the questions changed. Instead of asking:

"Can we connect multiple agents?"

We started asking:

• Why did this workflow take seven seconds?
• Why did the Care Agent retry twice?
• Which prompt version generated this recommendation?
• Which validation rule rejected this response?
• Which YAML configuration was active?
• Why did this workflow choose one route instead of another?

Those questions aren't primarily about AI. They're about operating distributed software. That's where our requirements began to diverge from what orchestration frameworks were designed to solve.

Abstraction Is Both a Strength and a Cost

One engineering lesson became increasingly clear. Every abstraction hides complexity. That's its purpose. But hidden complexity also means hidden decisions. Consider this simplified comparison.

Application
↓
Framework
↓
Routing
↓
Agent
↓
Response

The framework decides how several pieces interact. That's incredibly convenient. Until you need one of those decisions to behave differently.

We Needed Deterministic Routing

One requirement influenced almost every architectural decision. The Overseer—not the framework—should decide exactly which agents participate in every workflow. For example:

User Request
↓
Overseer
├── Member Report
│      │
│      ├── Analyst
│      ├── Care
│      └── Communications
└── Newsletter
       │
       └── Communications

Routing wasn't something we wanted inferred. It was business logic. That meant we wanted complete ownership over it.

Hidden Behaviour Becomes Operational Risk

Another lesson appeared once we started debugging production workflows. Suppose an execution behaves unexpectedly. Where do you investigate? Inside your code? Inside the framework? Inside callbacks? Inside middleware? Inside internal execution chains?

Every additional abstraction layer becomes another place where behaviour can hide. That doesn't mean abstractions are bad. It simply changes the debugging experience. For PHHM, we wanted every routing decision to be visible inside our own codebase.

We Wanted Explicit State Management

Earlier in this series, we explored workflow state. That requirement also influenced our architecture. Instead of allowing conversational state to evolve implicitly, PHHM manages it explicitly.

Workflow State
↓
Orchestrator
↓
Agent Context
↓
Validation
↓
Updated State

Every transition is intentional. Every update is observable. Every change is versionable. That level of explicit control became much easier once orchestration belonged to the platform.

YAML Became the Source of Truth

Another requirement gradually emerged. Non-technical administrators needed to adjust agent behaviour without modifying Python code. Instead of hardcoding orchestration rules, we moved configuration into YAML. For example:

analyst:
  enabled: true
  model: gpt-4.1
  prompt_version: "2.5.0"

care:
  enabled: true
  depends_on:
    - analyst

The orchestrator interprets configuration. It doesn't contain business logic. That separation made deployments significantly easier to reason about.

Observability Needed First-Class Support

One of the recurring themes throughout the PHHM series is observability. Every workflow records:

• execution ID
• prompt version
• configuration version
• routing decisions
• validation outcomes
• retry history
• token usage

Those weren't optional features. They were foundational requirements. Owning the orchestration layer meant we could make observability part of the architecture instead of integrating it later.

Predictability Beat Flexibility

One phrase appeared repeatedly during architecture reviews.

"Can we predict exactly what this workflow will do?"

If the answer was:

"Usually."

We weren't satisfied. Production platforms benefit from predictable execution. Deterministic routing. Explicit state transitions. Versioned configuration. Clear validation. Those characteristics made troubleshooting significantly easier.

Frameworks Solved Different Problems

Looking back, I don't think the frameworks failed us. I think they optimized for different goals. They excel at:

• rapid experimentation
• proof-of-concepts
• developer productivity
• reusable integrations
• flexible agent interactions

PHHM optimized for:

• deterministic execution
• operational visibility
• workflow ownership
• reproducibility
• long-term maintainability

Those aren't competing priorities. They're different engineering objectives.

The Decision Became Obvious

Eventually the architectural question became very simple. Do we want to own:

• routing?
• validation?
• workflow state?
• observability?
• deployment?
• versioning?

Or do we want those behaviours to emerge through another abstraction layer? For PHHM, those capabilities defined the platform. That made the answer surprisingly straightforward. We built our own orchestration layer—not because frameworks were insufficient, but because orchestration itself had become part of our product.

The Bigger Lesson

Looking back, one engineering principle explains the entire decision.

The closer a component is to your competitive advantage, the more carefully you should consider owning it.

Authentication? Use proven libraries. HTTP serving? Use FastAPI. Database access? Use mature tooling. But orchestration was different. It embodied our business workflows, operational practices, and engineering philosophy.

That made it worth owning.

What We Built Instead: A Lightweight Orchestration Layer We Fully Own

Choosing not to use an orchestration framework meant accepting a responsibility. We now owned orchestration. That sounds like a lot of work. In practice, it turned out to be surprisingly small. One lesson became obvious very quickly. We didn't need a large orchestration framework. We needed a small orchestration engine that solved our problems extremely well. Instead of adapting our workflows to match a framework...

...we built a framework around our workflows.

The Architecture Is Intentionally Simple

At a high level, PHHM consists of a small number of well-defined components.

                Client
                  │
                  ▼
             FastAPI API
                  │
                  ▼
          Authentication
                  │
                  ▼
        Overseer (Router)
                  │
      ├───────────┼────────────┐
      ▼           ▼            ▼
  Analyst      Care     Communications
      │           │            │
      └───────────┼────────────┘
                  ▼
            Validation Layer
                  │
                  ▼
          Workflow State
                  │
                  ▼
          Final Response

Notice what's missing. There isn't another orchestration framework sitting between the API and the business logic. The platform owns the workflow directly.

One misconception is that orchestration requires a dedicated orchestration framework. For PHHM, FastAPI already provided much of what we needed. It handled:

• HTTP endpoints
• dependency injection
• authentication
• request validation
• lifecycle management
• asynchronous execution

Instead of adding another abstraction layer, we simply built orchestration on top of those primitives. That kept the architecture remarkably easy to understand.

Asyncio Did the Heavy Lifting

Earlier in the series we discussed parallel execution. That capability came directly from Python's asynchronous programming model.

tasks = [
    analyst.run(context),
    care.run(context),
    communications.run(context),
]

results = await asyncio.gather(*tasks)

There wasn't a complex orchestration engine deciding concurrency. Python already solved that problem elegantly. The orchestrator simply decided what should execute. asyncio decided how it executed. That separation kept responsibilities clear.

The Overseer Is Deliberately Small

One mistake we avoided was turning the Overseer into a giant decision engine. Its responsibilities remain intentionally limited.

Receive Request
↓
Identify Workflow
↓
Load Configuration
↓
Select Agents
↓
Execute Workflow
↓
Validate Output
↓
Return Response

That's it. The Overseer coordinates. It doesn't perform business logic. Each specialist remains responsible for its own domain.

YAML Controls Behaviour

One of the biggest architectural wins was moving behaviour into configuration. Instead of editing Python every time an agent changed, we edited YAML. For example:

workflow:
  member_report:

    agents:
      - analyst
      - care
      - communications

    parallel:
      - analyst
      - care

    validation: member_schema

The orchestrator interprets configuration. It doesn't contain workflow definitions. That distinction dramatically simplified maintenance.

Adding a New Agent Became Predictable

Because orchestration is configuration-driven, introducing a new specialist follows a consistent process.

Create Agent
↓
Define Prompt
↓
Register YAML
↓
Add Validation
↓
Deploy

No internal framework modifications. No orchestration rewrites. No hidden execution chains. Everything follows the same lifecycle.

Validation Lives Outside the Agents

Another architectural decision proved valuable. Agents never validate themselves. Instead, every response passes through a dedicated validation layer.

Agent Output
↓
Schema Validation
↓
Business Validation
↓
Workflow Continues

Keeping validation independent has several advantages. It makes validation reusable. It keeps prompts focused on reasoning. And it ensures every agent follows the same quality standards.

Observability Was Built In—Not Added Later

Earlier articles explored structured logging and distributed tracing. Owning orchestration made those features much easier to implement. Every workflow automatically records:

• execution ID
• correlation ID
• prompt version
• configuration version
• routing decisions
• execution times
• token usage
• validation results

Nothing special has to happen inside each agent. The orchestration layer captures those events automatically. That's a major advantage of owning the control plane.

Versioning Becomes Consistent

Another unexpected benefit emerged. Everything now follows the same versioning strategy.

Application
↓
Workflow
↓
Configuration
↓
Prompt
↓
Deployment

Instead of tracking independent pieces manually, every execution captures a complete snapshot of the platform. Weeks later, we can reproduce exactly how a workflow behaved. That's invaluable during production investigations.

The Platform Became Easier to Reason About

Looking back, perhaps the biggest advantage wasn't flexibility. It was clarity. Every workflow answers the same questions.

• Which configuration loaded?
• Which agents executed?
• Which validations ran?
• Which prompt versions were active?
• Which model generated each response?

Nothing is hidden behind internal abstractions. The platform explains itself. That dramatically reduced debugging time.

What We Didn't Build

This is just as important. We deliberately avoided building:

• a custom LLM framework
• a prompt templating engine
• an agent runtime
• a plugin ecosystem
• a workflow DSL
• a visual orchestration designer

Those are fascinating engineering problems. They just weren't our problems. PHHM exists to solve organizational workflows. Every line of orchestration code had to support that goal.

Building our own orchestration layer wasn't free. We gave up several conveniences. For example:

That's an intentional trade-off. Our priorities favored long-term operability over short-term convenience. Another team building a different product might reasonably choose the opposite.

The Biggest Lesson

Looking back, one engineering principle explains why this architecture has remained successful.

Own the layer where your product becomes unique.

FastAPI isn't our differentiator.

Python isn't our differentiator. OpenAI isn't our differentiator. The orchestration layer is where PHHM's business rules, workflows, validation, observability, and deployment strategy come together. That made it worth owning.

Final Thoughts

People sometimes ask whether I'd make the same decision today. Yes. Not because orchestration frameworks have stood still—they've improved tremendously. But because the question was never:

"Which framework is best?"

It was:

"Which parts of the platform should belong to us?"

For PHHM, the answer was always the orchestration layer. Owning that layer gave us deterministic routing, explicit workflow state, built-in observability, consistent validation, configuration-driven behaviour, and complete deployment control. Those capabilities shaped every other engineering decision in the platform. The result isn't a framework. It's something much simpler. A lightweight orchestration engine that exists solely to solve the problems PHHM actually has. And, in my experience, that's often the best kind of software.

Key Takeaways

If you're deciding whether to adopt an orchestration framework or build your own, I'd recommend asking these questions first:

• Is orchestration part of your competitive advantage?
• Do you need deterministic workflows or flexible experimentation?
• How important are observability and operational debugging?
• Will configuration change more frequently than code?
• Do you need complete ownership of routing, validation, and deployment?
• Can existing Python primitives like asyncio solve most of your orchestration needs?
• Are you solving a general orchestration problem or a domain-specific workflow problem?
• Does introducing another abstraction simplify your architecture—or make it harder to understand?

Choose the option that makes your production system easier to operate five years from now—not just easier to prototype this week.