Guardrails for Multi-Agent AI: Schema Validation, Output Contracts, and Safe AI Workflows
How PHHM reduced workflow errors by 73% by treating every AI response like an API contract instead of trusted output.
Focus
Guardrails
Read time
10 min
Series
PHHM Journal
Theme
Production AI
The first version of PHHM trusted every AI response. That lasted about a week. Sometimes an agent returned invalid JSON. Sometimes required fields disappeared. Sometimes values had the wrong type. Sometimes the output looked perfectly reasonable... ...until another agent tried to use it. That's when we realized something.
The first hallucination usually isn't what breaks an AI workflow. The second agent trusting it is. That realization completely changed the platform. Instead of assuming models would always produce correct outputs... We started assuming every AI response was untrusted input. Everything improved after that.
Section 1
AI Responses Are API Responses
One mental model changed everything. We stopped treating AI outputs as conversations. We started treating them as API responses. Imagine calling a payment API. Would you trust this?
Example
{
"status": "probably_paid",
"amount": "quite a lot"
}
Of course not. You expect contracts. Types. Schemas. Validation. Yet many AI systems happily pass responses like this directly into another model. That's incredibly risky. Instead, every AI response in PHHM follows an explicit contract.
The Real Source of AI Failures
Most people blame hallucinations. Hallucinations are only part of the problem. The real danger is propagation. Imagine this workflow.
User Request
│
▼
Analyst
│
▼
Care
│
▼
Communications
Suppose the Analyst produces an invalid recommendation. Nothing crashes. The Care Agent simply trusts it. Then Communications trusts the Care Agent. One incorrect response becomes three. That's exactly how distributed system failures spread. AI systems behave the same way.
Every Boundary Is a Contract
One principle shaped the validation layer.
Whenever one agent hands work to another, that boundary should behave like an API.
That means:
defined inputs
defined outputs
validation
versioning
documentation
The agents don't exchange conversations. They exchange contracts. That single architectural decision reduced an enormous amount of uncertainty.
Designing Output Schemas
Instead of asking the Analyst to "summarize the member," we define exactly what success looks like. For example:
If validation fails... The workflow stops before bad data spreads.
Why Validation Beats Prompt Engineering
Many prompt engineering discussions focus on getting the model to "behave." That's useful. But prompts aren't guarantees. Validation is. A prompt asks. A schema verifies. Those are very different responsibilities. That's why PHHM treats prompt engineering and validation as complementary, not interchangeable.
Schema Validation Isn't Enough
Passing a schema doesn't mean the output is correct. Consider this response.
These checks aren't AI-specific. They're business-specific. And that's exactly why they shouldn't live inside prompts.
Two Layers, Two Responsibilities
Looking back, separating validation into two layers simplified the entire system.
Layer
Responsibility
Schema Validation
Is the structure correct?
Business Validation
Does the data make sense?
The first protects the software. The second protects the business. You need both.
Contracts Reduce Prompt Complexity
Another unexpected benefit appeared. As contracts became stronger... Prompts became simpler. Instead of asking the model to:
always include every field
never omit required values
format responses correctly
We let validation enforce those requirements. Prompts focused on reasoning. Validation focused on correctness. That's a much cleaner separation of responsibilities.
The Architecture That Emerged
Eventually every AI response followed the same path.
AI Response
│
▼
Schema Validation
│
▼
Business Validation
│
▼
Workflow Contract
│
▼
Shared State
│
▼
Next Agent
Notice what's missing. Blind trust. No response reaches another agent until every validation layer succeeds.
The Biggest Lesson
One sentence summarizes almost the entire architecture.
Models generate possibilities. Validation decides what becomes reality.
That's the difference between an AI assistant and a production AI platform.
Recovery Strategies: What Happens When Validation Fails?
One of the biggest misconceptions in AI engineering is that validation failures are exceptional. They're not. They're expected. Language models are probabilistic systems. No matter how carefully prompts are written, some outputs will eventually fail validation. The question isn't:
"Can we prevent every failure?"
Example
It's:
"Can the workflow recover without affecting the user?"
That became one of the most important design principles in PHHM.
Validation Failure Isn't Workflow Failure
Early versions of the platform treated validation failures like application errors.
Not every validation failure deserves the same response. We classify failures into categories.
Failure Type
Recovery Strategy
Invalid JSON
Retry generation
Missing required field
Retry with structured feedback
Business rule violation
Reject and regenerate
Provider timeout
Retry or fail over
Rate limit exceeded
Backoff and retry
Internal application error
Surface an operational error
Classifying failures keeps recovery predictable.
Retry Only When It Makes Sense
Retries are useful. Blind retries are expensive. Suppose the model returns malformed JSON. A retry has a good chance of succeeding. Suppose the workflow requests a care plan without analysis. Retrying won't help. The input itself is incomplete. One principle guided the retry system.
Knowing the difference saves both latency and cost.
Structured Retry Feedback
One lesson surprised us. Generic retries weren't nearly as effective as informed retries. Instead of simply asking the model again: Please try again. The Overseer explains exactly why validation failed. Validation failed. Reason: Missing required field: risk_score
Return the complete schema. Providing structured feedback dramatically increased first-retry success rates. The model wasn't guessing anymore. It knew exactly what needed to change.
Escalation Instead of Infinite Retries
Retries should always have limits. Otherwise, one bad workflow can consume resources indefinitely. PHHM defines a maximum retry count.
Example
MAX_RETRIES = 3
for attempt in range(MAX_RETRIES):
result = execute_agent()
if validate(result):
break
If every attempt fails, the workflow escalates. The important point is that retries are finite. Production systems should fail predictably.
Fallback Models
Sometimes the problem isn't the prompt. It's the provider. External AI services occasionally experience degraded performance. Rather than abandoning the workflow immediately, the Overseer can switch providers when appropriate.
Primary Model
│
Validation Failed
│
▼
Fallback Model
│
▼
Validation
The workflow continues without exposing infrastructure issues to the user. Not every workflow needs this capability. Critical workflows often benefit from it.
Confidence Isn't Binary
Another lesson emerged as workflows became more sophisticated. Not every response is simply "valid" or "invalid." Some outputs are technically correct but uncertain. Instead of binary thinking, we introduced confidence scoring.
Confidence doesn't replace validation. It complements it. Validation answers:
"Is this usable?"
Confidence answers:
"How certain are we?"
Those are different questions.
Confidence Drives Workflow Decisions
Confidence becomes another input to orchestration. For example:
High Confidence
↓
Continue Workflow
────────────
Low Confidence
↓
Retry
────────────
Very Low Confidence
↓
Request Human Review
The workflow adapts instead of treating every response identically.
Human Review as a First-Class Feature
One important realization changed how we thought about automation. The goal wasn't to eliminate humans. It was to involve them intentionally. Certain workflows automatically escalate when confidence falls below an acceptable threshold.
AI Response
│
▼
Validation Passed
│
▼
Confidence Check
│
├──────────────┐
▼ ▼
High Low
│ │
▼ ▼
Continue Human Review
Human review isn't a failure. It's another orchestration path.
Part 2
Output Contracts Between Agents
Once validation and recovery existed, another architectural pattern emerged. Every agent communicates through explicit contracts. For example, the Analyst always returns:
The Care Agent doesn't parse free-form text. It consumes a defined structure. That reduces ambiguity dramatically. The contract becomes more important than the prompt itself.
Contracts Make Agents Replaceable
Because every specialist adheres to the same interface, replacing an agent becomes much easier. Imagine introducing a completely new Analyst implementation. As long as it returns the same validated contract, nothing downstream changes. The Care Agent doesn't know which model generated the analysis. It only knows the contract was satisfied. That's classic interface-based design applied to AI systems.
The Validation Pipeline
Looking back, every AI response followed the same lifecycle.
AI Response
│
▼
Schema Validation
│
▼
Business Rules
│
▼
Confidence Evaluation
│
▼
Retry or Recovery
│
▼
Workflow Contract
│
▼
Next Agent
Notice that generation is only one stage. Everything after generation is equally important. That's where production reliability comes from.
The Bigger Lesson
One architectural principle guided the entire validation layer.
Never optimize for perfect AI. Optimize for predictable workflows.
Models will always make mistakes. Reliable platforms assume that. They detect mistakes. Recover from them. Prevent them from spreading. That's exactly what guardrails are designed to do.
Guardrails as an Architecture, Not a Feature
When people hear the word guardrails, they often think about prompt engineering. Add another instruction. Warn the model not to hallucinate. Ask it to follow the schema. Those techniques help. But they aren't where reliability comes from. The biggest lesson from building PHHM was this:
Guardrails don't belong inside prompts. They belong around prompts.
That one realization changed the entire platform. Instead of trying to convince models to always behave correctly... We built a system that assumes they occasionally won't. That's a very different philosophy.
The Layers of Protection
Looking back, PHHM never relied on a single safeguard. Instead, every workflow passed through multiple independent layers.
User Request
│
▼
Request Validation
│
▼
Workflow Planning
│
▼
AI Agent Execution
│
▼
Schema Validation
│
▼
Business Rule Validation
│
▼
Confidence Evaluation
│
▼
Retry / Recovery Logic
│
▼
Workflow State Commit
│
▼
Final Response
Notice what this architecture doesn't assume. It never assumes the model is correct. Every stage earns trust before the next stage continues.
Trust Should Increase Gradually
One mental model became incredibly useful during development. Think about trust as something that grows. Not something that's granted automatically.
Raw AI Output
↓
Validated Structure
↓
Verified Business Rules
↓
Accepted Workflow State
↓
Trusted Result
Every validation layer increases confidence. By the time another agent consumes the result, it has already survived multiple independent checks.
Defense in Depth
Security engineers often talk about defense in depth. No single security control protects an application. AI systems benefit from exactly the same idea. If one layer misses a problem, another catches it. For example:
A prompt encourages structured output.
Schema validation checks formatting.
Business rules validate meaning.
Confidence scoring detects uncertainty.
Retry logic attempts recovery.
Human review handles exceptional cases.
No single mechanism is perfect. Together, they create a dependable system.
Observability Completes the Loop
Validation without visibility only solves half the problem. Every guardrail event should produce telemetry. Examples include:
schema failures
business rule violations
retry attempts
confidence scores
fallback model usage
human review requests
Those events answer questions like:
Which agent fails validation most often?
Which prompt version increased retries?
Which workflows require the most human intervention?
Which business rules fail most frequently?
Without observability, guardrails become invisible. Without visibility, they can't improve.
Measuring Reliability
One surprising realization was that model quality wasn't our primary reliability metric. Workflow quality was. Instead of asking:
Example
"Did the model answer correctly?"
We asked:
Did the workflow complete successfully?
Did validation succeed?
Was recovery required?
How many retries occurred?
Did the user receive a usable result?
Those metrics reflect what users actually experience. The model is only one part of the workflow.
Reliability Emerges from the System
People often ask:
"Which model are you using?"
It's a fair question. But after building PHHM, I think it's the wrong first question. A better one is:
"What happens when the model is wrong?"
That's where architecture matters. Reliable systems don't exist because models never fail. They exist because failures are anticipated, contained, and recovered from before they affect users.
The Architecture That Emerged
By the end of the project, the validation layer had become just as important as the models themselves.
Notice the pattern. Every article in this series has reinforced the same idea. Each layer owns one responsibility. Together, they create a reliable platform.
The Five Principles of AI Guardrails
If I were building another multi-agent system tomorrow, these are the principles I'd keep.
1. Treat AI outputs as untrusted input
Never allow one model's output to become another model's input without validation.
2. Separate reasoning from validation
Models generate. The platform verifies. Those responsibilities should never be mixed.
3. Design explicit contracts
Every agent should publish a documented, versioned output schema. Consumers should rely on the contract—not the implementation.
4. Recover instead of failing immediately
Retries, fallbacks, and human review are all legitimate orchestration paths. Reliability comes from recovery, not perfection.
5. Measure the workflow—not the model
Users experience workflows. That's what should be optimized.
Final Thoughts
When I first started experimenting with language models, I assumed the challenge would be writing better prompts. It wasn't. The real challenge was building software that could safely incorporate probabilistic systems. That required a shift in thinking. Instead of asking:
"How do I make the model perfect?"
I started asking:
"How do I build a platform that remains reliable even when the model isn't?"
That single question shaped every architectural decision in PHHM. It led to:
orchestration instead of direct agent communication
configuration instead of hardcoded behavior
workflow state instead of shared conversations
prompt versioning instead of prompt editing
contracts instead of assumptions
validation instead of blind trust
The models improved over time. The architecture made those improvements safe to adopt. That's the difference between an AI demo and a production AI platform.
Key Takeaways
Example
If you're building production AI systems, I'd recommend adopting these practices from day one:
Treat every AI response like an API response.
Validate structure before meaning.
Separate schema validation from business validation.
Use confidence scoring to guide workflow decisions.
Retry intelligently instead of blindly.
Make human review part of the workflow, not an exception.
Log every validation and recovery event.
Design explicit contracts between agents.
Measure workflow reliability, not just model quality.
Build systems that expect failure—and recover gracefully.