State Management for Multi-Agent AI Systems: What Lives in Memory, What Lives in Storage

One of the biggest misconceptions in AI development is that context and state are the same thing. They aren't. In fact, confusing the two is one of the fastest ways to build unreliable multi-agent systems. Many AI applications simply keep appending messages to an ever-growing conversation. The longer the conversation becomes:

• the slower the model gets
• the more expensive requests become
• the harder prompts become to reason about
• the easier it is for unrelated context to influence new decisions

Eventually every agent knows everything. Ironically... That's exactly the problem. While building PHHM we discovered something surprising. The hardest question wasn't:

"How do agents communicate?"

It was:

"What information should they never share?"

The answer completely changed how we designed our orchestration layer.

Section 1

These two words get used interchangeably. They shouldn't. Here's how we define them.

Context

Information an agent needs to complete one task. Temporary. Disposable. Usually lives inside the prompt. Examples:

• user question
• recent conversation
• uploaded document
• previous response

State

Information the platform needs after the task finishes. Persistent. Shared. Owned by the orchestration layer. Examples:

• workflow progress
• member record
• validation status
• completed tasks
• execution metadata

That distinction changed everything.

A Mental Model

Think of a hospital. A doctor reads today's notes before seeing a patient. That's context. The patient's medical record survives after the appointment. That's state. The doctor doesn't own the record. The hospital does. PHHM follows exactly the same principle.

Agents consume context. The orchestrator owns state.

Early prototypes stored almost everything in one shared object.

memory = {
    "messages": [...],
    "analysis": ...,
    "care": ...,
    "newsletter": ...,
    "devotional": ...
}

Convenient? Absolutely. Scalable? Not even close. Every agent could see everything. Soon prompts contained:

• onboarding history
• care plans
• newsletter drafts
• analysis reports
• unrelated conversations

Agents became slower. Token costs increased. Unexpected reasoning appeared. The more context we shared... The worse the system became.

Bigger Context Isn't Better Context

One myth keeps appearing in AI discussions.

"Just give the model more context."

More context often means:

• more irrelevant information
• more distractions
• higher token costs
• longer latency

The better question is:

"What's the smallest amount of information this agent actually needs?"

That's what PHHM optimizes for. Not maximum context. Minimum useful context.

The Principle That Changed Everything

One rule became our north star.

Agents should know only what they need to complete their current task.

Nothing more. Nothing less. That sounds obvious. It turns out to be surprisingly difficult to maintain.

Another mistake we made early was allowing agents to "own" information. For example:

Analyst
↓
Stores recommendations
↓
Care reads directly
↓
Communications reads Care

Now every agent depends on another. Change one. Risk breaking three. Instead, ownership moved to the orchestration layer.

           Overseer
                │
      ┌─────────┼─────────┐
      ▼         ▼         ▼
  Analyst     Care   Communications
      │         │         │
      └─────────┼─────────┘
                │
                ▼
          Workflow State

No agent owns workflow data. The orchestrator does.

Rather than passing huge conversations around, PHHM maintains a structured workflow state.

workflow_state = {

    "member": {},

    "analysis": None,

    "care_plan": None,

    "communications": None,

    "completed_steps": [],

    "metadata": {}

}

Every agent receives only the slice it needs. The Analyst doesn't care about newsletters. The Communications Agent doesn't need raw analysis notes. Each specialist sees only the information relevant to its responsibility.

Why This Matters

Reducing shared state had unexpected benefits.

• Lower token usage
• Faster execution
• Fewer hallucinations
• Simpler prompts
• Easier debugging

Most importantly... Agents stopped influencing each other in unpredictable ways.

Designing a Workflow State Object

Once we stopped treating context as shared memory, another question emerged.

Where should long-lived information actually live?

The answer wasn't inside any individual agent. It belonged to the workflow itself. Instead of every agent maintaining its own memory, PHHM maintains a single workflow state that acts as the source of truth throughout execution. Think of it as a document that every specialist can contribute to—but nobody owns.

The Overseer Owns State

One architectural rule simplified almost every workflow.

Agents produce information. The Overseer owns information.

That distinction is subtle, but incredibly important. The Analyst doesn't own the analysis. It generates it. The Care Agent doesn't own the care plan. It proposes it. The Communications Agent doesn't own the email. It drafts it. Once an agent finishes its task, the output is handed back to the Overseer, validated, and committed to the workflow state.

Ownership never leaves the orchestration layer.

Designing the State Object

Rather than passing entire conversations between agents, PHHM uses a structured state object. A simplified version looks like this:

from dataclasses import dataclass, field

@dataclass
class WorkflowState:

    member: dict

    analysis: dict | None = None

    care_plan: dict | None = None

    communication: dict | None = None

    completed_steps: list[str] = field(default_factory=list)

    metadata: dict = field(default_factory=dict)

Every workflow starts with the same structure. As agents complete work, they enrich the state instead of replacing it. That makes the flow predictable and easy to inspect.

State Evolves Incrementally

Instead of creating entirely new objects, every completed step contributes one validated update. For example:

Workflow Started
↓
Member Loaded
↓
Analysis Added
↓
Care Plan Added
↓
Communication Added
↓
Workflow Complete

Every transition is intentional. Every change is traceable. That makes debugging dramatically easier.

Immutable Thinking

One lesson borrowed from functional programming improved reliability more than we expected. Whenever possible, agents should return new data instead of mutating existing data. Instead of this: workflow_state["analysis"] = analysis workflow_state["status"] = "complete" Think in terms of:

updated_state = {
    workflow_state,
    "analysis": analysis
}

Or, using immutable data structures, create a new state object from the previous one. Why? Because immutable thinking makes workflows predictable. You always know what changed. And more importantly... You know what didn't.

Why Mutability Becomes Dangerous

Imagine two agents running in parallel. Both receive the same state object. Both modify it. Which version wins? Without careful synchronization, you've introduced race conditions.

        Shared State
             │
     ┌───────┴────────┐
     ▼                ▼
 Analyst          Communications
     │                │
     └───────┬────────┘
             ▼
       Conflicting Updates

Instead, each agent should return its own result. The Overseer becomes the only component allowed to merge changes. That eliminates an entire class of concurrency bugs.

State Should Be Explicit

One anti-pattern we encountered early was hiding state inside prompts. For example: Remember that the member requested a follow-up next week... The prompt now contains business state. That's fragile. Instead, prompts should receive structured inputs.

agent_input = {
    "member": workflow.member,
    "analysis": workflow.analysis,
    "next_action": "create_follow_up"
}

Now the prompt doesn't need to remember anything. It simply reasons over structured data. The distinction is subtle but powerful.

State Has a Lifecycle

Not every piece of information deserves to live forever. One useful way to think about state is by its lifespan.

Grouping state by lifetime makes storage decisions much easier.

What Should Never Be Shared

One of the biggest improvements we made came from asking a simple question before exposing data to another agent.

Does this agent actually need this information?

If the answer was no... We didn't include it. Examples: The Communications Agent doesn't need raw reasoning from the Analyst. The Welcome Agent doesn't need previous newsletter drafts. The Gospel Agent doesn't need internal validation logs. Every unnecessary field increases:

• prompt size
• token usage
• cognitive load
• risk of unintended reasoning

Minimal state is usually better state.

State Is an Interface

Eventually we stopped thinking about the workflow state as a Python object. We started thinking about it as an interface. Every agent expects a contract. Every agent produces a contract. The Overseer enforces those contracts.

Workflow State
       │
       ▼
 Analyst Contract
       │
       ▼
 Validated Output
       │
       ▼
 Workflow State

That architecture keeps responsibilities clean. The state doesn't belong to any individual component. It belongs to the workflow itself.

Event Thinking Instead of Snapshot Thinking

Another architectural shift came from asking not just what the current state is, but how it got there. Instead of only storing the latest snapshot, we began thinking in terms of events.

Workflow Started
↓
Member Loaded
↓
Analysis Completed
↓
Validation Passed
↓
Care Plan Generated
↓
Communication Sent

This timeline tells a richer story than the final state alone. It also opens the door to replaying workflows, auditing decisions, and diagnosing failures after the fact. Even if you don't implement full event sourcing, thinking in events leads to more transparent systems.

The Principle That Scales

If there's one idea I'd carry into every future AI project, it's this:

State belongs to the workflow. Agents borrow it temporarily.

That one principle kept PHHM modular even as more agents, workflows, and capabilities were added. No agent became indispensable. No component accumulated hidden responsibilities. The workflow remained the source of truth.

State Persistence: What Should Live in Memory and What Should Live Forever?

One of the first production failures we experienced wasn't caused by an AI model. It was caused by memory. A workflow had completed several expensive steps. Analysis had finished. Validation had passed. The Care Agent had generated recommendations. Then the application restarted. Everything disappeared.

Not because the AI failed. Because we had stored critical workflow state in memory. That incident changed how we thought about persistence.

Not All State Deserves a Database

One mistake many systems make is treating all information the same. It isn't. Some data should disappear as soon as the request ends. Some should survive for minutes. Some should survive forever. One useful mental model is to classify state by lifetime.

Once we started thinking in lifetimes instead of objects, storage decisions became much easier.

Memory Is Fast—But Fragile

In-memory state is incredibly useful. It's also temporary. For example:

workflow = WorkflowState(
    member=member
)

This is perfect while a request is running. It's fast. No database queries. Minimal latency. But if the process dies... So does the workflow. Memory should only hold information you're willing to lose.

Databases Preserve Progress

Now imagine a workflow that takes several minutes. Analysis completes. Validation succeeds. Communications is halfway finished. The server restarts. Without persistence, the entire workflow begins again. Instead, the Overseer periodically saves progress.

workflow_repository.save(
    workflow_state
)

Now recovery becomes possible. The next execution simply resumes from the latest checkpoint.

Checkpointing Long Workflows

One design decision paid dividends almost immediately. We introduced checkpoints. Instead of waiting until the entire workflow completed, the Overseer saved progress after every major stage.

Workflow Started
        │
        ▼
Analysis Complete
   ✓ Checkpoint
        │
        ▼
Care Plan Complete
   ✓ Checkpoint
        │
        ▼
Communications Complete
   ✓ Checkpoint
        │
        ▼
Workflow Finished

If something failed halfway through, we resumed from the last successful checkpoint instead of starting over. That reduced unnecessary AI calls and significantly improved reliability.

Recovery Becomes Simple

Suppose the application crashes immediately after the Care Agent finishes. Without checkpoints:

Restart
↓
Run Everything Again

With checkpoints:

Restart
↓
Load Workflow
↓
Resume From Care
↓
Continue

The difference becomes enormous as workflows become longer and more expensive.

Cache Isn't a Database

Another lesson we learned was not to confuse caching with persistence. A cache improves performance. A database preserves truth. For example:

Memory Cache
↓
Fast Access
↓
Temporary

versus

Database
↓
Durable Storage
↓
System of Record

The workflow may use both. But they solve different problems.

Persist the Right Things

Not every object deserves permanent storage. We settled on a simple rule. Persist information that would be expensive, impossible, or risky to recreate. For example: Persist:

• workflow progress
• completed analysis
• care plans
• validation status
• execution metadata

Do not persist:

• temporary prompts
• intermediate reasoning
• token-by-token generation
• disposable context

Storing everything increases complexity without increasing value.

Designing for Idempotency

One subtle challenge appears when recovering workflows. Suppose the Communications Agent already sent an email. The application crashes before updating the workflow state. When recovery starts... Should the email be sent again? Probably not. That's where idempotency becomes essential. Every major action should be safe to retry.

Instead of asking:

"Did this function run?"

The system asks:

"Has this action already been completed?"

That small distinction prevents duplicate work and unintended side effects.

Tracking Workflow Progress

The simplest solution was surprisingly effective. Every workflow tracks completed steps.

workflow.completed_steps = [

    "analysis",

    "care_plan"

] Before executing an agent, the Overseer checks:

if "analysis" not in workflow.completed_steps:

    run_analysis()

This pattern makes retries safe and recovery deterministic.

State Expiration

Not every workflow needs to live forever. Some become irrelevant after a few minutes. Others after several days. We introduced expiration policies based on workflow type. For example:

Expiration policies keep storage manageable while preserving information that matters.

Durable Systems Assume Failure

One mindset changed how we designed every workflow. Instead of asking:

"What happens if everything works?"

We asked:

"What happens if the application crashes right here?"

At every stage. If the answer wasn't obvious, the architecture wasn't finished. Production systems aren't designed around success. They're designed around recovery.

The Bigger Lesson

Persistence isn't about databases. It's about resilience. Reliable AI platforms don't become reliable because models never fail. They become reliable because failures don't erase progress. That's exactly what checkpoints, durable state, and recovery mechanisms provide.

State Recovery Is an Orchestration Problem

Notice something interesting. The agents don't know anything about persistence. They don't load checkpoints. They don't save workflows. They don't recover after crashes. Only the Overseer manages durability. That separation keeps every specialist focused on one responsibility. Agents generate.

The Overseer coordinates. Persistence protects. It's the same architectural philosophy we've followed throughout PHHM.

Context Optimization: Why More Memory Usually Makes AI Worse

One of the most common assumptions in AI development is that more context produces better results. It's an understandable assumption. If the model knows more, surely it can make better decisions. Right? Not necessarily. While building PHHM, we found the opposite was often true. As prompts accumulated more conversation history, workflow state, and unrelated information, performance started to decline. Responses became slower.

Token costs increased. Reasoning became less focused. And occasionally, agents started incorporating information that wasn't even relevant to their task. The problem wasn't that the models lacked intelligence. The problem was that we were giving them too much to think about.

Modern language models can process enormous context windows. That's an impressive capability. But larger context windows don't eliminate the need for good system design. Think about an experienced software engineer. Giving them access to every file in your repository doesn't automatically help them solve a bug. In many cases, it slows them down. They still need the right information. AI agents are no different.

Large context windows increase capacity. They don't replace architecture.

Context Is a Budget

One mindset changed how we designed prompts. Instead of treating context as free, we started treating it like a budget. Every piece of information included in a prompt had to justify its existence. We began asking simple questions before adding anything.

• Does this agent actually need it?
• Will this information change the decision?
• Is it still relevant?
• Could it live somewhere else?

If the answer was no, we removed it. Over time, our prompts became smaller, faster, and easier to reason about.

Designing Minimal Context

The Analyst Agent doesn't need everything. It only needs enough information to produce a structured analysis. Instead of sending an entire workflow, we prepare a focused input.

agent_input = {
    "member": workflow.member,
    "history": workflow.member_history,
    "request": workflow.current_request
}

Notice what's missing. No communication history. No onboarding details. No validation logs. No unrelated workflow metadata. The agent receives exactly what it needs—and nothing more.

Context Is Not Shared Memory

One mistake we made early was assuming every agent should inherit everything that came before. Imagine this workflow.

User Request
      │
      ▼
 Welcome
      │
      ▼
 Analyst
      │
      ▼
 Care
      │
      ▼
 Communications

It seems natural for each agent to receive the complete conversation. But over time that conversation becomes cluttered with information only one specialist ever needed. Instead, every handoff is intentional. The Overseer prepares a fresh context for every execution. Each agent starts with a clean workspace.

Think Like an Operating System

One analogy helped us rethink memory. An operating system doesn't load every application into RAM all the time. It loads what is needed. Unloads what isn't. Schedules work efficiently. The Overseer performs the same role. Instead of exposing the entire workflow to every agent, it assembles just enough context for the current task. That approach keeps prompts focused while reducing latency and token usage.

Context Assembly

Rather than storing giant prompts, the Overseer assembles context dynamically.

Workflow State
       │
       ▼
Select Relevant Data
       │
       ▼
Load Prompt
       │
       ▼
Inject Current Task
       │
       ▼
Execute Agent

Notice the separation. The workflow state exists independently. The prompt exists independently. The context is assembled only when needed. That makes the system dramatically more flexible.

Forgetting Is a Feature

One of the most valuable architectural decisions we made was allowing the system to forget. That sounds counterintuitive. But forgetting irrelevant information improves reasoning. For example, once the Welcome Agent finishes onboarding, those conversational details rarely matter to the Analyst. Likewise, draft newsletter content has no value to the Care Agent. Keeping everything forever doesn't create intelligence. It creates noise. Good AI systems remember deliberately.

Great AI systems forget deliberately.

Context Boundaries

Every agent has a clearly defined boundary.

Only one component sees the whole picture. The Overseer. Everyone else sees only their slice of the workflow. That principle dramatically reduced accidental coupling between agents.

Performance Benefits

Optimizing context wasn't only about cleaner architecture. It produced measurable operational improvements. Smaller prompts led to:

• lower token consumption
• reduced latency
• more consistent outputs
• simpler prompt maintenance
• lower inference costs

Perhaps most importantly, agents became easier to understand because each prompt focused on a single responsibility.

The Architecture We Ended Up With

Looking back, PHHM isn't really a system of six AI agents. It's a set of architectural layers.

                 User Request
                      │
                      ▼
             Orchestration Layer
                      │
                      ▼
            Workflow State Engine
                      │
                      ▼
          Context Assembly Pipeline
                      │
                      ▼
            Specialized AI Agents
                      │
                      ▼
          Validation & Guardrails
                      │
                      ▼
              Final Response

Each layer has exactly one responsibility. That separation is what makes the platform maintainable.

The Five Principles of State Management

If I were building another multi-agent platform tomorrow, I'd follow the same five principles.

1. Separate context from state

Temporary information belongs in prompts. Persistent information belongs in the workflow.

2. Keep state ownership centralized

Agents should generate information. The orchestration layer should own it.

3. Share less

Every additional piece of shared information increases complexity. Only expose what an agent genuinely needs.

4. Design for recovery

Assume failures will happen. Persist progress. Checkpoint long workflows. Make retries safe.

5. Optimize context intentionally

The goal isn't maximum context. It's maximum relevance.

Final Thoughts

When people discuss memory in AI systems, the conversation usually focuses on larger context windows, vector databases, or long-term memory. Those tools are valuable. But they're only part of the story. The more important question is architectural.

Who owns information? Who should see it? How long should it live? When should it be forgotten?

Answer those questions well, and the technology becomes much easier to change. That's exactly what happened with PHHM. As the platform evolved, the orchestration layer remained simple because it treated state as a first-class architectural concern rather than an implementation detail. The models improved. The prompts evolved. New agents were added. But the principles remained the same. State belongs to the workflow.

Context belongs to the task. And the Overseer decides what every agent needs to know.

Key Takeaways

If you're designing a production multi-agent AI system, these are the practices I'd recommend from day one:

• Distinguish context from persistent state.
• Give every piece of state a clear owner.
• Build a structured workflow state object.
• Treat state as an API contract between agents.
• Persist progress with checkpoints.
• Design every workflow to recover safely.
• Assemble context dynamically instead of sharing everything.
• Remember intentionally—and forget deliberately.