AI Pipeline Design
Part of: Effective AI Utilization — Table of Contents
A single AI call is simple. Five AI calls that depend on each other's output, share context, and need to complete reliably is an engineering problem. BrianBot's 5-step episode pipeline is a concrete example of sequential AI orchestration.
BrianBot's Pipeline: 5 Steps, Sequential
Each episode runs through: Topic Extraction → Transcript Generation → Metadata → Memory Update → Companion Content. Each step uses the output of previous steps as input context. This is a linear pipeline — simple to reason about, simple to debug, but leaving performance on the table.
Sequential vs Parallel Execution
Topic extraction and metadata generation don't depend on each other — they both need the raw transcript but not each other's output. These could run in parallel, cutting wall-clock time. BrianBot runs everything sequentially because the pipeline was built incrementally and parallelism adds complexity.
The rule of thumb: parallelize when steps share inputs but not outputs. Sequence when step N needs step N-1's result.
State Management Between Steps
BrianBot passes state through the episode record in the database — each step reads from and writes to the same row. This is durable (survives crashes) but adds DB round-trips. An alternative is in-memory pipeline state that's persisted only at the end, with checkpointing for crash recovery.
Error Boundaries in Pipelines
BrianBot's pipeline has a single try/catch that marks the episode as FAILED if any step throws. A more resilient pattern: wrap each step in its own error boundary, with step-level status tracking. This way you know which step failed, and can potentially retry just that step or continue with degraded output (see Model Fallback and Resilience).
The MythOS Enrichment Step
BrianBot includes a mythos-enrich step that loads context from MythOS memos before generating companion content. This is the only step with graceful degradation — if MythOS enrichment fails, the pipeline continues with empty context. This pattern should be the norm, not the exception.
Related: Model Routing Strategies, Model Fallback and Resilience, Queue and Rate Limiting for AI Workloads, AI Observability and Debugging