Achieve reliability through structured orchestration, and adaptability through LLM agents focused on judgment and synthesis
We evaluated multiple architectures for AI-powered workflows similar to deep research pipelines. While multi-agent systems work well for open-ended tasks, our findings show that sequential, deterministic workflows deliver more consistent results for workflows with defined steps.
The Multi-Agent Challenge
A multi-agent system with a central orchestrator seems logical, but for complex tasks requiring state and memory management, we encountered several issues:
- Routing accuracy degraded after 5-10 messages
- Tool calls became redundant (same searches repeated multiple times)
- State tracking grew increasingly difficult
These problems led to higher costs and reduced reliability.
The Sequential Approach
We restructured our architecture using a state-machine based sequential workflow:
- Each stage (clarification, topic generation, research, summary) operates independently and maintains its own state
- Failures trigger stage-level retries instead of full pipeline restarts
- Tool calls are stage-driven - each stage knows exactly which tools to use
This design reduced errors significantly and made costs more predictable.
Key Design Principles
Our approach separates concerns:
- LLMs handle reasoning, analysis, and content generation
- Code handles orchestration, state management, and retry logic
This separation provides:
- Improved reliability through deterministic routing
- Better cost control through focused LLM calls
- Support for resumable, long-running sessions
- Granular retry capabilities at the stage level
Conclusion
Multi-agent systems have their place in dynamic tasks when they are heavily finetuned with RL. However, for workflows with clear stages and reliability requirements, sequential orchestration with targeted LLM calls offers a more practical solution.
About us: Introducing Shodha AI
Read more : Blog