Design Considerations and Limitations#
While the API is minimal—centered on Agent, Task, and Tool—there are important tradeoffs and design constraints to be aware of.
| Design Considerations | Limitations |
|---|---|
API centered on Agent, Task, and Tool | Tools are just Python functions decorated with @tool. There’s no centralized registry or schema enforcement, so developers must define conventions and structure on their own. |
| Provides flexibility for orchestration | No retry mechanism or built-in workflow engine |
| Supports evaluator-optimizer loops, routing, and fan-out/fan-in | |
| Agents are composed, not built-in abstractions | Must implement orchestration logic |
| Multi-Agent support | No built-in support for collaboration structures like voting, planning, or debate. |
| Token-level streaming is not supported | |
| No state or memory management out of the box. Applications that require persistent state—such as conversations or multi-turn workflows—will need to integrate external storage (e.g., a vector database or key-value store). | |
| There’s no native memory or “trajectory” tracking between agents. Handoffs between tasks are manual. This is workable in small systems, but may require structure in more complex workflows. |
Prompt Chaining#
This workflow breaks a task into smaller steps, where the output of one agent becomes the input to another. It’s useful when a single prompt can’t reliably handle the full complexity or when you want clarity in intermediate reasoning.
Notebook: Prompt Chaining with Keyword Extraction + Summarization The agent first extracts keywords from a resume, then summarizes what those keywords suggest.
How to evaluate: Check whether each step performs its function correctly and whether the final result meaningfully depends on the intermediate output (e.g., do summaries reflect the extracted keywords?)
-
Check if the intermediate step (e.g. keyword extraction) is meaningful and accurate
-
Ensure the final output reflects or builds on the intermediate output
-
Compare chained vs. single-step prompting to see if chaining improves quality or structure
Orchestrator + Worker Pattern#
In this approach, a central agent coordinates multiple agents, each with a specialized role. It’s helpful when tasks can be broken down and assigned to domain-specific workers.
Notebook: Recruiting Evaluator Orchestrator The orchestrator delegates resume review, culture fit assessment, and decision-making to different agents, then composes a final recommendation.
How to evaluate: Assess consistency between subtasks and whether the final output reflects the combined evaluations (e.g., does the final recommendation align with the inputs from each worker agent?)
-
Ensure each worker agent completes its role accurately and in isolation
-
Check if the orchestrator integrates worker outputs into a consistent final result
-
Look for agreement or contradictions between components (e.g., technical fit vs. recommendation)