Advanced45 minModule 3 of 6

Advanced Orchestration Patterns

Master the orchestrator-workers pattern, evaluator-optimizer loops, and autonomous agent architectures for complex AI tasks.

What you'll learn in this module

  • How the orchestrator-workers pattern delegates and synthesizes subtasks
  • How evaluator-optimizer loops iteratively improve outputs
  • How autonomous agents work and when they're appropriate
  • Trade-offs between each advanced pattern

Pattern 4: Orchestrator-Workers

An "orchestrator" LLM dynamically breaks a task into subtasks, delegates each to a "worker" LLM (or tool), and then synthesizes the results.

Subtask 1 Subtask 2 Subtask N Complex Task Orchestrator LLM:Analyze task → plan subtasks Worker 1 Worker 2 Worker N Orchestrator: Synthesize results Final Output

How it differs from parallelization

AspectParallelizationOrchestrator-Workers
SubtasksPredefined by developerDynamically determined by LLM
Number of subtasksFixedVariable per input
Worker specializationAll identical or predefinedOrchestrator chooses per subtask

When to use it

  • The task can't be decomposed in advance — the subtasks depend on the input
  • Different parts of the task require different approaches
  • You need the flexibility of agents with more structure than a free-form loop

Example: Codebase Refactoring Agent

Find endpoints Design schema Rewrite resolvers Update tests Refactor request:'Migrate from REST to GraphQL' Orchestrator:1. Identify all REST endpoints2. Design GraphQL schema3. Rewrite each resolver4. Update tests Worker: Code search+ analysis Worker: Schemageneration Worker: Codegeneration (×N) Worker: Testgeneration Orchestrator:Combine, validate,resolve conflicts Final changeset

The orchestrator prompt

The orchestrator needs to output a structured plan:

You are a task planner. Given a complex request, break it into 
independent subtasks that can be executed separately.

For each subtask, provide:
- task_id: a unique identifier
- description: what needs to be done
- dependencies: list of task_ids this depends on (can be empty)
- tools_needed: which tools the worker should use

Output as JSON array. Do not execute the tasks, only plan them.

Worker execution

Each worker receives:

  1. The subtask description from the orchestrator
  2. Access to the specific tools it needs
  3. Results from any dependency subtasks

The orchestrator then collects all worker outputs and makes a final synthesis call.

Orchestrator mapping: This pattern maps to a workflow where the first AI action dynamically generates a plan, followed by a loop or fan-out of action nodes that execute each subtask. The Orchestrator's template variable system passes the plan and results between steps.

Pattern 5: Evaluator-Optimizer

A generator LLM produces output, then an evaluator LLM reviews it and provides feedback. The loop continues until the evaluator passes the output or a maximum iteration count is reached.

Needs improvement Approved Max iterations Input Generator LLM:Produce output Evaluator LLM:Review output Feedback to Generator Final Output

Two-LLM design

RoleResponsibilityPrompt style
GeneratorProduce the best possible output given the task + feedbackCreative, detailed, follow instructions carefully
EvaluatorAssess output against criteria, provide actionable feedbackCritical, specific, reference concrete issues

When to use it

  • Output quality matters more than latency
  • There are clear, articulable evaluation criteria
  • Human review is too slow or expensive for every output
  • The task benefits from iteration (writing, code, design)

Example: Code Review Loop

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.SKYTELLS_API_KEY,
  baseURL: "https://api.skytells.ai/v1",
});

async function codeReviewLoop(spec: string, maxIterations = 3) {
  let code = "";
  let feedback = "No prior attempt.";

  for (let i = 0; i < maxIterations; i++) {
    // Generate
    const generated = await client.chat.completions.create({
      model: "deepbrain-router",
      messages: [
        {
          role: "user",
          content: `Write code that meets this specification:
${spec}

Previous feedback to address:
${feedback}

Return only the code, no explanation.`,
        },
      ],
    });
    code = generated.choices[0].message.content!;

    // Evaluate
    const evaluation = await client.chat.completions.create({
      model: "deepbrain-router",
      messages: [
        {
          role: "user",
          content: `Review this code against the specification.

Specification: ${spec}

Code:
${code}

If the code fully meets the spec, respond with exactly "APPROVED".
Otherwise, list specific issues to fix. Be concrete — reference line numbers and suggest corrections.`,
        },
      ],
    });

    const evalText = evaluation.choices[0].message.content!;
    if (evalText.trim() === "APPROVED") {
      return { code, iterations: i + 1 };
    }
    feedback = evalText;
  }

  return { code, iterations: maxIterations, warning: "Max iterations reached" };
}

Designing effective evaluators

The evaluator is often more important than the generator. A good evaluator:

  1. Has explicit criteria — don't ask "is this good?" Ask "does this meet criteria X, Y, Z?"
  2. Provides actionable feedback — "The function on line 12 doesn't handle the null case" > "Needs improvement"
  3. Can approve — must have a clear "done" signal to exit the loop
  4. Has iteration limits — always cap the loop to prevent runaway costs
Evaluator Checklist ✓ Explicit criteria ✓ Actionable feedback ✓ Clear approval signal ✓ Iteration cap

Orchestrator mapping: The evaluator-optimizer maps to a workflow with a loop structure. In Orchestrator, you can model this with a Condition node that checks the evaluator output — if not approved, route back to the generator action, passing the feedback via template variables. Set a counter variable as a safety limit.

Pattern 6: Autonomous Agents

The most flexible and the most complex pattern. The LLM operates in a loop, choosing which tools to call, processing results, and deciding when the task is complete.

No Yes Error / stuck Task + Tools Agent LLM:Decide next action Execute chosen tool Observe result Task complete? Return final answer Fallback:human escalation

The agent loop

Every autonomous agent follows this core loop:

  1. Think — assess the current state and decide the next action
  2. Act — execute the chosen tool or API call
  3. Observe — process the result
  4. Repeat until done or stuck

This is sometimes called the ReAct (Reasoning + Acting) pattern.

When to use autonomous agents

Good fitPoor fit
Research tasks with unknown scopeTasks with clear, fixed steps
Exploratory data analysisHigh-stakes financial transactions
Complex debuggingSimple CRUD operations
Multi-tool tasks where sequence can't be predeterminedTasks requiring audit trails

The critical guardrails

Autonomous agents need constraints to prevent runaway behavior:

GuardrailImplementation
Iteration limitCap the think-act-observe loop (e.g., 15 iterations max)
Tool restrictionsOnly expose tools the agent needs; don't give write access if read-only suffices
Cost budgetTrack token usage; terminate if exceeding budget
TimeoutHard wall-clock limit (e.g., 5 minutes)
Scope boundariesInstruct the agent what it should NOT do
Human escalationIf confidence drops below threshold, escalate to human

Example: Research Agent

import json
from openai import OpenAI

client = OpenAI(
    api_key="YOUR_SKYTELLS_API_KEY",
    base_url="https://api.skytells.ai/v1",
)

TOOLS = {
    "web_search": web_search_fn,
    "read_page": read_page_fn,
    "take_notes": take_notes_fn,
    "final_answer": final_answer_fn,
}

def research_agent(question: str, max_iterations: int = 10):
    notes = []
    history = []

    for i in range(max_iterations):
        response = client.chat.completions.create(
            model="deepbrain-router",
            messages=[
                {
                    "role": "system",
                    "content": "You are a research agent. Use the available tools to answer questions.",
                },
                {
                    "role": "user",
                    "content": f"""Answer this question: {question}

Your notes so far: {json.dumps(notes)}
Your action history: {json.dumps(history[-5:])}

Available tools: web_search(query), read_page(url), take_notes(text), final_answer(answer)

Respond with a JSON object: {{"tool": "tool_name", "args": {{...}}}}
If you have enough information, use final_answer.""",
                },
            ],
        )

        action = json.loads(response.choices[0].message.content)
        tool_name = action["tool"]

        if tool_name == "final_answer":
            return action["args"]["answer"]

        result = TOOLS[tool_name](**action["args"])
        history.append({"tool": tool_name, "result": str(result)[:500]})

        if tool_name == "take_notes":
            notes.append(action["args"]["text"])

    return f"Reached max iterations. Current notes: {notes}"

Autonomous agents are powerful but expensive and unpredictable. In production, always combine them with the guardrails above. Monitor executions closely, especially in the first weeks after deployment.

Comparing the Advanced Patterns

PatternLLM ControlPredictabilityCostBest for
Orchestrator-WorkersPlans subtasksMedium — plan varies, execution is structuredMediumComplex tasks needing dynamic decomposition
Evaluator-OptimizerReviews and iteratesMedium — loop count variesMedium-HighQuality-critical outputs
Autonomous AgentFull controlLow — path is entirely dynamicHighOpen-ended research and exploration

Decision tree

Yes Steps depend on input Yes Needs iterativerefinement Fully open-ended Can you definethe steps? Module 2 patterns(chain, route, parallel) Is the taskdecomposable? Orchestrator-Workers Evaluator-Optimizer Autonomous Agent

What you now understand

PatternArchitectureUse when
Orchestrator-WorkersLLM planner + dynamic workersTask decomposition depends on input
Evaluator-OptimizerGenerator + evaluator in a loopOutput quality is paramount
Autonomous AgentFull LLM-controlled loop with toolsTask is open-ended and exploratory
GuardrailsIteration limits, cost budgets, scope boundariesAlways — for any agent-like system

Up next: Tool Use & Function Calling — design tools that LLMs can use reliably and build robust function-calling interfaces.

PreviousWorkflow PatternsNextTool Use & Function Calling

Workflow Patterns

Master the three foundational workflow patterns — prompt chaining, routing, and parallelization — that form the building blocks of every AI system.

Tool Use & Function Calling

Design effective tools for AI agents, implement function calling with structured outputs, and connect agents to external APIs safely.

On this page

What you'll learn in this module
Pattern 4: Orchestrator-Workers
How it differs from parallelization
When to use it
Example: Codebase Refactoring Agent
The orchestrator prompt
Worker execution
Pattern 5: Evaluator-Optimizer
Two-LLM design
When to use it
Example: Code Review Loop
Designing effective evaluators
Pattern 6: Autonomous Agents
The agent loop
When to use autonomous agents
The critical guardrails
Example: Research Agent
Comparing the Advanced Patterns
Decision tree
What you now understand