State machine: tool, retrieve, reply

The orchestrator is a tiny state machine. Given a validated request, it picks one of three branches: run a tool, retrieve grounding context, or reply directly. That is it. No prompt engineering inside the orchestrator, no LLM calls, no side effects. Just a pure function that reads the message and returns a route.

The three branches of the orchestrator

A message either triggers a tool, asks a factual question, or falls through to a direct reply.

layers/orchestrator.py
python
TOOL_TRIGGER_PATTERNS = [
    re.compile(r"\b\d+\s*[\+\-\*/xX]\s*\d+"),
    re.compile(r"\bwhat\s+time\b", re.I),
    re.compile(r"\bcurrent\s+time\b", re.I),
    re.compile(r"\bcalculate\b", re.I),
]

RETRIEVE_KEYWORDS = [
    "what is", "who is", "explain", "define", "how does", "tell me about",
]


class OrchestratorLayer:
    def initial_state(self, message: str, thread_id: str) -> Dict[str, Any]:
        return {
            "message": message,
            "thread_id": thread_id,
            "route": None,
            "tool_result": None,
            "retrieved": [],
            "reply": None,
        }

    def route(self, state: Dict[str, Any]) -> str:
        """Pure decision node - inspect the message and pick a branch."""
        msg = state["message"]
        lower = msg.lower()

        for pat in TOOL_TRIGGER_PATTERNS:
            if pat.search(msg):
                state["route"] = "tool_use"
                return "tool_use"

        for kw in RETRIEVE_KEYWORDS:
            if kw in lower:
                state["route"] = "retrieve"
                return "retrieve"

        state["route"] = "reply"
        return "reply"

Notice there is no LLM call. Routing is a cheap, deterministic function. When you swap the rules for a classifier model later, the interface stays the same: state in, route string out.

A routing call adds latency and cost on every request. The rule-based decision is deterministic and testable: given the same message, you always get the same route. The orchestrator also keeps a crisp interface, so when you upgrade to a classifier model later you replace the body of route() and every test still applies. Start simple, measure, upgrade when data tells you to.

Quiz: Quiz

Loading practice…