Fallback when classifier confidence is low

A supervisor that always picks one specialist is a supervisor that occasionally picks the wrong one. When the dispatch prompt returns a tie or a low-probability answer, route the request to a generalist or a clarifying-question specialist instead. Confidence-aware routing keeps quality high.

Confidence gate before dispatch

Read the classifier scores. If the top score is well above the runner-up, dispatch. If not, route to a fallback that asks for clarification.

agents/supervisor.py

python

MARGIN_THRESHOLD = 0.15

def pick_specialist(scores: dict[str, float]) -> str:
    ranked = sorted(scores.items(), key=lambda kv: kv[1], reverse=True)
    top, runner_up = ranked[0], ranked[1]
    if top[1] - runner_up[1] < MARGIN_THRESHOLD:
        return 'clarify'
    return top[0]

Margin-based gating is cheap and effective. The threshold is tunable per workload. Falling back to a clarifying specialist keeps the conversation healthy.

Quiz: Quiz

Loading practice…