LLM picks applicable tasks

Forcing the caller to specify tasks works, but it pushes the decision onto clients who often do not know which NLP tasks matter for a given input. A router pattern flips this. A small, fast LLM call inspects the input and chooses the subset of tasks that actually apply. Long articles get summary. Short tweets get sentiment and emotion. Product reviews get entities and keywords.

agent/router.py
python
ROUTER_PROMPT = """Given the following text, choose which NLP tasks apply.
Respond with ONLY a JSON array of task names from this set:
["sentiment", "entities", "keywords", "summary", "emotion"].

Rules:
- Include "summary" only if the text is longer than 3 sentences.
- Include "entities" only if the text mentions people, places, products, or organizations.
- Always include "sentiment" unless the text is purely factual.
- Include "emotion" when the text expresses feelings, not just opinions.

Text:
\"\"\"{text}\"\"\"
"""


async def pick_tasks(text: str) -> list[str]:
    prompt = ROUTER_PROMPT.format(text=text)
    raw = await _ask(prompt, max_tokens=100)
    chosen = _extract_json(raw)
    if not isinstance(chosen, list):
        return ["sentiment", "summary"]
    return [t for t in chosen if t in VALID_TASKS] or ["sentiment", "summary"]

The router is just another focused LLM prompt, with rules that describe when each task applies. Max_tokens=100 keeps it cheap. The fallback to ["sentiment", "summary"] protects against malformed router output.

agent/graph.py
python
async def run_agent(text: str, tasks: List[str]) -> Dict:
    """Run all requested tasks and return a merged result dict."""
    tasks = _validate_tasks(tasks)
    results: Dict = {"text": text, "tasks": tasks, "errors": {}}

    if HAS_LANGGRAPH:
        results.update(await _run_with_langgraph(text, tasks))
    else:
        results.update(await _run_plain(text, tasks))

    return results

run_agent is the single entry point into the agent graph. It validates the task list, then picks the LangGraph path when the library is installed and the plain-async path when it is not. Either way the caller gets back one merged dict of task results plus the errors map.

service.py
python
def _shape_response(text: str, tasks: list[str], raw: dict) -> AnalyzeResponse:
    """Map the agent's raw dict into the typed response model."""
    sentiment = None
    if "sentiment" in raw:
        s = raw["sentiment"]
        sentiment = SentimentResult(
            label=s.get("label", "neutral"),
            score=float(s.get("score", 0.0)),
            rationale=s.get("rationale"),
        )

    entities = None
    if "entities" in raw:
        entities = [Entity(text=e["text"], type=e.get("type", "OTHER")) for e in raw["entities"]]

    emotion = None
    if "emotion" in raw:
        em = raw["emotion"]
        emotion = EmotionResult(
            primary=em.get("primary", "neutral"),
            confidence=float(em.get("confidence", 0.5)),
        )

    return AnalyzeResponse(
        text=text,
        tasks=raw.get("tasks", tasks),
        sentiment=sentiment,
        entities=entities,
        keywords=raw.get("keywords"),
        summary=raw.get("summary"),
        emotion=emotion,
        errors=raw.get("errors", {}),
    )

The raw dict from run_agent is loose by design. _shape_response is where it becomes a typed AnalyzeResponse. Present tasks get their Pydantic model constructed, absent tasks stay None, and the errors map passes straight through. This keeps the service layer thin and the boundary contract explicit.

service.py
python
async def analyze(request: AnalyzeRequest) -> AnalyzeResponse:
    tasks = request.tasks
    if not tasks or tasks == ["auto"]:
        tasks = await pick_tasks(request.text)

    raw = await run_agent(request.text, tasks)
    return _shape_response(request.text, tasks, raw)

The caller can still specify tasks explicitly. Passing ["auto"] or an empty list triggers the router. This keeps the simple case simple and adds the smart case as an opt-in.

The router prompt is tiny and the output is capped at 100 tokens. It costs a fraction of a single task. If the router then skips three tasks that did not apply, you save three full LLM calls. For a typical short input that only needs sentiment and emotion, the router plus two tasks beats running all five every time. For long inputs where every task applies, the router adds a small overhead, which is the right tradeoff.

Router decision flow

The router sits in front of the agent graph and trims the task list.