Zero-shot sentiment baseline

Welcome! I'm Param. In this course we are going to build a multi-task NLP service where every task is a focused LLM prompt. Sentiment, entities, keywords, summary, emotion. Each one lives behind the same interface, each one streams progress back over Server-Sent Events, and each one can be swapped or upgraded by editing a single prompt string.

Classic NLP pipelines glue together specialised models. One for sentiment, one for entities, one for summarisation. Each model ships with its own training data and its own labels. The moment product asks for a new label, you are back in a retraining loop. LLM-per-task flips that. The labels live in the prompt. Changing them takes seconds.

The service we are building

A FastAPI router calls a service that dispatches tasks to focused LLM prompts. Results either come back as one response, or stream back per task over SSE.

terminal
bash
# Clone the workshop repo
git clone https://github.com/learnwithparam/sentiment-nlp-agents.git
cd sentiment-nlp-agents

# Set up and run in one command
make dev

The Makefile creates a .env file, installs dependencies with uv, and starts the FastAPI server on port 8000.

agent/tasks.py
python
from utils.llm_provider import get_llm_provider

_llm = None


def _get_llm():
    global _llm
    if _llm is None:
        _llm = get_llm_provider()
    return _llm


async def _ask(prompt: str, max_tokens: int = 400) -> str:
    llm = _get_llm()
    return await llm.generate_text(prompt, temperature=0.2, max_tokens=max_tokens)

Before writing any task, meet the one helper every task will call. get_llm_provider reads your .env and returns the configured provider, OpenRouter by default with Fireworks, Gemini, and OpenAI supported. _ask sends the prompt to that LLM and returns the raw text reply. Low temperature keeps classification output consistent between runs.

agent/tasks.py
python
async def classify_sentiment(text: str) -> dict:
    """Return {label, score, rationale}."""
    prompt = f"""You are a sentiment classifier. Analyze the following text and
respond with ONLY a JSON object in this exact shape (no prose, no markdown):

{{"label": "positive" | "negative" | "neutral",
  "score": number between -1.0 and 1.0,
  "rationale": "one short sentence"}}

Text to analyze:
\"\"\"{text}\"\"\"
"""
    raw = await _ask(prompt, max_tokens=200)
    data = _extract_json(raw)
    label = str(data.get("label", "neutral")).lower()
    if label not in ("positive", "negative", "neutral"):
        label = "neutral"
    try:
        score = float(data.get("score", 0.0))
    except (TypeError, ValueError):
        score = 0.0
    score = max(-1.0, min(1.0, score))
    return {"label": label, "score": score, "rationale": data.get("rationale")}

The zero-shot prompt is specific about shape, range, and format. Notice the triple-quote wrapper around the text. It protects against accidental prompt injection when the user pastes content with their own JSON or instructions. _extract_json safely pulls the JSON object out of the model reply, and you will build it yourself in a later lesson. The closing lines normalise the label and clamp the score into the -1 to 1 range so a wild model value never leaks past this function.

Two reasons. First, asking the model to justify its answer is a lightweight form of chain-of-thought that often improves the label itself. Second, the rationale is the first thing you inspect when a prediction looks wrong. Without it, debugging a misclassification means staring at raw text and guessing. With it, you see the reasoning and can tweak the prompt.

Quiz: Quiz

Loading practice…

Validation checklist: Zero-shot baseline checklist

Loading practice…