LLM code review prompt

A free-form code review is useless for writing back to GitHub. You need structured output with a predictable shape: a summary plus a list of comments, each tied to a file and line. The prompt has two jobs: constrain the model to real issues, and force it to emit JSON that matches your schema.

review_engine.py
python
SYSTEM_PROMPT = """You are a senior code reviewer. You read unified diffs and \
produce concise, high-signal review comments. You only flag real issues: bugs, \
security problems, correctness gaps, missing error handling, unclear naming. \
You never nitpick formatting. Respond with JSON only."""

The system prompt constrains the reviewer persona. Saying "you never nitpick formatting" upfront saves you from five comments about whitespace on every PR. Ending with "Respond with JSON only" reduces the odds of prose wrapping around the payload.

review_engine.py
python
USER_PROMPT_TEMPLATE = """Review the following pull request diff and return a JSON object with this exact shape:

{{
  "summary": "one short paragraph",
  "comments": [
    {{"file": "path/to/file", "line": 42, "severity": "info|warning|critical", "suggestion": "..."}}
  ]
}}

Only include comments for real issues. If the diff looks clean, return an empty comments list.

PR title: {title}
Changed files: {files}

Unified diff:
```
{diff}
```"""

The user prompt bakes the JSON shape into the instruction with an example. Double braces escape the Python str.format curly braces. The explicit permission to return an empty comments list stops the model from inventing nitpicks when the diff is fine.

review_engine.py
python
import json
import re


def _extract_json(text: str) -> dict:
    """LLMs sometimes wrap JSON in prose or code fences. Strip both."""
    fence = re.search(r"```(?:json)?\s*(\{.*?\})\s*```", text, re.DOTALL)
    if fence:
        text = fence.group(1)
    start = text.find("{")
    end = text.rfind("}")
    if start == -1 or end == -1:
        raise ValueError("No JSON object found in LLM output")
    return json.loads(text[start : end + 1])

Even with strict prompts, models sometimes wrap JSON in prose or fence it with ```json. The extractor peels those layers off so you always parse the actual object. If nothing is found, you raise and let the caller return a safe fallback.

review_pr pipeline

From raw diff to a list of typed ReviewComment objects.

Quiz: Quiz

Loading practice…