LLM Engineering8 min read
Choosing the LLM judge for evaluation pipelines
How to pick the LLM that grades your LLM. The cost-quality tradeoffs, the calibration check, and why a weaker judge is sometimes the right call.
Build Your First Software Factory Execution Harness live Thursday, Oct 1, 12:00 PM ET
Oct 1, Reserve a seatTopic
LLM
Explore our latest articles and insights about LLM.
Explore posts
28 posts in total
LLM Engineering8 min read
Hallucination testing for RAG pipelines
How to test a RAG pipeline for hallucinations systematically. Adversarial prompts, the out-of-scope set, and the metric that catches confabulation.
LLM Engineering8 min read
Fact-checking RAG answers: grounding with verification
How to fact-check RAG answers with a second LLM pass that verifies every claim against the retrieved context. The prompt, the rejection rule, and the loop.
LLM Engineering8 min read
Query rewriting in RAG with LLMs: the rewrite loop
How LLM-powered query rewriting fixes vague user questions before retrieval. The prompt, the multi-query fan-out, and when rewriting hurts more than helps.
LLM Engineering8 min read
LLM-based content filtering for RAG pipelines
How to filter irrelevant retrieved chunks with a cheap LLM call before the final answer. The prompt, the batch pattern, and the 40 percent noise reduction.
AI Engineering in Practice9 min read
Agent cost optimization from trace data
How to use Langfuse trace data to find where your agent burns tokens. The 4 queries, the cost-per-user view, and the 50 percent savings patterns.
LLM Engineering9 min read
LLM judges: enforcing reasoning with explicit rationales
Why LLM judges without explicit reasoning drift, and how chain-of-thought rationales make their scores defensible. The prompt, the parser, the trust.
LLM Engineering9 min read
LLM-as-a-judge: production evaluation framework for agents
How to build an LLM-as-a-judge evaluation framework for agentic AI. The prompt, the rubric, the bias controls, and the loop that catches regressions.
AI Engineering11 min read
Circuit breakers for LLM calls: stop cascading failures
How circuit breakers prevent LLM outages from cascading through your agent. The 3 states, the failure window, and the 50-line implementation.
AI Engineering in Practice9 min read
Resilient LLM services with Tenacity and fallback models
How to survive LLM provider outages with Tenacity retries and fallback models. The retry policy, the fallback chain, and the 60-line pattern.
AI Engineering11 min read
Context window management for production AI agents
How to manage context windows in production AI agents. The 4 strategies that keep long sessions bounded without losing critical context.
LLM Engineering11 min read
Chain-of-thought reasoning in RAG: a practical guide
How to add chain-of-thought reasoning to a RAG pipeline. The prompt, the parsing, and the cases where CoT beats a straight answer by a wide margin.