LLM Engineering8 min read
Automated evaluation pipelines for agentic AI systems
How to wire eval pipelines into CI so every agent change is scored automatically. The nightly job, the regression gate, and the dashboard that matters.
Loading...
Loading...
Build Your First Software Factory Execution Harness live Thursday, Oct 1, 12:00 PM ET
Oct 1, Reserve a seatDeep dives into Large Language Models - how they work, prompt engineering, behavior control, and optimization techniques.
31 posts in total
LLM Engineering8 min read
How to wire eval pipelines into CI so every agent change is scored automatically. The nightly job, the regression gate, and the dashboard that matters.
LLM Engineering8 min read
How to load evaluation metrics dynamically in a Python eval pipeline. The registry pattern, entry points, and the test override that makes CI fast.
LLM Engineering9 min read
Why LLM judges without explicit reasoning drift, and how chain-of-thought rationales make their scores defensible. The prompt, the parser, the trust.
LLM Engineering9 min read
How to build an LLM-as-a-judge evaluation framework for agentic AI. The prompt, the rubric, the bias controls, and the loop that catches regressions.
LLM Engineering11 min read
Why linear LangChain chains fall over on real agents and how LangGraph's stateful graphs replace them. The state model, loops, and upgrade path.
LLM Engineering11 min read
How to use sub-graphs in LangGraph to keep complex RAG pipelines sane. The composition pattern, the state isolation rule, and when to split.
LLM Engineering10 min read
How to render a RAG pipeline as a graph with LangGraph StateGraph. The diagram, the state schema, and the debugging workflow that saves hours.
LLM Engineering11 min read
How to add chain-of-thought reasoning to a RAG pipeline. The prompt, the parsing, and the cases where CoT beats a straight answer by a wide margin.
LLM Engineering11 min read
How to pick an embedding model for production RAG. The 5 criteria that matter, the benchmarks that lie, and the migration cost nobody warns you about.
LLM Engineering11 min read
How RecursiveCharacterTextSplitter works, why it beats naive chunking, and the separator order that makes or breaks retrieval quality.
LLM Engineering11 min read
How hybrid retrieval combines vector search and graph traversal in RAG. The when, the why, and the 60-line fusion that beats either alone.
LLM Engineering11 min read
How to use RAGAS to evaluate RAG pipelines. The 4 metrics that matter, the eval loop, and the trap that makes most RAG evals dishonest.