Benchmark the three routes

A benchmark is the only way to back up the decision framework you learned earlier. Same URL, same question, three routes. Record latency, answer quality, and a rough cost note. Then write down the pattern you would ship for each class of question.

Benchmark harness

Feed one question and one URL, collect outputs from all three routes.

scripts/benchmark.ts
typescript
const url = 'https://lilianweng.github.io/posts/2023-06-23-agent/';
const question = 'What are the main components of an autonomous agent?';

async function hit(route: string, body: object) {
  const t0 = performance.now();
  const res = await fetch(`http://localhost:3000${'$'}{route}`, {
    method: 'POST',
    headers: { 'content-type': 'application/json' },
    body: JSON.stringify(body),
  });
  const text = await res.text();
  return { ms: performance.now() - t0, text };
}

const a = await hit('/api/chat', { messages: [{ role: 'user', content: question }] });
const b = await hit(`/api/rag?url=${'$'}{encodeURIComponent(url)}`, { prompt: question });
const c = await hit(`/api/rag2?url=${'$'}{encodeURIComponent(url)}`, { prompt: question });

console.table({
  chat: { latencyMs: a.ms.toFixed(0) },
  ragNative: { latencyMs: b.ms.toFixed(0) },
  ragCustom: { latencyMs: c.ms.toFixed(0) },
});

A quick and honest benchmark. This measures total wall time, which includes network, server processing, and streaming completion. Read the actual text outputs to compare faithfulness to the source.

You are right to notice. For a strict model comparison, align the model across all three routes and rerun. For a decision-making benchmark, leave each route on its best default so you see realistic performance. This workshop uses the second lens on purpose, because what you really want to know is how each pattern behaves when configured the way you would actually ship it.

Matching exercise: Predict each route's profile

Loading practice…

Quiz: Quiz

Loading practice…