When to pick each flavor
Every RAG decision is a tradeoff between speed, control, and cost. Direct chat is fastest but has no grounding. Native tools are fast and zero setup, but you inherit the provider's retrieval logic. A custom pipeline gives you full control at the cost of engineering time and latency.
Matching exercise: Match the use case to the route
Loading practice…
Direct chat (/api/chat)
Latency: lowest. One provider call.
Control: none over retrieval, because there is none.
Cost: one LLM call per turn.
Use when: the model already knows, or you do not need grounding.
Native tool RAG (/api/rag)
Latency: low to medium. Provider handles retrieval in the same call.
Control: limited. You pick the URL and the prompt, the provider picks everything else.
Cost: one LLM call, possibly with tool-use surcharges.
Use when: the content is reachable by URL and you trust the provider to retrieve well.
Custom pipeline RAG (/api/rag2)
Latency: highest. Scrape, split, embed, search, then generate.
Control: total. Chunking, embedding model, retrieval prompt, grounding.
Cost: embedding calls plus generation.
Use when: private data, custom chunking, or you need citations and tuning.Keep this in your head when you are designing any retrieval feature. Start as low on the ladder as the requirement allows.
Absolutely, and you probably should. A real product might use direct chat for small talk, the native urlContext tool when the user pastes a link, and a custom pipeline for questions about your own docs. The client picks the right route based on the input. Everything streams back through the same UI because the transport is the same.
Quiz: Quiz
Loading practice…
AI prompt: Try it: pick the right flavor
Loading practice…
Checkpoint: Flavors checkpoint
Loading practice…