Cost and latency budget for each flavor
Tradeoffs feel real once you put numbers on them. Each flavor has a different shape of cost and a different shape of latency. Walking through a back-of-the-envelope example for each makes the right choice obvious for your product.
cost-and-latency-table
text
Flavor Per-question cost Time to first token Notes
-------------------------- ------------------ -------------------- --------------------------------
Streaming chat (no RAG) ~1x base tokens low No grounding, fastest perceived
Native-tool RAG (urlContext) ~1.3x base tokens medium Provider does retrieval for you
Custom pipeline RAG ~1.5x to 2x medium to high You pay for embeddings + scrapeNumbers are illustrative, not benchmarks. The shape is what matters: native-tool sits in the middle, custom pipeline trades cost and latency for control.
Three rules of thumb. Pick streaming chat when freshness matters less than feel. Pick native-tool RAG when the URL is the source of truth and the provider already supports it. Pick the custom pipeline when you need full control over chunking, ranking, or persistence across questions.
Quiz: Quiz
Loading practice…