Swap the LLM provider via env vars

The ChatOpenAI class in langchain-openai talks to any OpenAI-compatible endpoint. That includes OpenAI itself, OpenRouter, Together AI, a local vLLM server, or any compatible gateway. Swapping is two env vars and a model name, with no change to the graph.

.env
bash
# Default: OpenAI
OPENAI_API_KEY=sk-...
# (no base_url needed, defaults to https://api.openai.com/v1)

# Alternative: OpenRouter, free tier available
# OPENAI_API_KEY=sk-or-v1-...
# OPENAI_BASE_URL=https://openrouter.ai/api/v1
# SYNTH_MODEL=google/gemini-2.5-flash-lite

# Alternative: local vLLM on your GPU box
# OPENAI_API_KEY=not-needed
# OPENAI_BASE_URL=http://localhost:8000/v1
# SYNTH_MODEL=meta-llama/Llama-3.1-8B-Instruct

The base_url override is all it takes to point at a different backend. Local development on a small model, production on a bigger one, same code.

mini-perplexity.ipynb
python
import os
from langchain_openai import ChatOpenAI

def make_llm(model_env: str, default_model: str, temperature: float, max_tokens: int):
    return ChatOpenAI(
        model=os.getenv(model_env, default_model),
        base_url=os.getenv("OPENAI_BASE_URL"),  # None means default OpenAI
        api_key=os.getenv("OPENAI_API_KEY"),
        temperature=temperature,
        max_tokens=max_tokens,
    )

synth_llm = make_llm("SYNTH_MODEL", "gpt-4o-mini", 0.0, 1200)
refine_llm = make_llm("REFINE_MODEL", "gpt-4o-mini", 0.1, 1000)

One helper, two LLM instances, each with its own model env var. When the base_url is unset, langchain-openai defaults to OpenAI. When it is set, it routes to your chosen gateway.

Quiz: Quiz

Loading practice…

AI prompt: Try it: pick the right model for each pass

Loading practice…