The lock-in problem

Hey, I am Param. In this course we build a thin interface that sits between your app and any LLM provider. By the end you will flip one environment variable to run the same chat service on OpenAI, OpenRouter, Gemini, or a local Ollama model, with fallback and real observability.

Imagine your team ships a working chat feature on GPT-4o-mini. Two weeks later, Gemini 2.5 Flash becomes cheaper and faster for your workload. Your PM asks for a two-day A/B test. You open the code and realise every service file calls the OpenAI SDK directly.

before/service.py
python
from openai import AsyncOpenAI

client = AsyncOpenAI(api_key=OPENAI_KEY)

async def answer(question: str) -> str:
    response = await client.chat.completions.create(
        model='gpt-4o-mini',
        messages=[{'role': 'user', 'content': question}],
        temperature=0.3,
    )
    return response.choices[0].message.content

async def summarise(text: str) -> str:
    response = await client.chat.completions.create(
        model='gpt-4o-mini',
        messages=[
            {'role': 'system', 'content': 'Summarise in one sentence.'},
            {'role': 'user', 'content': text},
        ],
    )
    return response.choices[0].message.content

Two small functions, but the OpenAI client leaks into both. Every new feature copies this shape. That is how a single SDK becomes a dependency on every layer of the app.

The model string is the easy part. The hard parts are the client object (different SDK per vendor), the message format (Gemini has contents and parts, not messages), the auth flow, and the error shapes. A rename never works. A seam always does.

Quiz: Quiz

Loading practice…