The one-function contract

A provider abstraction is not a framework. It is one function that every backend implements the same way. Pick the surface well and the rest of the course writes itself.

The chat() seam

Application code talks to one function. The function routes to the right backend based on the model string.

utils/llm_provider.py
python
from abc import ABC, abstractmethod
from typing import AsyncIterator, Literal
from pydantic import BaseModel

Role = Literal['system', 'user', 'assistant']

class Message(BaseModel):
    role: Role
    content: str

class ChatResult(BaseModel):
    text: str
    model: str
    prompt_tokens: int | None = None
    completion_tokens: int | None = None

class LLMProvider(ABC):
    @abstractmethod
    async def chat(self, messages: list[Message], model: str) -> ChatResult:
        ...

    @abstractmethod
    async def stream(self, messages: list[Message], model: str) -> AsyncIterator[str]:
        ...

Each piece of the contract pulls its weight. Message is a neutral shape. ChatResult normalises the response. The abstract base class forces every backend to implement both chat and stream.

You can, and many libraries do. Two methods keeps the types honest so callers who only want a string never accidentally iterate a coroutine. If you prefer one method with a stream flag, the contract still holds. What matters is that the signature is the same for every provider.

Quiz: Quiz

Loading practice…