Load faster-whisper with device auto-detect

Whisper is a transformer model, which means it needs compute. faster-whisper is a reimplementation on CTranslate2 that runs the same model with less memory and higher throughput. You get near-real-time transcription on CPU and GPU acceleration when it is available.

Whisper model loading

How faster-whisper picks the right compute backend.

backend/transcription.py
python
from faster_whisper import WhisperModel

class TranscriptionService:
    def __init__(self, whisper_model: str, llm_base_url: str, llm_api_key: str, llm_model: str):
        print(f"Loading Whisper model '{whisper_model}'...")
        self.whisper = WhisperModel(
            whisper_model,
            device='auto',        # Metal on Mac, CUDA on NVIDIA, CPU otherwise
            compute_type='int8',  # Smaller memory footprint, fast inference
        )

device="auto" is the magic. faster-whisper inspects your hardware and loads the best backend available. compute_type="int8" quantizes model weights so they fit in RAM and run fast on CPU.

base.en is a good default for English. It runs fast on CPU and produces readable output. Jump to small.en when you need better accuracy on noisy audio, and medium.en when you have a GPU and the extra latency is acceptable. Start small and upgrade only if you hit quality issues.

Quiz: Quiz

Loading practice…