The polished story

Raw LLM output is messy. Streaming chunks sometimes have spacing issues, punctuation errors, or formatting inconsistencies. A production application needs a processing pipeline that cleans the output before it reaches the user.

The processing pipeline

Pre-processing happens before the LLM call, post-processing happens on the streamed chunks.

utils/llm_provider.py
python
def _fix_streaming_chunk_spacing(chunk: str) -> str:
    """
    Fix spacing and punctuation issues in streaming chunks.

    Some LLM providers return chunks without proper spacing:
    - Numbers followed by words: "123abc" -> "123 abc"
    - Missing spaces after punctuation: "word,word" -> "word, word"
    """
    # Fix missing space after punctuation
    import re
    chunk = re.sub(r'([.,!?;:])([A-Za-z])', r'\1 \2', chunk)
    # Fix numbers running into words
    chunk = re.sub(r'(\d)([A-Za-z])', r'\1 \2', chunk)
    return chunk

This runs on every streamed chunk before it reaches the client. Regex-based fixes handle the most common spacing issues across providers.

Yes. Even the best models produce formatting inconsistencies when streaming. Chunks can split in the middle of words, punctuation marks, or numbers. The post-processing layer is lightweight (regex on each chunk) and makes a noticeable difference in output quality. Think of it as the same reason you lint your code even when you write it carefully.

Ordering exercise: Order the processing pipeline steps

Loading practice…

Quiz: Quiz

Loading practice…