The sliding-window limiter

Rate limiting protects two things: OpenRouter costs and the experience of honest users when one account goes wrong. We apply the limit per username so a single abusive session cannot affect others.

The naive approach is a fixed-window counter: "10 requests per calendar minute". It has a classic flaw: a user can burst 10 requests at 12:00:59 and 10 more at 12:01:00 for a real rate of 20 requests in two seconds. The sliding window avoids that by tracking the actual timestamps and discarding any older than the window length.

Sliding vs fixed window

A fixed window lets the user burst at the boundary. A sliding window tracks real timestamps and refuses cleanly.

Sliding window cost is O(active timestamps). For 10 requests per minute that is ten timestamps per user, max.
rate_limiter.py
python
class RateLimiter:
    def __init__(self, window_seconds=60, max_requests=10) -> None:
        self.window = window_seconds
        self.max_requests = max_requests
        self.user_requests: dict[str, list[float]] = defaultdict(list)

    def check_rate_limit(self, username: str) -> bool:
        now = time.time()
        window_start = now - self.window

        self.user_requests[username] = [
            ts for ts in self.user_requests[username] if ts > window_start
        ]

        if len(self.user_requests[username]) >= self.max_requests:
            return False

        self.user_requests[username].append(now)
        return True

Every call drops expired timestamps, counts what is left, and either appends the new timestamp or refuses. No rolling buckets, no mod-arithmetic.

Quiz: Quiz

Loading practice…

Validation checklist: Verify the limiter

Loading practice…