The link unfurler

A chat app wants to show a preview card whenever a user posts a URL. Your job is to accept a URL, fetch the page, parse the open graph tags, and return the title, description, and image. Easy in principle, brutal under load.

Two different traffic profiles drive two different solutions. Low traffic: one request at a time, first-time fetches are fine. High traffic: the same URL gets unfurled a thousand times a minute and the upstream target starts rate-limiting you. Same feature, different engineering.

solution-1-low-traffic/unfurl.ts
typescript
import * as cheerio from 'cheerio';

app.get('/unfurl', async (req, res) => {
  const url = req.query.url as string;
  const html = await fetch(url).then((r) => r.text());
  const $ = cheerio.load(html);
  res.json({
    title: $('meta[property="og:title"]').attr('content') || $('title').text(),
    description: $('meta[property="og:description"]').attr('content'),
    image: $('meta[property="og:image"]').attr('content'),
  });
});

Simple version: fetch, parse, return. No caching.

solution-2-high-traffic/unfurl.ts
typescript
app.get('/unfurl', async (req, res) => {
  const url = req.query.url as string;
  const cacheKey = 'unfurl:' + url;

  const cached = await redis.get(cacheKey);
  if (cached) return res.json(JSON.parse(cached));

  // Stampede protection: only one request for this URL does the fetch
  const lockKey = 'lock:' + cacheKey;
  const gotLock = await redis.set(lockKey, '1', 'NX', 'EX', 5);
  if (!gotLock) {
    // Wait briefly, then read the cache
    await new Promise((r) => setTimeout(r, 100));
    const late = await redis.get(cacheKey);
    if (late) return res.json(JSON.parse(late));
  }

  const data = await doUnfurl(url);
  await redis.set(cacheKey, JSON.stringify(data), 'EX', 3600);
  res.json(data);
});

High-traffic version: Redis cache with stampede protection via a short lock.

The interesting word above is stampede. Without the lock, a thousand concurrent requests for the same uncached URL each trigger their own fetch, the upstream throttles you, and the cache fills up a thousand times. The lock ensures only the first request does the work. Everything else waits briefly and reads the cache. This is the kind of detail senior interviewers love to discuss.

One more refinement worth knowing. The cache above uses a plain TTL: the entry lives for an hour, then the next unlucky reader pays for a fresh fetch. The gentler pattern is stale-while-revalidate: once the entry passes its TTL, serve the stale copy immediately and trigger a background refresh. Readers never wait on the upstream, the cache converges to fresh data, and the TTL becomes the worst-case staleness you are willing to accept. For preview cards, a slightly old title beats a slow response every time.

Quiz: Quiz

Loading practice…