Async queues, done right

Your platform sends confirmation emails, generates PDFs, and syncs data to external systems. All of that belongs in a queue. Building the queue is easy. Building one you can trust with real money flowing through it is the interesting part.

Solution A is the simplest BullMQ setup. A queue, a worker, a handler function. Failed jobs retry a few times and then give up. Good enough for emails and non-critical work.

Solution B adds three habits. Exponential backoff so retries space out instead of hammering. A dead letter queue that catches permanently failed jobs so they can be investigated. And idempotency keys so replaying a job never causes duplicate side effects. That last one is the most underrated piece.

solution-2-reliable/worker.ts
typescript
new Worker(
  'payments',
  async (job) => {
    const { idempotencyKey, amount } = job.data;

    // Idempotency: refuse to run the same logical operation twice
    const already = await db.processedJobs.findOne({ key: idempotencyKey });
    if (already) return already.result;

    const result = await chargeCard(amount);
    await db.processedJobs.insert({ key: idempotencyKey, result });
    return result;
  },
  {
    connection,
    removeOnComplete: { count: 1000 },
    removeOnFail: { count: 1000 },
  },
);

// Dead letter handler
worker.on('failed', async (job, err) => {
  if (job?.attemptsMade >= job!.opts.attempts!) {
    await deadLetterQueue.add('failed', { job: job!.toJSON(), error: err.message });
  }
});

Backoff, dead letter queue, and idempotency key check.

Quiz: Quiz

Loading practice…

Checkpoint: Data infrastructure checkpoint

Loading practice…