Background jobs that never get lost: the outbox pattern with BullMQ in NestJS

Most apps queue background work the same way: save something to the database, then add a job to a queue in Redis. Send the welcome email, report the usage, generate the export.

await this.db.insert(invoices).values(invoice);
await this.queue.add('send-receipt', { invoiceId: invoice.id });

Postgres and Redis cannot share a transaction, so there are two ways this goes wrong:

• The process stops between the two lines. The invoice is saved, and the receipt is never sent.
• The job is added inside a transaction that later rolls back. The worker picks it up and looks for an invoice that does not exist, or sends a receipt for one that was never saved.

The fix is old and reliable: the outbox pattern. Write the job to a table in the same transaction as the change, and let a separate loop move it to the queue. A job exists if and only if its change was saved.

This guide builds it in NestJS with Postgres, Drizzle and BullMQ.

1. The outbox is a table

export const jobs = pgTable('jobs', {
  id: uuid().primaryKey().defaultRandom(),
  type: text().notNull(),
  payload: jsonb().notNull(),
  /** pending, then queued once BullMQ has it. */
  status: text().notNull().default('pending'),
  maxAttempts: integer().notNull().default(5),
  runAt: timestamp({ withTimezone: true }).notNull(),
  /** Optional. A second job with the same key is dropped. */
  key: text().unique(),
  finishedAt: timestamp({ withTimezone: true }),
}, (t) => [index().on(t.status, t.runAt)]);

2. Define jobs with a schema

A job is a name and the shape of its payload. Put ids in the payload, not whole records: the handler loads what it needs when it runs.

export const sendReceipt = defineJob(
  'receipt.send',
  z.object({ invoiceId: z.uuid() }),
  { maxAttempts: 10 },
);

Check the payload twice: when the job is queued, so a bad one fails where it was written, and again when it runs. A job saved by last week’s code may no longer fit this week’s schema, and it should fail loudly rather than reach the handler.

3. Enqueue in the same transaction as the change

async enqueue<S extends z.ZodType>(
  tx: Transaction,
  job: JobDefinition<S>,
  payload: z.input<S>,
  options: { runAt?: Date; key?: string } = {},
): Promise<boolean> {
  const added = await tx.insert(jobs).values({
    type: job.type,
    payload: job.schema.parse(payload),
    maxAttempts: job.maxAttempts,
    runAt: options.runAt ?? new Date(),
    key: options.key,
  }).onConflictDoNothing({ target: jobs.key })
    .returning({ id: jobs.id });
  return added.length > 0;
}

And the invoice from the start becomes:

await this.db.transaction(async (tx) => {
  const [invoice] = await tx.insert(invoices)
    .values(data).returning();
  await this.jobs.enqueue(tx, sendReceipt,
    { invoiceId: invoice.id },
    { key: `receipt:${invoice.id}` });
});

If the transaction rolls back, the job was never there. If it commits, the job is saved, even if the process dies a millisecond later.

The key makes enqueuing safe to repeat: a second receipt:<id> is dropped by the unique constraint. Use one whenever the same event could queue the same work twice, like a webhook that arrives again.

Passing tx everywhere gets tedious. Keeping the open transaction in AsyncLocalStorage, so any repository joins it without being handed it, lets enqueue take just the job and its payload.

4. The relay moves jobs to BullMQ

A loop takes waiting rows, adds them to BullMQ, and marks them, all in one transaction:

async relay(): Promise<number> {
  return this.db.transaction(async (tx) => {
    const rows = await tx.select().from(jobs)
      .where(eq(jobs.status, 'pending'))
      .orderBy(asc(jobs.runAt))
      .limit(100)
      .for('update', { skipLocked: true });
    if (rows.length === 0) return 0;

    await this.queue.addBulk(rows.map((row) => ({
      name: row.type,
      data: row.payload,
      opts: {
        jobId: row.id,
        delay: Math.max(0, row.runAt.getTime() - Date.now()),
        attempts: row.maxAttempts,
        backoff: { type: 'custom' },
      },
    })));
    await tx.update(jobs)
      .set({ status: 'queued', finishedAt: new Date() })
      .where(inArray(jobs.id, rows.map((r) => r.id)));
    return rows.length;
  });
}

Each line here is doing a job:

• FOR UPDATE SKIP LOCKED lets several API instances run the relay at once. Each takes rows the others have not locked, so none waits and none takes the same row.
• jobId: row.id makes a double hand-over harmless. If the relay adds the jobs and then fails before marking them, the next pass adds them again, and BullMQ ignores a job id it already has.
• If Redis is down, addBulk throws, the transaction rolls back, and the rows stay pending for the next pass. Nothing is lost while Redis is away.
• Jobs for later go to BullMQ straight away with a delay, and BullMQ holds them until they are due.

Run the relay every second, and also right after a transaction that queued something commits, so most jobs start within milliseconds instead of waiting for the next tick.

5. Handlers: safe to repeat, and allowed to fail

A worker can do the work and then crash before telling BullMQ. The job runs again. So every handler must be safe to run twice: check whether the receipt was already sent, use upserts, pass idempotency keys to the APIs you call.

@OnJob(sendReceipt)
async send({ invoiceId }: PayloadOf<typeof sendReceipt>) {
  const invoice = await this.invoices.find(invoiceId);
  if (!invoice || invoice.receiptSentAt) return;
  await this.mail.send(receiptFor(invoice));
  await this.invoices.markReceiptSent(invoiceId);
}

To retry, throw. Do not catch an error just to log it, or BullMQ records the job as done. Space the retries out, so a service that is down for an hour is not hammered for that hour:

// 15s, 30s, 1m, 2m… at most an hour, give or take 15%
function retryDelay(attempts: number): number {
  const doubled = 15_000 * 2 ** Math.max(0, attempts - 1);
  const capped = Math.min(doubled, 60 * 60_000);
  return Math.round(capped * (0.85 + Math.random() * 0.3));
}

new Worker('jobs', run, {
  connection,
  concurrency: 10,
  settings: { backoffStrategy: retryDelay },
});

The randomness matters: when a provider comes back, a thousand jobs that failed together should not all retry in the same second. With 5 attempts this gives up after a few minutes. Work that must survive a longer outage, like an email, gets more attempts.

6. Schedules that run once, on any number of servers

Cron jobs have the opposite problem: run the timer on three servers and the nightly report goes out three times. The outbox already solves it. Queue the next occurrence of each schedule, with the time in the key:

async queueSchedules(now = new Date()) {
  for (const { job, cron } of this.schedules) {
    const next = cron.nextRun(now); // croner, in UTC
    if (!next) continue;
    await this.jobs.enqueue(job, {}, {
      runAt: next,
      key: `${job.type}:${next.toISOString()}`,
    });
  }
}

Every server runs this every minute. The first one to queue report.nightly:2026-10-16T03:00:00.000Z wins, and the rest are dropped by the key. When it runs, the next pass queues the one after.

7. Clean up, but not too soon

Rows that BullMQ has can be deleted, but keep them for a while first: as long as a row exists, its key still drops a repeat. A daily job that removes handed-over rows after a week, and failed ones after a month so you can look at them, keeps the table small.

Mistakes to avoid

• setTimeout, setInterval or an in-memory list for work that has to happen. A deploy or a crash and it is gone.
• Whole records in the payload. They go stale between queuing and running. Send the id.
• The database’s clock and the app’s. Pick one to decide when a job is due. Two clocks a few seconds apart make “run now” mean “run soon”.
• Assuming a job runs once. It runs at least once. Make the handler safe to repeat.

This is how SaaS Stack runs background work: emails, usage reports, scheduled cleanups. Its tests check that a job exists only if its transaction commits, that each job goes to exactly one of several workers, and that a schedule is queued once however many servers ask. The jobs docs show how to add one.