Retry & Dead-Letter Queues
Bounded retries, delayed retry, and poison-pill handling
For production consumers, prefer bounded retries with a dead-letter queue. Rabbit Relay keeps every part of that pipeline explicit and broker-native.
The deadLetter helper wires DLX, DLQ, binding, and queue arguments in one place.
const sub = await broker
.queue("orders.q")
.exchange("orders.ex", {
exchangeType: "topic",
routingKey: "orders.*",
deadLetter: {
exchange: "orders.dlx",
queue: "orders.dlq",
routingKey: "orders.dead",
autoDeclare: true,
},
});One key, two places
deadLetter.routingKey sets x-dead-letter-routing-key on the source
queue and, with autoDeclare: true, is used for the auto-declared
DLQ→DLX binding. Omitting it preserves original routing keys and binds
the DLQ to "#". Never change one side without the other, or
dead-lettered messages are silently dropped.
await sub.consume({
prefetch: 10,
concurrency: 5,
onError: "retry",
retry: {
attempts: 3,
delayMs: 5000,
backoff: "exponential",
then: "dead-letter",
},
});- Immediate retry (no
delayMs): re-delivers right away, good for deterministic in-process failures. - Fixed delay (
delayMs): parks the message in a broker-native TTL queue. - Exponential backoff (
backoff: "exponential"): attempt n waitsdelayMs * 2^(n-1), good for transient outages like downstream 429/503 or DB locks.
Dead-lettered messages arrive as regular events on the DLQ consumer:
const dlq = await broker
.queue("orders.dlq")
.exchange<{ orderCreated: EventEnvelope<OrderCreated> }>("orders.dlx", {
exchangeType: "topic",
routingKey: "orders.dead",
topologyMode: "passive",
});
dlq.handle("*", async (_id, ev) => {
// inspect ev.meta.headers for x-rabbit-relay-retry-count,
// x-rabbit-relay-last-error, first/last failed timestamps
});Retry headers
Every retry copy carries observability headers:
| Header | Meaning |
|---|---|
x-rabbit-relay-retry-count | Attempt number (1-based) |
x-rabbit-relay-retry-delay-ms | Actual wait scheduled before this attempt |
x-rabbit-relay-first-failed-at | ISO timestamp of the first failure |
x-rabbit-relay-last-failed-at | ISO timestamp of the latest failure |
x-rabbit-relay-last-error | Last error message (truncated to 500 chars) |
What happens on exhaustion
After attempts retries, the final action runs:
retry.then | Behavior |
|---|---|
"dead-letter" (default) | Message goes to the DLQ via nack(requeue=false) |
"requeue" | Message returns to the queue, use with care, can loop |
"ack" | Message is acknowledged and dropped, with a message.dropped lifecycle event |
Auto-declare mode
When autoDeclare: true, Rabbit Relay declares:
- the dead-letter exchange
- the dead-letter queue
- the binding from DLQ to DLX
- the main queue dead-letter arguments
deadLetter.routingKey drives two things at once:
- It is set as
x-dead-letter-routing-keyon the source queue, so every dead-lettered message is republished to the DLX with this key. - The auto-declared DLQ→DLX binding uses the same key.
The two sides always agree when both come from relay. If you change one of them outside relay (for example re-binding the DLQ yourself), dead-lettered messages can be silently dropped, published to the DLX with a key nothing listens for. If you manage bindings externally, keep autoDeclare: false and mirror the key on your own binding.
Silent drop warning
Never change one side of the routing key without the other, or dead-lettered messages are silently dropped.
If routingKey is omitted, RabbitMQ preserves each message's original routing key and relay binds the DLQ to "#" so all keys are caught.
deadLetter: {
exchange: "orders.dlx",
queue: "orders.dlq",
routingKey: "orders.dead",
autoDeclare: true,
}External infrastructure mode
If your team manages RabbitMQ topology using Terraform, Helm, or another setup process, keep autoDeclare false or omit it.
deadLetter: {
exchange: "orders.dlx",
routingKey: "orders.dead",
}Rabbit Relay will configure the main queue with DLQ arguments, but it will not create the DLX/DLQ.
Since relay also does not create the DLQ→DLX binding in this mode, your own binding must use the same key as deadLetter.routingKey (or "#" to catch all keys).
Using DLQ with consumer errors
await sub.consume({
onError: "dead-letter",
});If the handler throws, Rabbit Relay calls:
nack(requeue=false)RabbitMQ then routes the message to the configured DLQ.
Using DLQ after retries
await sub.consume({
onError: "retry",
retry: {
attempts: 3,
delayMs: 5000,
then: "dead-letter",
},
});After retries are exhausted, the message is dead-lettered.
DLQ redrive
After the root cause of failures is fixed, you can replay DLQ messages back to a target exchange.
CLI
# Inspect queue depth
rabbit-relay dlq inspect orders.dlq --url amqp://localhost
# Peek at messages without consuming them
rabbit-relay dlq peek orders.dlq --limit 5 --url amqp://localhost
# Dry-run redrive (safety check)
rabbit-relay dlq redrive orders.dlq orders.ex --dry-run --url amqp://localhost
# Redrive with a limit
rabbit-relay dlq redrive orders.dlq orders.ex --limit 50 --url amqp://localhostProgrammatic
const result = await broker.redriveDlq({
fromQueue: "orders.dlq",
toExchange: "orders.ex",
routingKey: "orders.created",
limit: 100,
});You can also call it from a broker interface:
await sub.redriveDlq({
fromQueue: "orders.dlq",
toExchange: "orders.ex",
routingKey: "orders.created",
limit: 50,
});Dry-run first
Always dry-run before redriving in production.
const result = await broker.redriveDlq({
fromQueue: "orders.dlq",
toExchange: "orders.ex",
routingKey: "orders.created",
limit: 100,
dryRun: true,
});Dry-run checks queue depth without consuming, publishing, or ACKing messages.
Result shape
type DlqRedriveResult = {
fromQueue: string;
toExchange: string;
routingKey?: string;
dryRun: boolean;
available: number;
attempted: number;
republished: number;
acked: number;
failed: number;
empty: boolean;
errors: Array<{ message: string; error?: unknown }>;
};Safety behavior
Rabbit Relay redrive is intentionally conservative:
- bounded by
limit - supports
dryRun - preserves message body and AMQP properties
- adds redrive headers
- ACKs the original DLQ message only after successful republish
- requeues the original DLQ message if republish fails
Redrive headers
x-rabbit-relay-redrive-count
x-rabbit-relay-redriven-at
x-rabbit-relay-redriven-from-queue
x-rabbit-relay-redriven-to-exchange
x-rabbit-relay-redriven-routing-keyThese are visible in event.meta.headers when the redriven message is consumed.
Recommended operation flow
- Find and fix the root cause
- Start the normal consumer
- Dry-run redrive
- Redrive a small limit
- Watch logs and metrics
- Increase limit gradually if needed
Consumers must still be idempotent - redrive does not guarantee the message will succeed after replay.
Important note about existing queues
RabbitMQ queue arguments are immutable.
If a queue already exists without DLQ arguments, declaring it again with DLQ arguments may fail with a precondition error.
Fix by:
- deleting/recreating the queue in development
- using a new queue name/version
- managing topology externally and using
passiveQueue: true
Common mistakes
Infinite requeue
Avoid using onError: "requeue" as a retry strategy.
It can create a hot loop.
No DLQ
If messages matter, configure a DLQ.
Redrive before fixing the bug
If the consumer is still broken, redrive only fails again.
Large redrive without dry-run
Always dry-run and start with a small limit.
Summary
- DLQs isolate poison messages
- Rabbit Relay can configure DLQ arguments for you
autoDeclare: truecreates DLX/DLQ topology- Use DLQ with retry for production-safe failure handling
- Use
redriveDlq()to safely replay messages after fixes