Redis vs RabbitMQ for Work Order Queuing: Choosing a Message Broker

Every intake adapter that accepts a maintenance request faster than the CMMS can commit it needs somewhere to put the backlog, and that decision happens once, early, and is expensive to reverse. Async batch processing already assumes a durable queue sits between intake and dispatch — this page answers the question that guide defers: which broker. The choice matters because it is not really about throughput; both Redis and RabbitMQ can move more messages per second than any CMMS API can absorb. It is about what happens at the edges — a burst that outpaces the CMMS commit rate, a worker that crashes mid-write, a poison message that fails validation on every retry — and which broker’s native behavior makes those edges easy or hard to handle correctly. Get this wrong and you either rebuild acknowledgment and dead-lettering by hand on top of a broker that never intended to provide them, or you carry the operational weight of a full AMQP broker for a workload that never needed one.

This decision sits squarely inside the broader work order ingestion and parsing pipelines domain: every producer that feeds the queue — a scheduled PM sweep, a webhook intake adapter, an NLP-routed email — hands off a work order the moment it is validated, and the broker is the only thing standing between that handoff and a silent loss if the CMMS commit fails. The rest of this page walks the criteria that actually separate Redis and RabbitMQ for this workload, gives you runnable Python for both, and ends with a recommendation keyed to scenario rather than to brand preference.

Decision Criteria

Seven criteria decide this, and they interact — a broker that wins on throughput can lose on operational burden, and a broker with native dead-lettering can still lose on routing flexibility.

Delivery guarantees and acknowledgment. Both brokers support at-least-once delivery, but they get there differently. Redis Streams use consumer groups: a message is claimed with XREADGROUP, stays in a per-consumer pending entries list until XACK confirms it, and an unacknowledged entry can be reclaimed by another consumer with XCLAIM. RabbitMQ uses per-message acknowledgment on a channel: basic_ack confirms, basic_nack or basic_reject fails it explicitly, and an unacked message returns to the queue (or its dead-letter target) the moment the consumer’s connection drops. RabbitMQ’s ack model is older, more granular, and enforced at the protocol level; Redis Streams’ consumer-group model is newer and requires your application to track delivery counts itself.

Built-in dead-letter handling. This is the sharpest functional difference. RabbitMQ has first-class dead-lettering: declare a queue with an x-dead-letter-exchange argument, and any message that is rejected, expires, or overflows the queue is automatically republished to that exchange — no application code required. Redis has no equivalent primitive. A dead-letter queue against a Redis stream is something you build yourself, typically by inspecting XPENDING for entries that exceed a retry ceiling and re-adding them to a separate stream with XADD. It works well, but it is your code, not the broker’s guarantee.

Routing and topologies. RabbitMQ’s exchange types — direct, topic, fanout, headers — let one publish reach multiple queues based on routing keys, which matters once work orders need to fan out to a priority queue, an audit log consumer, and a metrics sink from a single publish. Redis Streams give you one ordered log per stream; multiple consumer groups can read the same stream independently, which covers fan-out for read-only consumers, but there is no routing-key logic — every consumer group sees every entry and filters in application code.

Throughput and latency. Redis, as an in-memory data structure server, answers commands in sub-millisecond time and comfortably outpaces RabbitMQ on raw enqueue/dequeue latency, because RabbitMQ’s protocol overhead and broker-side routing logic add real cost per message. For work-order volumes — hundreds to low thousands of messages per second at a single facility group — both are far beyond what the CMMS commit path can consume, so this criterion rarely decides the choice by itself.

Persistence and durability. Redis persists via AOF (append-only file) or RDB snapshots; a stream entry survives a restart only if persistence is configured and the fsync policy tolerates the loss window you can accept. RabbitMQ durability is declared per queue and per message (delivery_mode=2) and is the broker’s default operating mode — it was built to guarantee messages survive a broker restart, whereas Redis was built to be fast and gained durability as an add-on.

Operational burden. If Redis is already running in your stack for caching, session storage, or rate limiting, adding Streams costs nothing operationally — same instance, same monitoring, same failover story. RabbitMQ is a dedicated broker: it needs its own multi-node deployment, its own users and virtual hosts, its own upgrade cadence, and its own alerting for queue depth and consumer lag. That weight buys you the routing and dead-letter features above; it is not free.

Ecosystem fit. Celery’s transport layer, Kombu, treats RabbitMQ as its most mature and best-tested backend — connection recovery, exchange declaration, and priority queues all assume an AMQP broker under the hood. Kombu supports Redis too, but with a narrower feature surface (no true routing, priority via a workaround). If your batch orchestration already runs on Celery for async work order batching, RabbitMQ is the path of least resistance; Redis works but you inherit its Kombu limitations.

Comparison Matrix

Hands-On: Enqueue and Consume a Work Order

Both examples queue the same canonical WorkOrderPayload — the SLA fields priority, requested_completion, and escalation_tier travel with every message so a consumer can make dispatch decisions without a second lookup, matching the work order schema standards used across the site.

Redis Streams: Enqueue, Consume, and Dead-Letter

This uses a consumer group so multiple workers can share the backlog, acknowledges each entry only after a successful CMMS commit, and dead-letters an entry once it exceeds a delivery ceiling.

import json
import time
from dataclasses import dataclass, asdict, field
from datetime import datetime, timezone
from enum import Enum
from typing import Dict, List, Optional

import redis


class Priority(str, Enum):
    CRITICAL = "critical"
    HIGH = "high"
    STANDARD = "standard"
    PLANNED = "planned"


@dataclass
class WorkOrderPayload:
    """Canonical CMMS work order — SLA fields are mandatory site-wide."""
    work_order_id: str
    asset_id: str
    part_skus: List[str]
    required_quantities: Dict[str, int]
    location_id: str
    priority: Priority = Priority.STANDARD
    requested_completion: Optional[datetime] = None
    escalation_tier: int = 0
    status: str = "open"
    created_at: datetime = field(default_factory=lambda: datetime.now(timezone.utc))


def serialize_work_order(wo: WorkOrderPayload) -> str:
    raw = asdict(wo)
    raw["priority"] = wo.priority.value
    raw["created_at"] = wo.created_at.isoformat()
    raw["requested_completion"] = (
        wo.requested_completion.isoformat() if wo.requested_completion else None
    )
    return json.dumps(raw)


STREAM = "workorders:intake"
GROUP = "cmms-commit-workers"
DLQ_STREAM = "workorders:intake:dlq"
MAX_DELIVERIES = 3


def ensure_group(client: redis.Redis) -> None:
    try:
        client.xgroup_create(STREAM, GROUP, id="0", mkstream=True)
    except redis.ResponseError as exc:
        if "BUSYGROUP" not in str(exc):
            raise


def enqueue(client: redis.Redis, wo: WorkOrderPayload) -> str:
    return client.xadd(STREAM, {"payload": serialize_work_order(wo)})


def commit_to_cmms(payload: dict) -> None:
    # Replace with the real CMMS write; raise to simulate a transient failure.
    print(f"committed work order {payload['work_order_id']} to CMMS")


def handle_failed_delivery(client: redis.Redis, entry_id: str) -> None:
    """Dead-letter an entry once it exceeds MAX_DELIVERIES; otherwise leave it pending for retry."""
    pending = client.xpending_range(STREAM, GROUP, min=entry_id, max=entry_id, count=1)
    if not pending:
        return
    delivery_count = pending[0]["times_delivered"]
    if delivery_count >= MAX_DELIVERIES:
        raw = client.xrange(STREAM, min=entry_id, max=entry_id)
        if raw:
            _, fields = raw[0]
            client.xadd(DLQ_STREAM, fields)
        client.xack(STREAM, GROUP, entry_id)  # remove from the pending list either way


def consume_once(client: redis.Redis, consumer_name: str) -> None:
    response = client.xreadgroup(GROUP, consumer_name, {STREAM: ">"}, count=10, block=2000)
    if not response:
        return
    for _stream_name, entries in response:
        for entry_id, fields in entries:
            try:
                commit_to_cmms(json.loads(fields["payload"]))
                client.xack(STREAM, GROUP, entry_id)
            except Exception:
                handle_failed_delivery(client, entry_id)


if __name__ == "__main__":
    client = redis.Redis(host="localhost", port=6379, decode_responses=True)
    ensure_group(client)
    wo = WorkOrderPayload(
        work_order_id="WO-77210",
        asset_id="CONV-09",
        part_skus=["BELT-220"],
        required_quantities={"BELT-220": 1},
        location_id="WH-02",
        priority=Priority.HIGH,
        escalation_tier=1,
    )
    enqueue(client, wo)
    consume_once(client, consumer_name="worker-1")

RabbitMQ (pika): Enqueue, Consume, and Dead-Letter

This declares the dead-letter exchange and queue first, binds the intake queue to it via x-dead-letter-exchange, and lets a basic_nack with requeue=False do the dead-lettering — no manual scan required.

import json
from dataclasses import dataclass, asdict, field
from datetime import datetime, timezone
from enum import Enum
from typing import Dict, List, Optional

import pika


class Priority(str, Enum):
    CRITICAL = "critical"
    HIGH = "high"
    STANDARD = "standard"
    PLANNED = "planned"


@dataclass
class WorkOrderPayload:
    """Canonical CMMS work order — SLA fields are mandatory site-wide."""
    work_order_id: str
    asset_id: str
    part_skus: List[str]
    required_quantities: Dict[str, int]
    location_id: str
    priority: Priority = Priority.STANDARD
    requested_completion: Optional[datetime] = None
    escalation_tier: int = 0
    status: str = "open"
    created_at: datetime = field(default_factory=lambda: datetime.now(timezone.utc))


def serialize_work_order(wo: WorkOrderPayload) -> str:
    raw = asdict(wo)
    raw["priority"] = wo.priority.value
    raw["created_at"] = wo.created_at.isoformat()
    raw["requested_completion"] = (
        wo.requested_completion.isoformat() if wo.requested_completion else None
    )
    return json.dumps(raw)


EXCHANGE = "workorders.direct"
QUEUE = "workorders.intake"
ROUTING_KEY = "intake"
DLX_EXCHANGE = "workorders.dlx"
DLQ = "workorders.intake.dlq"


def declare_topology(channel) -> None:
    channel.exchange_declare(exchange=DLX_EXCHANGE, exchange_type="direct", durable=True)
    channel.queue_declare(queue=DLQ, durable=True)
    channel.queue_bind(queue=DLQ, exchange=DLX_EXCHANGE, routing_key=ROUTING_KEY)

    channel.exchange_declare(exchange=EXCHANGE, exchange_type="direct", durable=True)
    channel.queue_declare(
        queue=QUEUE,
        durable=True,
        arguments={
            "x-dead-letter-exchange": DLX_EXCHANGE,
            "x-dead-letter-routing-key": ROUTING_KEY,
        },
    )
    channel.queue_bind(queue=QUEUE, exchange=EXCHANGE, routing_key=ROUTING_KEY)


def enqueue(channel, wo: WorkOrderPayload) -> None:
    channel.basic_publish(
        exchange=EXCHANGE,
        routing_key=ROUTING_KEY,
        body=serialize_work_order(wo),
        properties=pika.BasicProperties(delivery_mode=2, content_type="application/json"),
    )


def commit_to_cmms(payload: dict) -> None:
    print(f"committed work order {payload['work_order_id']} to CMMS")


def on_message(channel, method, properties, body: bytes) -> None:
    try:
        commit_to_cmms(json.loads(body))
        channel.basic_ack(delivery_tag=method.delivery_tag)
    except Exception:
        # requeue=False routes the message to the DLX bound above instead of looping forever.
        channel.basic_nack(delivery_tag=method.delivery_tag, requeue=False)


if __name__ == "__main__":
    connection = pika.BlockingConnection(pika.ConnectionParameters(host="localhost"))
    channel = connection.channel()
    declare_topology(channel)

    wo = WorkOrderPayload(
        work_order_id="WO-77311",
        asset_id="CONV-11",
        part_skus=["BELT-220"],
        required_quantities={"BELT-220": 1},
        location_id="WH-02",
        priority=Priority.HIGH,
        escalation_tier=1,
    )
    enqueue(channel, wo)

    channel.basic_qos(prefetch_count=10)
    channel.basic_consume(queue=QUEUE, on_message_callback=on_message)
    channel.start_consuming()

The shape of the failure handling is the tell: the Redis version tracks delivery counts and re-queues into a DLQ stream itself, while the RabbitMQ version delegates that entirely to the broker’s topology — one basic_nack call is all the application needs to write.

Recommendation by Scenario

• Bursty, spiky intake (webhook floods, shift-change email storms). Redis Streams absorb a burst cheaply and, if Redis already backs your rate limiting or deduplication cache for webhook intake adapters, adding a stream costs no new infrastructure. Favor Redis when the spike is the main operational risk and you can tolerate building the dead-letter scan yourself.
• Simple pipeline, one producer type, small team. If the queue only ever has one or two consumer groups and no need to fan out by routing key, Redis Lists or Streams keep the topology — and the on-call surface — small. Reach for RabbitMQ only when the simplicity of Redis stops covering what you actually need.
• Already standardized on Redis for caching or sessions. Reusing an existing instance beats standing up a new broker class purely for operational reasons; this is the single strongest pull toward Redis regardless of the other criteria.
• Strict at-least-once semantics, complex routing, or native dead-lettering as a requirement. RabbitMQ’s protocol-level acks, exchange-based routing, and built-in x-dead-letter-exchange remove an entire category of application code you would otherwise own. Favor RabbitMQ when a poison message must never require a manual pending-entry scan to catch.
• Celery-orchestrated batching. If dispatch already runs through Celery for async work order batching, that guide’s orchestration layer is the one to consult for broker configuration — Kombu treats RabbitMQ as its best-supported transport, so staying on RabbitMQ avoids fighting the framework’s defaults.

Decision Checklist

• Confirm whether Redis already runs in the stack for caching, sessions, or rate limiting before provisioning a new broker class.
• Decide whether any consumer needs routing-key fan-out (priority queue, audit sink, metrics) — if yes, RabbitMQ’s exchanges are the simpler path.
• Choose whether dead-lettering must be a broker guarantee (RabbitMQ) or is acceptable as application logic you own (Redis XPENDING scan).
• Verify the Celery/Kombu transport assumptions if batching already runs through Celery, since RabbitMQ is Kombu’s most mature backend.
• Size the durability requirement — set Redis AOF/fsync policy or RabbitMQ delivery_mode=2 explicitly rather than relying on defaults.

Frequently Asked Questions

Can I start with Redis and migrate to RabbitMQ later without changing the CMMS commit logic?

Yes, if the commit function is written against the deserialized WorkOrderPayload dict rather than against broker-specific objects. Both examples above call the same commit_to_cmms(payload: dict) function; only the enqueue/consume plumbing around it differs, so swapping brokers means rewriting the transport layer, not the business logic.

Does Redis Streams’ consumer-group ack provide the same guarantee as RabbitMQ’s manual ack?

Both give at-least-once delivery, but RabbitMQ enforces it at the protocol level — an unacked message is requeued or dead-lettered automatically when the consumer disconnects. Redis Streams require your code to periodically check XPENDING for entries that were claimed but never acknowledged; the guarantee is real, but the enforcement is yours to build and monitor.

What happens to dead-lettered work orders — do they get retried automatically?

Neither broker retries a dead-lettered message automatically; a dead-letter queue or stream is a holding area for manual or scripted reprocessing, not a retry mechanism. Build a separate consumer (or an operator runbook) that inspects the DLQ, fixes the root cause, and re-publishes to the original queue or stream.

Do I need to pick one broker for the whole system, or can different stages use different ones?

Different stages can use different brokers — it is common to use Redis for a high-volume, low-stakes deduplication cache while RabbitMQ carries the commit-critical dispatch queue. The cost is operational: every additional broker type is another failure mode, another set of credentials, and another thing to monitor, so only split them when a single criterion above genuinely conflicts across stages.

Related

This decision sits inside async batch processing for the broader work order ingestion and parsing pipelines domain; pair the chosen broker’s message shape with the work order schema standards, route producers such as webhook intake adapters into whichever queue you select, and configure orchestration through Celery for async work order batching once the transport is settled.

Part of: Async Batch Processing.