Middleware & System Architecture
DDS vs Zenoh, QoS, and monolith vs. multi-process robot architectures.
Prerequisites
Why it matters in robotics
Beyond a single ROS graph, a real robot is a distributed system, and its middleware and architecture determine how robust it is. ROS2's messaging layer is built to be agnostic to which underlying communication library actually moves the bytes between processes and machines, and that underlying layer behaves in specific, sometimes surprising ways once it's running on a real network with multiple robots and computers talking at once. Whether a publisher and subscriber actually manage to connect, reliable vs. best-effort delivery, whether a late-joining subscriber gets missed messages, and how discovery holds up as the fleet grows are the practical concerns, alongside latency, determinism, and failure isolation across processes and machines.
What to study
- ✓The ROS2 RMW abstraction: how rcl/rclcpp/rclpy sit above the vendor-agnostic rmw interface, and how pluggable implementations (Fast DDS, Cyclone DDS, rmw_zenoh) are swapped via RMW_IMPLEMENTATION.
- ✓DDS internals and QoS: RTPS, distributed discovery, and the request-vs-offered compatibility rules for reliability, durability, deadline, and history.
- ✓Transport tradeoffs: UDP/RTPS vs shared-memory vs intra-process, and when node composition / component containers and lifecycle nodes are the right architectural choice.
- ✓Comparing DDS vs Zenoh vs MQTT vs gRPC, discovery scaling at fleet size, fault isolation, and security via SROS2 / DDS-Security.
Where to study
Cheatsheet
RMW: one API, swappable backend
ROS2 user code calls `rclcpp`/`rclpy`, which calls the vendor-neutral `rcl` layer, which calls the `rmw` interface — the actual wire protocol (Fast DDS, Cyclone DDS, rmw_zenoh, ...) is swapped at runtime via the `RMW_IMPLEMENTATION` environment variable with no code changes.
This is why two robots can silently fail to talk to each other after a vendor mismatch, or why switching DDS vendors can change discovery time and CPU load without touching a single node.
DDS discovery and QoS compatibility
DDS (using the RTPS wire protocol) is fully decentralized: every participant announces itself and matches with compatible endpoints by multicast/multicast-like discovery — there's no broker or central registry.
A publisher and subscriber only connect if every requested QoS policy is satisfied by what's offered: reliability (best-effort/reliable), durability (volatile/transient-local), deadline, and history depth all follow a request-vs-offered compatibility rule, checked policy by policy.
Transport choice is a latency/isolation tradeoff
UDP multicast over RTPS crosses process and machine boundaries but pays serialization + kernel network stack cost. Shared memory transport (e.g. iceoryx via Fast DDS, or Cyclone's shared-memory support) skips the network stack for same-host nodes. Intra-process comms (ROS2 component composition) skips serialization entirely by passing a pointer within one process.
Composing nodes into one process (component containers) minimizes latency but couples their lifetimes and crash domains — a segfault in one component takes down the whole container, trading fault isolation for speed.
Fleet-scale discovery cost
Decentralized discovery is in participant pairs by default — every participant discovers and matches with every other. At fleet scale this shows up as discovery storms and CPU/bandwidth overhead on join; discovery servers (Fast DDS) or static peer lists trade some flexibility for bounded, centralized discovery traffic instead of full-mesh gossip.
Security is bolted on, not default
Plain DDS traffic is unauthenticated and unencrypted. SROS2 layers DDS-Security on top: participant authentication, access control (which topics a node may publish/subscribe), and encryption — configured via a keystore and permission files per node, not free by default the way TLS-everywhere web traffic is.
| DDS | Zenoh | MQTT | gRPC | |
|---|---|---|---|---|
| Topology | Decentralized peer-to-peer | Peer-to-peer or routed | Broker-based | Point-to-point (client/server) |
| Discovery | Multicast, full-mesh | Scalable routed discovery | None (broker is fixed endpoint) | None (direct address/DNS) |
| Delivery model | Pub/sub with rich QoS | Pub/sub + query, unified | Pub/sub, simple QoS (0/1/2) | RPC (unary + streaming) |
| Best fit | LAN robot fleets, ROS2 default | WAN/constrained links, ROS2 alt. | IoT, low-bandwidth telemetry | Service APIs, cross-language RPC |