Eclipse Cyclone DDS

Eclipse Cyclone DDS is a very performant and robust open-source implementation of the OMG DDS specification. Cyclone DDS is developed completely in the open as an Eclipse IoT project (see eclipse-cyclone-dds) with a growing list of adopters (if you’re one of them, please add your logo). It is a tier-1 middleware for the Robot Operating System ROS 2.

• What is DDS?
• Getting Started
• Performance
• Configuration

What is DDS?

DDS is the best-kept secret in distributed systems, one that has been around for much longer than most publish-subscribe messaging systems and still outclasses so many of them. DDS is used in a wide variety of systems, including air-traffic control, jet engine testing, railway control, medical systems, naval command-and-control, smart greenhouses and much more. In short, it is well-established in aerospace and defense but no longer limited to that. And yet it is easy to use!

Types are usually defined in IDL and preprocessed with the IDL compiler included in Cyclone, but our Python binding allows you to define data types on the fly:

from dataclasses import dataclass
from cyclonedds.domain import DomainParticipant
from cyclonedds.core import Qos, Policy
from cyclonedds.pub import DataWriter
from cyclonedds.sub import DataReader
from cyclonedds.topic import Topic
from cyclonedds.idl import IdlStruct
from cyclonedds.idl.annotations import key
from time import sleep
import numpy as np
try:
    from names import get_full_name
    name = get_full_name()
except:
    import os
    name = f"{os.getpid()}"

# C, C++ require using IDL, Python doesn't
@dataclass
class Chatter(IdlStruct, typename="Chatter"):
    name: str
    key("name")
    message: str
    count: int

rng = np.random.default_rng()
dp = DomainParticipant()
tp = Topic(dp, "Hello", Chatter, qos=Qos(Policy.Reliability.Reliable(0)))
dw = DataWriter(dp, tp)
dr = DataReader(dp, tp)
count = 0
while True:
    sample = Chatter(name=name, message="Hello, World!", count=count)
    count = count + 1
    print("Writing ", sample)
    dw.write(sample)
    for sample in dr.take(10):
        print("Read ", sample)
    sleep(rng.exponential())

Today DDS is also popular in robotics and autonomous vehicles because those really depend on high-throuhgput, low-latency control systems without introducing a single point of failure by having a message broker in the middle. Indeed, it is by far the most used and the default middleware choice in ROS 2. It is used to transfer commands, sensor data and even video and point clouds between components.

The OMG DDS specifications cover everything one needs to build systems using publish-subscribe messaging. They define a structural type system that allows automatic endianness conversion and type checking between readers and writers. This type system also supports type evolution. The interoperable networking protocol and standard C++ API make it easy to build systems that integrate multiple DDS implementations. Zero-configuration discovery is also included in the standard and supported by all implementations.

DDS actually brings more: publish-subscribe messaging is a nice abstraction over “ordinary” networking, but plain publish-subscribe doesn’t affect how one thinks about systems. A very powerful architecture that truly changes the perspective on distributed systems is that of the “shared data space”, in itself an old idea, and really just a distributed database. Most shared data space designs have failed miserably in real-time control systems because they provided strong consistency guarantees and sacrificed too much performance and flexibility. The eventually consistent shared data space of DDS has been very successful in helping with building systems that need to satisfy many “ilities”: dependability, maintainability, extensibility, upgradeability, … Truth be told, that’s why it was invented, and publish-subscribe messaging was simply an implementation technique.

Cyclone DDS aims at full coverage of the specs and today already covers most of this. With references to the individual OMG specifications, the following is available:

• DCPS the base specification
  • zero configuration discovery (if multicast works)
  • publish/subscribe messaging
  • configurable storage of data in subscribers
  • many QoS settings - liveliness monitoring, deadlines, historical data, …
  • coverage includes the Minimum, Ownership and (partially) Content profiles
• DDS Security - providing authentication, access control and encryption
• DDS C++ API
• DDS XTypes - the structural type system (some caveats here)
• DDSI-RTPS - the interoperable network protocol

The network stack in Cyclone DDS has been around for over a decade in one form or another and has proven itself in many systems, including large, high-availability ones and systems where interoperation with other implementations was needed.

This repository provides the core of Cyclone DDS including its C API, the OMG C++ and the Python language bindings are in sibling repositories.

File truncated at 100 lines see the full file

Changelog for Eclipse Cyclone DDS

Unreleased

V0.7.0 (2020-08-06) -----------------------------------------------------------------------------------------------

This release brings support for the DDS Security 1.1 specification:authentication, access control and encryption. It also provides significant performance improvements with large samples, by better scheduling of retransmit requests and by avoiding the occasional excessive latency caused by dropping the heartbeat rate too soon.

One can choose to build Cyclone DDS without support for DDS Security if one wants to reduce the size of the resulting library. The default plug-ins are built only if security is enabled and OpenSSL is available, as those are implemented using the cryptographic operations and I/O primitives for handling certificates and exchanging keys that OpenSSL provides. If one chooses to exclude security support from the build, setting any security related property or adding it to the configuration files will result in participant creation failing with a "precondition not met" error.

A lot of effort has gone into testing and checking that malformed or unexpected messages are handled correctly, that message authentication codes are checked and that no data never goes out unencrypted by accident. Still, it is only prudent to assume the existence of vulnerabilities.

Noteworthy bug fixes:

• DATA_AVAILABLE was not always triggered when by a dispose and sometimes triggered in the absence of an observable state change (arrival of a dispose for an already-disposed instance where the dispose had not yet been read);
• Restores functionality of the "raw ethernet" mode as well as IPv6 with link-local addresses, both accidentally broken in 0.6.0;
• Fixes a crash in processing endpoint discovery data containing unrecognised locator kinds;
• Fixes type conversion for local historical data (e.g., mixed use of ROS 2 C/C++ type supports in combination with transient-local endpoints within a single process);
• Fixes a use-after-free of "lease" objects with manual-by-topic writers;
• Mark instance as "alive" in the reader history and generate an invalid sample to notify the application even if the sample itself is dropped because the same or a later one is present already (e.g., on reconnecting to a transient-local writer);
• Fix a crash when doing an instance lookup on a built-in topic using the key value;
• No longer auto-dispose instances as soon as some registered writer disappears, instead do it only when all of them have unregistered it;
• Fix performance of read_instance and take_instance by performing a proper instance lookup.

V0.6.0 (2020-05-21) -----------------------------------------------------------------------------------------------

• Support for mixed-language programming by supporting multiple (de)serializers for a single topic in a single process. This way, a program that consists of, say, C and C++ can use a different representation of the data in C than in C++. Before, all readers/writers in the process would be forced to use the same representation (or perform an additional copy). Currently C is still the only natively supported language, but one can use an evolving-but-reasonable-stable interface to implement different mappings.
• Improved QoS support: full support for deadline, lifespan and liveliness. The first is for generating notifications when a configured instance update rate is not achieved, the second for automatically removing stale samples, the third for different modes of monitoring the liveliness of peers.
• Improved scalability in matching readers and writers. Before it used to try matching a new local or remote reader or writer with all known local & remote entities, now only with the right group with the correct topic name.
• Improved tracing: discovery data is now printed properly and user samples have more type information allowing floating-point and signed numbers to be traced in a more readable form.
• Extension of platform support
  • Known to work on FreeBSD, CheriBSD
  • Known to work with the musl C library
• Windows-specific changes
  • Fixes multicasts to addresses also used by non-Cyclone processes (caused by accidentally linking with an old sockets library)
  • Correct handling of non-English network interface names

0.5.1 (2020-03-11)

An interim tag for the benefit of ROS2

• Enable QOS features: liveliness, lifespan, deadline
• Fix issues on Windows where multicast data was not received

V0.5.0 (2019-11-21)

This is the fist release candidate for the long overdue second release

File truncated at 100 lines see the full file