dds_take and dds_read looks completely same

[ami-GS]

Hello,

I noticed dds_read and dds_take are completely same. is this expected implementation?
Or forget to remove one side?
I'm confused when I reading example codes, one is using dds_read, one another is using dds_take

Thanks

[eboasson]

Hi,

They look the same, but they are very different in behaviour: read returns a copy of the samples, leaving them available in the reader’s history cache so you can read them again later (marking it as “read” along the way); take returns them but also removes them from the reader’s history cache (but not from any other readers’ history caches).

Sometimes it doesn’t really make much of a difference — for a trivial example like HelloWorld there is no functional difference at all: it reads one sample and then exits. For others, it makes a world of difference, for example, doing a throughput test using read would quickly end up using all available memory.

So a simple answer could be: use read if you want to be able to read the data again; in all other cases use take. But I think that simple answer misses the point (even though it is technically correct) because it doesn’t give any suggestion why you would ever want to read the data again.

(TL;DR hint: skip to the end, but hopefully you’ll take the time to read on.)

The more complicated answer is almost philosophical in nature, because it really is asking “what is DDS?” To that question, you’ll probably get a different answer from every person you ask, which is probably the case because DDS is two things at the same time.

The first is that is an eventually consistent shared data space. Shared data spaces go back to early ‘80s with two independent strains in existence: the synchronous ones along the lines of Linda (from D. Gelernter at Yale; Java’s TupleSpace and many others are direct descendants); and the eventually consistent ones, starting with SPLICE (M. Boasson at Hollandse Signaalapparaten BV) and of which DDS is a direct descendant.

The shared key feature of both kinds of shared data space is that they fully focus on the data (they’re the opposite of OOP because they encapsulate functionality and expose data) and decouple data availability from the existence of application processes. So it doesn’t matter when a process starts because the data is available anyway; if the processes are stateless, they can be killed and restarted at will.

Despite sharing this key feature, they are also very different. Linda (and descendants) is like a database server: applications send requests to the server to add tuples, read a copy of a tuple or remove a tuple. All operations are atomic, which turns the server into a choke point. Of course you can build a fault-tolerant cluster for this, but you will still have the round-trip latencies and the inherent scalability problems that come with a synchronous model.

DDS (and its predecessors) are more like a distributed caching scheme, where each process subscribes to what it needs and caches its own copy of that part of the data space. These caches can contain stale data when a new sample has already been published but hasn’t arrived yet, but in this model, this is considered perfectly acceptable as long as it gets updated eventually.

This makes updating and reading very fast and predictable because there is no system-wide synchronisation, but the price is that deleting data becomes more complicated. For deleting data, basically all you can do is indicate that an instance (= key value) is no longer relevant using the dispose operation, and then rely on the subscribers to remove it from their caches themselves.

An important consequence is that the content generally should describe the state of the system rather than events, because you decouple the processes so much that they may not always get all updates — think the difference between a light dimmer sending commands like “brighten by 1 unit” vs “set the brightness to X”: if you skip one of the former it affects the result, but skipping one of the latter has no meaningful effect. If you need to deal with events, you can, but that is the other face of DDS. Ideally, then, your data would be similar to what you would store in relational database. Seen in that light, the read operation would be the normal operation, and the take operation is the exception, used exclusively to remove disposed data.

The other face of DDS is that it is also a publish-subscribe messaging system — like MQTT, like Kafka, like ROS2, like so many others. In this view of DDS, you publish messages and you consume them. So then, you would generally want to use take.

The QoS settings determine which face of DDS you are dealing with: the first is (as a guideline): reliable, no auto-dispose, transient durability, keep-last history, by-source-timestamp destination order; the second is: reliable, volatile durability, keep-all history (on reader and writer), by-reception-timestamp destination order. If you want some measure of historical data for late-joining applications in the second mode, upgrade from volatile to transient-local. Similarly, if, in the first mode, you can tolerate waiting until the next update comes (or you don't have late joiners), you downgrade it from transient to volatile.

Two important notes: (1) Cyclone DDS isn’t done yet and support for the transient setting isn’t there yet. In practice, you can build many applications by simply using transient-local instead. (2) The ability to do proper queries on the data is not there yet, so saying “use read all the time” might not work out so well at this point in time: if you’re resolving a foreign key and just need the sample with key value X, being forced to read everything because the API call that you need isn’t there yet kinda sucks.

So for the time being, you probably want to use take, extract whatever you need and keep that in your own custom data structurre in the application.

With apologies if I have only made it more confusing ...

[ami-GS]

Hi,

Thanks for kind reply, your note can get me more familiar with DDS :)
Let me ask several question again.

• if read and take work differently, which code is doing such "functional switch"?, in code, there is no difference.
• dds_read is for milti-read, OK, but the multi mean that single subscriber read multi time? or multi subscriber read mutil time?

[eboasson]

I'm not sure I understand your first question ... in the implementation, read and take definitely are different. The difference may not be very large, but one removes each sample as it is read while the other marks it as read. Is that what you were asking?

The "multi" is about a single subscriber (very technically speaking, a single DDS DataReader) reading multiple times. Different subscribers (DataReaders, to be exact) are always completely independent. Even the take operation only affects a single DataReader.

[ami-GS]

Oh, finally I could notice the small difference, true or false with dds_read_impl, this is bool to indicate take or read. Sorry for bothering you.

ret = dds_read_impl (true, rd_or_cnd, buf, bufsz, maxs, si, NO_STATE_MASK_SET, DDS_HANDLE_NIL, lock, false);

ret = dds_read_impl (false, rd_or_cnd, buf, bufsz, maxs, si, NO_STATE_MASK_SET, DDS_HANDLE_NIL, lock, false);

And thanks for the "multi" meaning :)

[eboasson]

No worries — indeed, thanks for being interested :)