This is a lock free circular shared memory queue. Hopefully you find it useful. It's a header-only C++ library.
There are three headers and some unit tests to verify data integrity. The headers are located in include/fr/shm.
Defines a queue API. This defines two pure virtual methods, read and write, which queue objects must implement.
Read and Write will return true if they were able to write the queue. You must check their return! If they fail, you can retry or whatever, otherwise NO BYTES FOR YOU!
Write takes a buffer and the number of bytes to write into the buffer. The size is actually a reference, although none of the write-things currently change the value. If you make a queue-thing that wants to write the bytes it can write into the queue and return some "data remaining" kinda value, you can do that, though. Probably not advisable, but hey, it's code! Play with it!
Read takes a buffer and the size of the buffer. Again buffer size is a reference. The underlying queue object will not change this size, but the SizeProtocol object will. More on that in a moment.
The actual queue. You need exactly one producer queue and exactly one consumer queue. These can be used in the same thread, two different threads or between two processes on the system. The Queue takes a key, which is used to create the IPC segment. The key must be the same on the producer and the consumer or NO BYTES FOR YOU!
You can have one "/" at the beginning of your key. You can't have two or more "/" in your key. You can try but if you do NO BYTES FOR YOU!
Writes shovels however many bytes you want into the queue up to the maximum size of the queue. Read shovels however many bytes you want out of the queue up to however many bytes were in the queue.
You call create on your producer queue with the size of the queue you want. This size must be a power of two for bytewise and reasons. You call open on your consumer queue and just have to specify the unix file mode you're opening it in (0600, 0660 or 0666 are the common ones.) The queues go away when the objects are destroyed.
This implements a queue interface that takes ownership of a queue so you can write varying sized messages onto the queue. When you write to a SizeProtocol object, it constructs a buffer with the size you passed to write and the data you told it to write, and then writes that data to the queue.
When you read from a sizeprotocol, it reads a size first and then reads however many bytes was in the size.
There are other ways this could be implemented, but this seemed like the most likely to be stable and not require read to retry if write had written the size but not the rest of the bytes yet. But this is also why I wrote this library, so feel free to experiment with your own implementations!
Make a build directory somewhere and point cmake at this one. It'll basically just build some unit tests that you can run with test/test.
Here are some results from the fun ones:
[ RUN ] TwoProcessDataIntegrity.DataTest
Read 1073741824 bytes from /dev/urandom
Child hash len: 32
Child hash: 7176314e35678475510fcd28eaba42d91d527333857e0e06bcbcd5d23a1cdc4e
Writing digest
Hash from child: 7176314e35678475510fcd28eaba42d91d527333857e0e06bcbcd5d23a1cdc4e
Child wrote 5421669 random-sized blocks of data
Write retries: 26196860
Child finished reading data
Child hash: 7176314e35678475510fcd28eaba42d91d527333857e0e06bcbcd5d23a1cdc4e
My hash: 7176314e35678475510fcd28eaba42d91d527333857e0e06bcbcd5d23a1cdc4e
[ OK ] TwoProcessDataIntegrity.DataTest (6294 ms)
[ RUN ] TwoProcessDataIntegrity.Throughput
Read 1073741824 bytes from /dev/urandom
Child hash len: 32
Child hash: 5820372e2d65175fb08085cd2444a06246ee6da64ec41c2017bd0bbc4049c997
Writing digest
Child wrote 2023463 random-sized blocks of data
Write retries: 0
Transferred 1073741824 bytes in 316 ms
[ OK ] TwoProcessDataIntegrity.Throughput (4072 ms)
Both those tests fork a child process and read a gigabyte of data from /dev/urandom. The child process computes a sha256 checksum of the data. The first test verifies that the data it reads matches the checksum at the end of the run. The second test just gives you a very rough idea of the throughput you can get with the queue. Since it's using SizeProtocol objects with random chunk sizes (in the range of 37 to 1024 bytes,) throughput numbers can vary pretty widely.
I'll get around to building some CMake packaging for these in the next couple of days, but if you're in a hurry to play with it, just put the header files somewhere you can include them from and have fun!