ZMQ minimal support

[jlblancoc]

Write:

• New mrpt::utils:: functions to read/write objects to ZMQ sockets / messages.
• Example code for client & server.

[jolting]

It would be nice to be able to extend this later to various types of transport middle-ware(i.e. ROS transport and DDS) as long as it doesn't add an undue amount of complexity. It might make it easier to create ROS nodes.

Using zmqpp is easier than using zmq directly. It would make a wrapper significantly lighter.

If you're planning on using zmqpp, it's worth noting that the debian maintained version is quite old and doesn't support many of the newer features, some of which are quite useful.

[jlblancoc]

@jolting It would be great to have your opinion on the new examples & code.
In the end I decided to keep it simple, so there is no polymorphism or alike to support other transport layers.

The only missing part for now is the automatic splitting of large objects into ZMQ multipart msgs. I'm unsure about the real need for this feature, but it'll not much effort to introduce it now.

[jlblancoc]

For the records: The new functions are in <mrpt/utils/serialization_zmq.h>

[jolting]

@jlblancoc I like it.

One inconsistency is with pointers and references.
CSerializable is a pointer

void mrpt::utils::mrpt_send_to_zmq (ZMQ_SOCKET_TYPE zmq_socket, const mrpt::utils::CSerializable *obj, const size_t max_packet_len=0)

And here CSerializable is a reference

bool    mrpt::utils::mrpt_recv_from_zmq_into (ZMQ_SOCKET_TYPE zmq_socket, mrpt::utils::CSerializable &target_object, bool dont_wait=false, size_t *rx_obj_length_in_bytes=NULL)

I think both should be references.

Also since both could potentially return an error shouldn't both return bool?

Here is an idea for c++11 PubSub that's fairly general for ZMQ and can be extended for other types of transports.

typedef std::function<bool(const mrpt::utils::CSerializable &)> Pub;
typedef std::function<bool(mrpt::utils::CSerializable &)> Sub;

Sub subscriber = std::bind(&mrpt::utils::mrpt_recv_from_zmq_into, recv_zmq_socket, std::placeholder::_1, false, NULL);

Pub publisher = std::bind(&mrpt::utils::mrpt_send_to_zmq, send_zmq_socket, std::placeholder::_1, 0);

If you build classes that accept Pub Sub functions as callbacks then it should be fairly easy to create classes that can be used by multiple types of transports. This approach is very loosely coupled and is similar to how PCL uses callbacks.

You might have to wait until version 2.0 to do that.

[jolting]

@jlblancoc
This isn't really useful for ZMQ, but it would be nice to have some way of annotating the serializable objects so that it would be possible to extract a protocol definition. The definition could be converted into something like a protobuf definition or a ros msg. This really isn't necessary for ZMQ, but could be useful for other types of transport middle-ware.

https://developers.google.com/protocol-buffers/docs/proto#simple
http://wiki.ros.org/msg

[jlblancoc]

Thanks @jolting ! Sure, I'll change the CSerializable * to &, it's clearly unnnecesary to use pointers here...

I know protobuf & ROS msgs, and I would love to have some unique and non-redundant serialization mechanism for: binary streams (the current implementation), protobuf, ROS msgs (for now it's only implemented in a custom mrpt_bridge pkg, manually adding object by object...), and MEX for MATLAB (only a few objects have this feature, it was done by @jesusbriales ).
But I can't imagine any "elegant" way of doing this without redundancy... My only idea in this line would be adding new (pure) virtual read/write methods in CSerializable, one for each middleware or container format... it's done like that for MEX now.

Perhaps there is no elegant solution, but if you come to some brilliant solution... it'll be more than welcome! ;-)

[jolting]

@jlblancoc
Reflection in c++ is something that has yet to be solved. Compile time reflection would solve our problems.

I really don't have a great idea for this. It's one of those neglected topics in c++.

There is something available for DDS, but it has a few boost dependencies and looks like it uses macros.
https://github.com/rticommunity/rticonnextdds-reflex

I haven't seen an elegant solution for c++11 yet, but I imagine one could use variadic templates.

template <typename OBS, typename ...FIELDS>
class CObservationSerDes
{
public:
  static std::string getSpec();
  static serialize(CStream & stream, const OBS & obs, FIELDS ... fields);
  static deserialize(CStream &stream, OBS & obs, FIELDS ... fields);
}

There isn't a way to get a list of member variables from a class in c++, you'll still have to pass in the members you'll want to serialize.