add user encode/decode

[mdavidsaver]

Attempt to address #9 with a user API for (de)serialization. Three pairs of functions are added for the 3 contexts in which PVD are found in the PVA protocol: Type, Full structure, and Partial structure. The xcode::encode*() functions append to an std::vector<uint8_t>. The xcode::decode*() functions operate on a pair of pointers (like iterators). The first pointer is passed by reference, and updated based on the number of bytes consumed by each operation.

[AppVeyorBot]

❌ Build pvxs 1.0.269 failed (commit aa4b3b1941 by @mdavidsaver)

[mdavidsaver]

@karlosp Can you comment on whether this change would be flexible enough to support your use case?

[karlosp]

I think that we have a bit of special use cases for serialization.

Proposed changes would be great if we would need to save/send std::vector<uint8_t> buf.

Because we use the EPICS transport layer in a real-time framework for custom structures, we need to (de)serialize every field separately and not the whole Value.

In the example below I will try to describe how we solve it for now.
Do not know if this is the best approach but for now, somehow works.

In initialization time we must dynamically construct Value, because we must support custom user types.
For example, let say that we have a struct below we would like to send over EPICS.
e.g.

struct TestStructure {
  std::vector<size_t> values;
  size_t index;
  size_t hash;
}

Currently, we dynamically create Value and reserve buffer as follows:

pvxs::TypeDef pvxs_field_builder_{pvxs::TypeCode::Struct,"simple_t",{}};
std::vector<pvxs::Member>& pvxs_members;
std::vector<uint8_t> pvxs_buffer_;

// pseudocode
size_t buffer_size{0};
for member in TestStructure:
 pvxs_members.push_back(pvxs::Member(member_type, member_name));
 buffer_size += sizeof(member_type);

pvxs_field_builder_ += pvxs_members_;
Value value_ = pvxs_field_builder_.create();
pvxs_buffer_ = std::vector<uint8_t>(buffer_size, '\0');

Example of putting data from TestStructure to PutBuilder.

// Sending data
// pseudocode
copy data from TestStructure to pvxs_buffer_;
unsigned char* storage_location = nullptr;
auto put = pvxs::client::PutBuilder(context_.put(topic_name));
size_t position{0};
// pseudocode
for (pvxs::Value fld : value_.iall()) {
  if(fld.type().scalarOf().code == pvxs::TypeCode::Struct)
     continue;
  storage_location = pvxs_buffer_.data() + member_position[position].offset;
  put.set(value_.nameOf(fld), pvxs::shared_array<const int8_t>((int8_t*)(storage_location), sizeof(member) / sizeof(int8_t)));
++position;
}
put.exec()->wait(5.0);

receiving data

//pvxs::client::Subscription> monitor;
value_.assign(monitor->pop());
// pseudocod
//create empty TestStructure (take special care for initializing vectors and arrays to be correct size)
TestStructure TS;
for (auto fld : value_.iall()) {
  if (fld.type() == pvxs::TypeCode::Struct) {
    continue;
  }
 read and copy data from fld to TS // type end size taking into account
}

[mdavidsaver]

Proposed changes would be great if we would need to save/send std::vector<uint8_t> buf.

What interface would work?

FYI. If your libraries are able to interface with a shared_ptr, then the aliasing constructor of std::shared_ptr may be helpful here. eg.

auto buf = std::make_shared<std::vector<uint8_t>>({1,2,3});
std::shared_ptr<uint8_t> aliased(buf, buf.data());
buf.reset();
aliased.reset(); // ~vector happens here

The aliased shared_ptr shares the same ref. counter, and the same (virtual) destructor.

Also, I'm having difficulty in parsing through your example. A couple of things I pick up:

 buffer_size += sizeof(member_type);

You would like a way to find the serialized size of a field based solely on the type (TypeCode)? The PVD encoding has a number of variable sized elements, which makes this difficult for all but the simplest definitions.

Is the intent to avoid resize()ing the vector during serialization?

put.set(value_.nameOf(fld), pvxs::shared_array<const int8_t>((int8_t*)(storage_location), sizeof(member) / sizeof(int8_t)));

Do I understand this correctly? You would like to send serialized data as a byte array over PVA?

[mdavidsaver]

For lack of feedback, I have dropped this change from my development area.