Add support for unions (like variants)

[pawcam]

Hello there, new to using this library so trying to find the best way to use it to serialialize/unserialize the following struct to a known JSON format:

union type_union {
  type1
  type2
  type3
}

struct Data {

 type_union m_union;
 union_type m_unionType;

....
}

So basically, I have a struct that contains some members that are dependent on each other, several of which require transformation functions to/from json, specifically focusing on m_union, m_unionType for this example.

I've been experimenting with the glaze metadata and so far was able to come up with this for serializing the union based on the type:

namespace glz {

template <>
struct meta<Data>
{
  using T = Data;
  static constexpr auto value = object(
      "custom_field_name", [](auto&& self) -> std::string { return transformUnionToString(self.m_union, self.m_unionType); }
}

Data data{};
data.m_unionType = union_type::type1;
string buf;
glz::write_json(data, buf);

The output JSON in buf is: "{"custom_field_name":"valueForType1" }, which is great, just what I intended.

This works for glz::write_json(data, buffer), where custom_field_name is the name I want the field to have. However, it isn't clear to me how to provide the analogous glz::read meta method (does the lambda get applied for both read/write or is there a way to define the read lambda explicitly & separately?)

How would I use glaze::meta to deserialize (read_json or read) a field (potentially with different JSON field name) back into my union/union type members?

My thought is that there would be an analogous meta struct defined for doing glz::read into the same struct, just with different lambdas or static utility methods I have defined outside my Data struct, but I haven't been able to figure out how this is done yet.

My current approach is using to_json and from_json but I'm not sure if it's the correct way to do this. Right now the API seems to have a hook for transforming a struct, but Im not sure I can specify which field Im reading/writing from/to. I've seen examples specializing this template for each individual struct field type, but none yet for working with other struct fields that depend on each other. To do that I would imagine we would just specialize the whole struct and read/write to each field inside each to/from method:

  template <>
  struct from_json<Data>
  {
    template <auto Opts>
    static void op(Data& value, auto&&... args)
    {
        std::string typeString{};
       read<json>::op<opts>(typeString, args...); // Can I specify the JSON field name I want to read here? Would be ideal since the from JSON is formatted differently
        value.m_unionType = toUnionType(typeString)

        std::string unionString
        read<json>::op<opts>(unionString, args...); // specify known JSON field name here?
       
       // read the rest of the fields by JSON field name... 
    }
  };

   template <>
   struct to_json<Data>
   {
      template <auto Opts>
      static void op(Data& value, auto&&... args) noexcept
      {
          write<json>::op<opts>(transformUnionToString(value.m_union, value.m_unionType);, args...); // specify custom field name here
         // write the rest of the fields by JSON field name...
      }
   };

Thanks, let me know if I can clarify my issue any further or if this is the proper way to accomplish this.

[stephenberry]

Your single lambda approach invokes the function for both reading and writing. To split out different behaviors for reading and writing with lambdas you can register separate lambdas using glz::custom.

A basic example:

struct custom_load_t
{
   std::vector<int> x{};
   std::vector<int> y{};

   struct glaze
   {
      static constexpr auto read_x = [](auto& s) -> auto& { return s.x; };
      static constexpr auto write_x = [](auto& s) -> auto& { return s.y; };
      static constexpr auto value = glz::object("x", glz::custom<read_x, write_x>);
   };
};

[stephenberry]

You probably have a good reason for using unions, but Glaze also has a lot of support for std::variant that can achieve similar goals: docs/variant-handling.md

[pawcam]

Thanks for the quick response! This looks like what I'm going for and I'll give it a try.

In the meantime I came up with another solution where I define a "normalizing" struct to act as a shim for reading/writing data that just uses the glz::meta approach with normal reflection

template <>
struct glz::meta<normalized_data>
{
  using T = normalized_data;
  static constexpr auto value = object(
      "actor-name"                 , &T::actor_name
      "actor-type"                 , &T::actor_type
    )
}


struct normalized_data
{
   string actor_name
   string actor_type
}

struct Data
{
  union u1
  int unionType
}

void serialize(Data &data)
{
  normalized_data.field1 = transformUnion(data.u1, data.unionType)
  glz::write(normalized_data, buffer)
}

I'll weigh the two options if I ever need to add new structs, and if I ever move to variants I'll give that a second look.

One last question (slightly related), my glz::meta seems to output the incorrect key (just uses member name for reflection rather than how I specified it). Is there an option needed in order to enable dasherized/hyphenated key names? Below is the output

{"actor_name":"John Smith","actor_type":"developer"}

[stephenberry]

Cool, that sounds good. And, std::variant is really nice if you get around to experimenting with it.

One last question (slightly related), my glz::meta seems to output the incorrect key (just uses member name for reflection rather than how I specified it). Is there an option needed in order to enable dasherized/hyphenated key names? Below is the output

This issue is something I haven't seen before. Maybe your glz::meta specialization needs to be declared after your class, or it isn't in the same translation unit as your serialization. If you could provide me with a simple example of code that produces the incorrect output that would help me find the problem.

[pawcam]

No problem. I think it's what you're saying, I changed the names entirely and they don't seem to have any effect.

This should be a working example (I added a bit more of my actual code to illustrate what's happening):
glazeMetadata.h

#include "Reflection.h"

#include <glaze/glaze.hpp>

namespace glz {

template <>
struct meta<event::Data>
{
  using T = event::Data;
  static constexpr auto value = object(
      "actor-name"               , &T::actor_name
    , "actor-type"                , &T::actor_type
    , "job-id"        , &T::job_id
  );
};

} // namespace glz

Reflection.h

namespace event
{
  struct Data
  {
    uint32_t         job_id;
    std::string      actor_name;
    std::string      actor_type; 
  };

} // namespace event

These two structs are in separate header files inside of the same library, built using CMake. Here's my test code:

event::Data data{};
data.job_id = 1;
data.actor_name = "John Smith";
data.actor_type = "developer";
std::string_view buffer;
glz::write_json(data, buffer);

[pawcam]

No need to chase this down, it was the include order that was throwing things off. I fixed it by just moving the meta specialization into the same header.

Appreciate all the help!

[pawcam]

Sorry to reopen this, but kind of ran into trouble with this approach further down the line, but it seems like there isn't a way to access the data read from the json string inside my lambda:

template<>
struct glz::meta<Data>
{
  static constexpr auto read_type = [](const auto &s)  {
    return // how do I get the raw string value read from JSON here in order to transform it back into my struct field?
  };
  static constexpr auto write_type = [](const auto& s) {
    return printTypeString(s.type));
  };

  static constexpr auto value = object( "type"            , glz::custom<read_type, write_type>)
}

When I run this code all I get in the lambda is the empty struct Im trying to read into. I guess the real thing I'm after here is full control over how data gets written/read from json based on access to both the struct we're reading into, and the raw json data. Is there a way to pass more arguments to custom readers and writers than just const auto&?

something like:
static constexpr auto read_type = [](const auto &s) {
string tmp = glz::read("type")
return toType(tmp); // how do I get the raw string value read from JSON here in order to transform it back into my struct field?
};

[pawcam]

I think I see how to do this now based on the compile error messages

static constexpr auto const read_instrument_type = [](Data&s, const std::string_view& value) -> void {
    s.type = serialize::FieldHelper::parseInstrumentTypeString(value);
  };
  static constexpr auto write_instrument_type = [](const auto& s) {
    return printType(type);
  };

It still doesn't eliminate the need for tagged union support (I'd imagine it'd be very similar to the variant support) We do have strong reasons for wanting to control the size of the union but if we ever make the change it should work for our purposes. Are there any plans to support tagged unions? e.g.

union typeUnion
{
   type1 cool
   type2 beans
}
enum type;

I started down my own workaround to imitate what you do for variant, but it seems there's no support for union:

template<>
struct meta<typeUnion>
{
  using T = typeUnion;
  static constexpr std::string_view tag = "type";
  static constexpr auto ids = std::array{"type1Tag", "type2Tag};
};

/Serializer.h:82:26: error: no matching function for call to ‘read_json(typeUnion&, std::string_view&)’
   82 |     errc = glz::read_json(m_union, buffer);
      |            ~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~
In file included from /home/user/.cache/CPM/glaze/2c5e2f53ecc3ddc14bae077f3e1c198222641b8e/include/glaze/json/json_ptr.hpp:13,
                 from /home/user/.cache/CPM/glaze/2c5e2f53ecc3ddc14bae077f3e1c198222641b8e/include/glaze/binary/write.hpp:11,
                 from /home/user/.cache/CPM/glaze/2c5e2f53ecc3ddc14bae077f3e1c198222641b8e/include/glaze/binary/ptr.hpp:7,
                 from /home/user/.cache/CPM/glaze/2c5e2f53ecc3ddc14bae077f3e1c198222641b8e/include/glaze/binary.hpp:7,
                 from /home/user/.cache/CPM/glaze/2c5e2f53ecc3ddc14bae077f3e1c198222641b8e/include/glaze/glaze.hpp:35,
                 from Reflection.h:8,
                 from Serializer.h:4,
                 from SerializerTest.cpp:9:
/home/user/.cache/CPM/glaze/2c5e2f53ecc3ddc14bae077f3e1c198222641b8e/include/glaze/json/read.hpp:2709:37: note: candidate: ‘template<class T, class Buffer>  requires  read_json_supported<T> glz::parse_error glz::read_json(T&, Buffer&&)’
 2709 |    [[nodiscard]] inline parse_error read_json(T& value, Buffer&& buffer) noexcept
      |                                     ^~~~~~~~~
/home/user/.cache/CPM/glaze/2c5e2f53ecc3ddc14bae077f3e1c198222641b8e/include/glaze/json/read.hpp:2709:37: note:   template argument deduction/substitution failed:
/home/user/.cache/CPM/glaze/2c5e2f53ecc3ddc14bae077f3e1c198222641b8e/include/glaze/json/read.hpp:2709:37: note: constraints not satisfied
...
/home/user/.cache/CPM/glaze/2c5e2f53ecc3ddc14bae077f3e1c198222641b8e/include/glaze/core/opts.hpp:223:12:   required for the satisfaction of ‘read_json_supported<T>’ [with T = typeUnion]
/home/user/.cache/CPM/glaze/2c5e2f53ecc3ddc14bae077f3e1c198222641b8e/include/glaze/core/opts.hpp:223:34:   in requirements  [with T = typeUnion]
/home/user/.cache/CPM/glaze/2c5e2f53ecc3ddc14bae077f3e1c198222641b8e/include/glaze/core/opts.hpp:223:53: note: the required expression ‘glz::detail::from_json<typename std::remove_cvref<_Tp>::type>{}’ is invalid
  223 |    concept read_json_supported = requires { detail::from_json<std::remove_cvref_t<T>>{}; };
      |                                                     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

[stephenberry]

The problem is that I don't think C++ has a mechanism to determine the alternative types of a union. Whereas, with a std::variant we can look at the types at compile time and build logic about how to serialize/deserialize it.

In the glz::meta we could have the user list the union types. This requires more typing, but I think makes sense.

I do think it would be useful to add union support to Glaze (for trivial types). I'll update this issue's name appropriately.