Need more examples and improved documentation

[prashant-saxena]

Hello,
From the beginner to intermediate level c/c++ user I'm finding it a bit harder to understand the concept because of fewer examples and no proper documentation. I hope it will be up and running soon. Here is the JSON structure I'm trying to read:

{
    "config": {
        "min": int,
        "max": int
    },
    "state": {
        "K": {
            "coef": {
                "min": int,
                "max": int,
                "mean": [float array 20 elements],
                "duration": [float array, size unknown],
                "length": [int array, size unknown],
                "data": [[20 floats], [20 floats], [20 floats], .... size unknown]
            },
        },
        "PH": {
            "coef": {
                "min": int,
                "max": int,
                "mean": [float array 20 elements],
                "duration": [float array, size unknown],
                "length": [int array, size unknown],
                "data": [[20 floats], [20 floats], [20 floats], .... size unknown]
            },
        },
        .... roughly 40+ more state values
    }
}

Here is the c++ code

#include <iostream>
#include <fstream>
#include <sstream>
#include <vector>
#include <glaze/glaze.hpp>

using namespace std;

struct my_struct2 {
    int min;
    int max;
    vector<float> mean;
    vector<float> duration;
    vector<int> length;
    vector<vector<float>> data;
};

struct my_struct {
    map<string, int> config{{"min", 0}, {"max", 0}};
    map<string, map<string, my_struct2>> state{};
};

int main() {
    std::ifstream myFile("model.json");
    std::ostringstream tmp;
    tmp << myFile.rdbuf();
    std::string jsbuff = tmp.str();
    
    my_struct s{};

    auto err = glz::read_json(s, jsbuff);
    if (err) {
        std::cerr << glz::format_error(err, jsbuff) << '\n';
        return(EXIT_FAILURE);
    }
    
    // HOW DO YOU READ CONFIG AND STATE MEMBERS?

    return 0;
}

Question 1: How do you get 'config' and 'state' members?
Question 2: If one has to read arrays in nc::NdArray instead of vector, than how do you do it?

Cheers

[stephenberry]

I agree that more documentation is needed. It is slowly growing and I'll try to make a beginners documentation soon.

For the best performance and ease of use, you would just put config and state members in a struct.

struct config_t
{
   int min{};
   int max{};
};

struct coef_t
{
   int min{};
   int max{};
   std::array<float, 20> mean{};
   std::vector<float> duration{};
   std::vector<int> length{};
   // TODO: implement "data"
};

struct state_t
{
   coef_t coef{};
};

struct top_level_t
{
   config_t config{};
   std::map<std::string, state_t> state{};
};

// Now that you have all the structures defined you can read in a file
void some_function()
{
   top_level_t top_level{};
   std::string buffer{};
   auto err = glz::read_file_json(top_level, "model.json", buffer);
   if (err) {
      std::cerr << glz::format_error(err, buffer) << '\n';
   }
   else {
       // top_level is now populated with all your state information
       // top_level.state.at("K") would give you the "K" state
       int config_min = top_level.config.min;
       // etc.
   }
}

[stephenberry]

Question 2: If one has to read arrays in nc::NdArray instead of vector, than how do you do it?

I have not used NumCpp before, but if they follow typical C++ conventions then you should just be able to use their types just like std::vector, std::array, etc. Glaze uses C++20 concepts to determine how types should be parsed without requiring additional dependencies. So, try to read and write from nc::NdArray and if you run into issues let me know and I can probably add some concepts to support them.

[prashant-saxena]

Thanks @stephenberry,
When changing this structure's member length to nc::NdArray

struct coef_t
{
   int min{};
   int max{};
   std::array<float, 20> mean{};
   std::vector<float> duration{};
   nc::NdArray<int> length{};
   // TODO: implement "data"
};

The error thrown is

 D:\projects\glaze\include\glaze\reflection\to_tuple.hpp(82,38): error C3448: the number of identifiers must match the
 number of array elements or members in a structured binding declaration

Unfortunately, I can't use glaze right now as 2D arrays are not supported. Let me know your plans for the implementation.

[stephenberry]

This just means that nc::NdArray cannot be automatically reflected. This is because their class is not aggregate initializable, which it probably should be.

When you get reflection issues the next step is to try the glz::meta approach, where you list out the fields explicitly. See the README for more explanation.

For multi-dimensional data you need to determine how you want your JSON to be formatted. Glaze has a header for handling the Eigen matrix library you could look at glaze/ext/eigen.hpp. And you can always write your own custom serialization/deserialization. Just look in the docs folder for the associated documentation.

Glaze definitely supports this library, it just depends on whether you need to write a little more code.

I can take a look at this next week and add a an extension header for their library if they don't comply with typical interfaces.

[stephenberry]

I just realized the eigen code isn't as generic as I thought. I'll work on making that more generic so that other matrix libraries can be easily used with glaze.