In Memory Deserialization

[HamzaHajeir]

Hi there,
I'm wondering whether there's a way I can deserialize and point to input buffer(s) when deserializing?

Use case: I'm building a system where I get input data by communication channels, pass it to a deserializer function that writes data out to std::string_view.

Currently When the callee function uses the document-based deserialization, which copies from the buffer, the std::string_view points to local memory that will be invalidated when it returns the parsed data.

Is there a way to get the deserialized data pointing to the input buffer?
Regards,
Hamza Hajeir

Hamza

[bblanchon]

Hi Hamza,

Can you provide a code example of what you're trying to achieve?

Best regards,
Benoit

[HamzaHajeir]

Something like:

std::string input = "...";
auto ec = deserializeJson(doc, input);
JsonObject root = doc.as<JsonObject>();
for (JsonPair kval : root){
    auto keyview = kval.key().get_view(); // points to input buffer (within input.begin() and input.end())
    auto valview = kval.value().as<std::string_view>(); // same as above
}

Or something like a callback:

std::string input = "...";
std::map<std::string_view,std::string_view> myDataStructure;
auto ec = deserializeJsonAsync(input, 
    [&](const std::string_view key, const JsonVariant& value) {
        myDataStructure[key] = value.as<std::string_view>();
        // Can process and test the input type .. with limitation when converting to numbers and booleans as it will create data in memory (Not pointing to the input buffer).
        // With a potential directive to continue or stop:
        return CONTINUE; // DROP (Perhaps useful for dropping the memory it allocates if using doc/Or the user had already taken a copy of) / STOP (If the user had finally found what he was looking for / ...
    });

* Don't know whether a document is necessary or not...

[bblanchon]

points to input buffer

This was the behavior of ArduinoJson 6 when the input was a char*.
I removed this feature from ArduinoJson 7 because it was misunderstood and caused many bugs.

If this optimization is crucial to your project, I recommend you downgrade to ArduinoJson 6 or switch to another library, such as jsmn.

[HamzaHajeir]

points to input buffer

This was the behavior of ArduinoJson 6 when the input was a char*. I removed this feature from ArduinoJson 7 because it was misunderstood and caused many bugs.

If this optimization is crucial to your project, I recommend you downgrade to ArduinoJson 6 or switch to another library, such as jsmn.

I see, thanks for your note.

I'm on ArduinoJson 6, and just tested that and that do work. I'll depend on it, and I ask you to reconsider such a feature in future versions..

With Thanks,
Hamza