Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MsgPack bin8/bin16/bin32 support #2078

Closed
wants to merge 7 commits into from
Closed

MsgPack bin8/bin16/bin32 support #2078

wants to merge 7 commits into from

Conversation

Sanae6
Copy link
Contributor

@Sanae6 Sanae6 commented Apr 14, 2024

I've added binary support to the msgpack deserializer and serializer, along with a fallback for the json serializer. I've also added some tests to the relevant files. Let me know if there's anything I can do to make this implementation better. Thank you for the amazing library :)

This would fix and close #922.

@bblanchon
Copy link
Owner

Hi @Sanae6,

Thank you for this excellent PR!

Before starting the review, I'd like you to remove LinkedBinary.
Indeed, binary values should be treated like raw JSON strings and always be copied; this is part of a larger (incomplete) refactoring to reduce the size of VariantContent.
Then, of course, simplify the names accordingly, since we no longer need the distinction between "owned" and "linked" binary values.

Best regards,
Benoit

@Sanae6
Copy link
Contributor Author

Sanae6 commented Apr 14, 2024

I removed LinkedBinaryValue and renamed OwnedBinaryValue to BinaryValue. Not sure what to name Binary, but I left it as something sensible enough.

REQUIRE(reinterpret_cast<const char*>(binary.data())[0] == 5);
}

#if ARDUINOJSON_STRING_LENGTH_SIZE >= 4
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please move this test to a dedicated file in MixedConfiguration, so you can control the value of ARDUINOJSON_STRING_LENGTH_SIZE

src/ArduinoJson/Json/JsonSerializer.hpp Show resolved Hide resolved
src/ArduinoJson/Variant/VariantContent.hpp Outdated Show resolved Hide resolved
@Sanae6 Sanae6 requested a review from bblanchon April 18, 2024 00:09
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test currently fails. It seems that even when I apply the string length size, it doesn't actually apply to StringNode::length_type. Printing StringNode::maxLength in the StringNode::create function returned 0xffff instead of 0xffffffff. Any advice on how to solve that?

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is surely due to the fact that the namespace doesn't include the value of ARDUINOJSON_STRING_LENGTH_SIZE.
Let me fix that.

Each file in this folder is named according to the setting name, so this file should be named string_length_size_4.cpp.

Anyway, thank you again for this excellent work 👍

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright! I'll leave it to you :)

extras/tests/JsonSerializer/JsonVariant.cpp Outdated Show resolved Hide resolved
src/ArduinoJson/Json/JsonSerializer.hpp Outdated Show resolved Hide resolved
src/ArduinoJson/Memory/StringNode.hpp Outdated Show resolved Hide resolved
@Sanae6 Sanae6 requested a review from bblanchon April 18, 2024 13:07
Copy link
Owner

@bblanchon bblanchon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just pushed the changes to fix this issue with string_length_size_4.cpp.
Please rebase your branch onto 7.x.
Note that I created string_length_size_{1,2,4}.cpp to check that the fix actually works, so you'll have a conflict on string_length_size_4.cpp.

@Sanae6
Copy link
Contributor Author

Sanae6 commented Apr 18, 2024

Alright! Force pushed, seems nothing is broken 👍

@bblanchon
Copy link
Owner

It seems that you lost your version of string_length_size_4.cpp while merging the conflict.

Copy link
Owner

@bblanchon bblanchon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks really good.
I'll do an in-depth review tomorrow.
Thank you for this outstanding PR.

extras/tests/MixedConfiguration/CMakeLists.txt Outdated Show resolved Hide resolved
extras/tests/MsgPackSerializer/serializeVariant.cpp Outdated Show resolved Hide resolved
extras/tests/MsgPackSerializer/serializeVariant.cpp Outdated Show resolved Hide resolved
extras/tests/MsgPackSerializer/serializeVariant.cpp Outdated Show resolved Hide resolved
src/ArduinoJson/MsgPack/MsgPackBinary.hpp Outdated Show resolved Hide resolved
src/ArduinoJson/Variant/VariantCompare.hpp Show resolved Hide resolved
Signed-off-by: Aubrey (Sanae) <aubrey@hall.ly>
@Sanae6 Sanae6 requested a review from bblanchon April 20, 2024 02:14
@Sanae6
Copy link
Contributor Author

Sanae6 commented Apr 20, 2024

Made the changes you wanted, testing MsgPackBinary comparison actually revealed my initial implementation didn't work, so I fixed that as well.

Signed-off-by: Aubrey (Sanae) <aubrey@hall.ly>
@coveralls
Copy link

Coverage Status

coverage: 99.465% (-0.1%) from 99.589%
when pulling c47d19d on Sanae6:7.x
into cd4bf33 on bblanchon:7.x.

writeByte(0xC5);
writeInteger(uint16_t(value.size()));
} else {
writeByte(0xC6);
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line is not covered by the tests.
Please add a test in string_length_size_4.cpp

return MsgPackBinary(content_.asOwnedString->data,
content_.asOwnedString->length);
default:
return MsgPackBinary(nullptr, 0);
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line is not covered by the tests.

static void setBinary(VariantData* var, MsgPackBinary value,
ResourceManager* resources) {
if (!var)
return;
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line is not covered by the tests.

@bblanchon bblanchon closed this in ae0136d Apr 29, 2024
bblanchon pushed a commit that referenced this pull request Apr 29, 2024
@bblanchon
Copy link
Owner

I added the three missing tests, squashed, and merged.

Thank you again for this excellent contribution ❤️

@bblanchon
Copy link
Owner

This feature is available in ArduinoJson 7.1.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add binary support for MessagePack (C4, C5, C6)
3 participants