Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix fragmentation #297

Merged
merged 21 commits into from
Dec 18, 2023
Merged

Fix fragmentation #297

merged 21 commits into from
Dec 18, 2023

Conversation

jean-roland
Copy link
Contributor

@jean-roland jean-roland commented Dec 15, 2023

There are currently a few issues regarding fragmentation messages:

This PR is attempting to resolve that and also adds a CI non-regression test for this feature.

@@ -45,6 +45,8 @@
#define _Z_FLAG_Z_T 0x20 // 1 << 5 | QueryTarget if T==1 then the query target is present
#define _Z_FLAG_Z_X 0x00 // Unused flags are set to zero

#define _Z_FRAG_BUFF_BASE_SIZE 128 // Base size of the buffer to encode a fragment message header

// Flags:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why 128bytes? If there is a reason for it, please document it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's an arbitrary value we decided with @Mallets, that seemed not to big and not to small. The buffer being expandable if needed.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In fragmentation, the zbuf is used to copy the header bytes and to append the pointer to the payload so as to have a linear reading of the bytes.
128 is a rules of thumb heuristic about the size of memory you need to serialize the header. If more is needed, another block can be allocated straight after and so on an so forth.

@@ -296,8 +296,7 @@ int8_t _z_bytes_val_encode(_z_wbuf_t *wbf, const _z_bytes_t *bs) {
int8_t ret = _Z_RES_OK;

if ((wbf->_expansion_step != 0) && (bs->len > Z_TSID_LENGTH)) {
// ret |= _z_wbuf_wrap_bytes(wbf, bs->start, 0, bs->len);
ret |= _z_wbuf_write_bytes(wbf, bs->start, 0, bs->len);
ret |= _z_wbuf_wrap_bytes(wbf, bs->start, 0, bs->len);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems that this line is being changed frequently. Last time it was from wrap to write in 5238f25 .

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yah that's one of the whole point of this PR, reverting this change from @p-avital that outlived its usefulness.

@@ -0,0 +1,90 @@
//
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not to integrate the test in the client and/or multicast tests instead of having a new one? Also, because this shall be tested while interoperating with the Rust implementation.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think a standalone test has value as fragmentation will eventually be a removable feature.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with both, but the fact we don't have a test that actually test fragmentation with the Rust implementation is the at root of #291. So, if I have to prioritise, having a test that tests against Rust is more important than on UDP multicast. Keeping in mind that we don't have reliability on UDP multicast.

@jean-roland jean-roland marked this pull request as ready for review December 18, 2023 11:25
@Mallets
Copy link
Member

Mallets commented Dec 18, 2023

Approving this PR. @jean-roland I'd like to see in a separate PR the fragmentation tests working as well with the Zenoh router.

@Mallets Mallets merged commit 6316838 into eclipse-zenoh:master Dec 18, 2023
49 checks passed
@jean-roland jean-roland deleted the fix_fragment branch December 18, 2023 15:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants