Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use msgpack for the chunks section in frame format #76

Open
FrancescAlted opened this issue Aug 29, 2019 · 0 comments
Open

Use msgpack for the chunks section in frame format #76

FrancescAlted opened this issue Aug 29, 2019 · 0 comments

Comments

@FrancescAlted
Copy link
Member

Packing the chunks section of the frame format would bring more encapsulation and better maneagibility in the end. With that, fields like uncompressed_size and compressed_size and blocksize can be moved into the new chunks section.

Suggestion: add a chunks_size field to the new chunks so that computing the size of the chunk for offsets would be easier, and specially, faster (e.g. remove the need for get_trailer_offsets() in frame_get_chunk()).

After doing this, I am not sure on whether we should add another msgpck field at the front of the header for specifying that three other msgpack sections come later (header, chunks and trailer). Probably this is not a good idea because then utilities like msgpack2json may want to read the whole file, and this can be too much for large files.

@FrancescAlted FrancescAlted changed the title Use msgpack for the chunks section in fram format Use msgpack for the chunks section in frame format Aug 29, 2019
@FrancescAlted FrancescAlted self-assigned this Aug 29, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant