Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support serialization with bincode v2 #23

Merged
merged 3 commits into from Mar 25, 2022
Merged

Support serialization with bincode v2 #23

merged 3 commits into from Mar 25, 2022

Conversation

vbkaisetsu
Copy link
Member

This branch uses bincode for serialization and deserialization features of Model and Predictor.

The deserialization feature of the Predictor is unsafe and users should not load serialized data provided by thirdparty distributors, so the predict command does not support loading serialized Predictors.

I will add an example usage in another branch.

@vbkaisetsu vbkaisetsu marked this pull request as ready for review March 24, 2022 03:31
Copy link
Member

@kampersanda kampersanda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for your update.

Before carefully reviewing the code, I remember that bincode was removed once in #18 (of internal repo) because of the write/read speed.

With this modification, are there any major speed issues?

@vbkaisetsu
Copy link
Member Author

vbkaisetsu commented Mar 24, 2022

Previously, I tried bincode v1.
bincode v2 is a little bit slower than our implementation, but the model size can be reduced.

w/o tags

bincode:            1,956,943 bytes
our implementation: 1,993,550 bytes

w/ tags

bincode:            2,834,891 bytes
our implementation: 3,014,008 bytes

@vbkaisetsu
Copy link
Member Author

Since it is not very slow, I want to employ bincode v2 for ease of maintenance.

@vbkaisetsu vbkaisetsu merged commit adbf490 into main Mar 25, 2022
@vbkaisetsu vbkaisetsu deleted the serialization branch March 25, 2022 01:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants