rkyv support

[djahandarie]

I'm trying to use openraft with 20k+ messages per second, and I seem to be running into quite a bit of overhead simply allocating/copying/serializing stuff. I'm currently using bincode, but would like to switch to rkyv as I believe it will do better.

Unfortunately rkyv doesn't work with serde Serialize/Deserialize (due to fundamental differences around supporting zero-copy), and thus there would need to be support added into openraft for me to be able to use rkyv to serialize things like VoteRequest/AppendEntriesRequest/InstallSnapshotRequest.

I think it would just be a matter of adding some different derives behind a feature flag for rkyv. Is there interest in this? If so I will try to make a PR.

[github-actions]

👋 Thanks for opening this issue!

Get help or engage by:

• /help : to print help messages.
• /assignme : to assign this issue to you.

[drmingdrmer]

There is an ongoing PR to make serde an optional feature:

• Improve: Make serde optional, a dirty work #243

What you need looks similar to this? An optional feature that enables several rkyv derive?

I do not have any experience on rkyv other than reading its API doc:)
A PR is welcomed! :DDD

[MikaelCall]

Adding rkyv support would be a nice addition indeed. It's a very efficient format wrt (de)serialization speed which can easily become a bottleneck.

IMHO the solution outlined in the comment #243 (review) in the currently open serde PR would be a very simple and good solution wrt maintainablity.

[zach-schoenberger]

Hi All, I'm looking at adding this feature and ran into a small issue. Is there any issue with moving away from using AnyError directly in some of the messages? An example: https://github.com/datafuselabs/openraft/blob/db047b978e08fdb6f23bc4c27f94d4bd36ab925b/openraft/src/error.rs#L314-L320
The issue being that AnyError is defined in the external crate so we can't easily add the rkyv traits to it

(Edit)
I see now that the AnyError crate is actually apart of drmindrmer's repositories. I guess my question becomes if rkyv support should be added there?

(Edit 2)
Looking more closely at this, it doesn't appear that the error's themselves would/should ever be sent over the network so it doesn't look like there would be any reason to add this serialization to them. I should have a pr for this sometime today.

[drmingdrmer]

Hi All, I'm looking at adding this feature and ran into a small issue. Is there any issue with moving away from using AnyError directly in some of the messages? An example:

https://github.com/datafuselabs/openraft/blob/db047b978e08fdb6f23bc4c27f94d4bd36ab925b/openraft/src/error.rs#L314-L320

The issue being that AnyError is defined in the external crate so we can't easily add the rkyv traits to it

(Edit) I see now that the AnyError crate is actually apart of drmindrmer's repositories. I guess my question becomes if rkyv support should be added there?

(Edit 2) Looking more closely at this, it doesn't appear that the error's themselves would/should ever be sent over the network so it doesn't look like there would be any reason to add this serialization to them. I should have a pr for this sometime today.

It is possible to add rkyv to AnyError.

But AFAIK, only types that need to be persisted on disk need rkyv support.
Errors in openraft are only used for transport.
Does it have to be encoded in the rkyv format for transport?
E.g., for `send_append_entries():
https://github.com/datafuselabs/openraft/blob/1202d1b059f7e07f0f3353b74f0b2795f063ab56/openraft/src/network.rs#L49-L52

The response and response errors are small structs, maybe it's possible to use some other serialization such as serde to encode responses?