Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prototype use of zerocopy crate in zerovec #2798

Open
sffc opened this issue Nov 1, 2022 · 0 comments
Open

Prototype use of zerocopy crate in zerovec #2798

sffc opened this issue Nov 1, 2022 · 0 comments
Labels
C-zerovec Component: Yoke, ZeroVec, DataBake help wanted Issue needs an assignee S-medium Size: Less than a week (larger bug fix or enhancement) T-task Type: Tracking thread for a non-code task

Comments

@sffc
Copy link
Member

sffc commented Nov 1, 2022

Some of the operations in the zerovec crate, especially those involving ULE, could be modeled with abstractions in the zerocopy crate. Parts of the zerocopy crate are being proposed to the standard library in Safe Transmute. It would be good to see how much unsafe code could be eliminated in zerovec by adding zerocopy as a dependency, so that we can influence the Safe Transmute RFC and prepare longer-term for zerovec sitting on top of those operations.

The biggest catch is that zerocopy doesn't currently support fallible conversions, which is quite fundamental to how ULE works. This could be partly resolved by a private infallible conversion function invoked from a custom public fallible function, but zerocopy also does not currently support private conversion functions. @joshlf said in an email:

The problem is that there's no way, using zerocopy, to enable a transmute only for a single module - if your type is public and the transmute is possible, it's possible for everyone. I'd think that you could support your use case by defining a private type, defining your public type as a repr(transparent) wrapper around the private type with extra invariants, doing a transmute using zerocopy operations on the private type, and then doing a single unsafe step at the end to convert to your public type (thanks to repr(transparent), the safety argument should be very simple).

We'd also like const conversions; we have them in some places, but not everywhere. I would love to see a generalized const conversion API so that we can make a proper zeroslice! macro as proposed in #1935.

CC @joshlf @ezrosent

@sffc sffc added T-task Type: Tracking thread for a non-code task C-data-infra Component: provider, datagen, fallback, adapters S-medium Size: Less than a week (larger bug fix or enhancement) labels Nov 1, 2022
@sffc sffc added help wanted Issue needs an assignee backlog labels Dec 22, 2022
@sffc sffc added this to the Backlog milestone Dec 22, 2022
@sffc sffc added C-zerovec Component: Yoke, ZeroVec, DataBake and removed backlog C-data-infra Component: provider, datagen, fallback, adapters labels Dec 22, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-zerovec Component: Yoke, ZeroVec, DataBake help wanted Issue needs an assignee S-medium Size: Less than a week (larger bug fix or enhancement) T-task Type: Tracking thread for a non-code task
Projects
None yet
Development

No branches or pull requests

1 participant