New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
UnsafeCell in Archived type incompatible with miri #303
Comments
I also tried wrapping the serialized buffer in an
|
Expanding on this a bit more, I'm hesitant to wrap the mmap'd buffer in a More concretely, I imagine that having the buffer behind a mutable reference signals to the compiler that nothing inside will be mutated behind its back, since it has the only reference. This assumption is false, of course, because the buffer will be mmap'd into other processes, with their own (mutable) references. |
I actually think that using an |
In my use case, multiple processes would be mmap'ing the same shared memory pages and deserializing it. If those processes each construct a |
Hm, that's true. It seems like this kind of external shared memory is not something Rust was intended to work with natively. I do still think that using a mutable buffer would work, but there is a problem with reads. We would need to treat everything as potentially volatile, which is something that rkyv is not able to do right now. |
I think treating the buffer as a shared/const reference and later using interior mutability ought to be workable. We need to be careful that the Archive types are The snag we're hitting here is just how to express it in a way compatible with the Stacked Borrows proposal being checked by miri. I'm still fuzzy on the details, but I followed a hunch that it didn't like some of the intermediate steps. If I get rid of the intermediate slice and cast the pointer directly, miri seems to think it's ok:
|
Actually this is starting to make sense. In the current helpers, there's a point in time where the same underlying data has both a slice of |
I realized another issue when archiving types with interior mutability - rkyv::archived_root doesn't require pinning, unlike rkyv::archived_root_mut. When IDK if this is a bug per se, or covered by the safety requirements already put on the user via the I'm starting to sketch out a new helper for my own use that addresses both issues:
|
If a type has interior mutability and produces a mutable reference to some internal value, then it must ensure that the reference obeys the typical aliasing rules. For example, I think the correct way to solve this issue would be to create an archived variant of Your idea for returning a pinned shared reference is on the right track, but not quite correct. It's not your fault though, pinning is notoriously difficult to get right. Pinning a shared reference doesn't actually do anything because you can just get the shared reference back out with |
Hah, whoops!
Hmm, yeah I think that makes sense. |
I think I've addressed the pinning issues in my WIP mutex API. I have a PR out now being reviewed by a teammate; would be happy to have your review there too. Otherwise if you're interested I could separately take a stab at merging it into rkyv itself, and you can review there. shadow/shadow#2386 |
I think the PR looks good in general. Because of the reliance on libc, I think this would be better suited for rkyv_contrib if you wanted to upstream it. I think it would be beneficial to add the pointer-only API for |
Thanks! Yeah, I could drop the nix dependency pretty easily, but dropping libc would be trickier. I'll have a look at adding to rkyv_contrib |
Since it seems like this issue has been resolved, I'm going to close this issue. Feel free to reopen if I've missed something! |
I'm still using my own replacement for
|
I'm attempting to create a mutex type that can be used in its Archived form, to facilitate safe mutation of archived data mmap'd into multiple processes.
I have it implemented, and some basic tests working, but noticed that
rkyv::archived_root
results in an error in miri.rkyv::archived_root_mut
works ok. Maybe there's a fundamental reason for that, and the answer is "just userkyv::archived_root_mut
", but I think it might be worth digging into a bit more.I created a smaller test case that seems to show the problem is more fundamentally around
UnsafeCell
, which would be needed for interior mutability in anyArchive
type.The test passes natively, but under miri fails:
The text was updated successfully, but these errors were encountered: