Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use case for multi-memory: Sharing persistent datastructures between threads #43

Open
makoConstruct opened this issue Mar 1, 2023 · 0 comments

Comments

@makoConstruct
Copy link

makoConstruct commented Mar 1, 2023

Functional programmers are going to want to be able to share memory with other processes, because that's one of functional programming's big claims, persistent datastructures consist basically entirely of immutable parts so there's no obstacle to sharing, but from what I can tell, with today's shared memory features, this is unsafe. It could be made safe with a different type of memory that follows different rules, which I'll name Only-Write-On-Allocation (OWOA) memory.

I define OWOA memory as follows:

  • Each allocation can only be written once, during allocation, and is immutable and so safely sharable from then on.
  • References are resource handles. When all handles are released, the memory is freed. In a functional programming language this would tend to involve GC, but rust interfaces could do it with an intrinsic Rc type.
  • All accesses are checked, memory can only be accessed via valid references and accesses to offsets from that reference would need to be bounds checked, I do not know whether this is feasible at all. (You could remove this rule, correct and polite code would still work, but you'd usually want to share an OWOA with broad communities of processes, so incorrect code would be able to browse the OWOA, steal information from other processes by just looking at their allocations, and steal a lot more information through race-conditioned timing attacks)

A purely functional language could get by with just an OWOA memory, but a rust component couldn't, so it seems to me like we'd need multiple memories to make OWOA sharing practical.

I'm bringing this up because this (serialization, copy, deserialization, inability to share persisted datastructures in a large pool) seems like the main disadvantage wasm runtimes have next to erlang runtimes or other functional, compartmented runtimes, if wasm-gc had this, then I think every language could be together, at that point? 🙏

Potential alternative to this, though: I could imagine this being implemented by generally allowing wasm processes to lock parts of their memories and send a resource handle to other processes, that controls the lifetime of that stretch and enables access. This wouldn't require multiple memories, and it's notable that there only needs to be one OWOA memory for the entire runtime, so the need for multiple memories there is already kinda tenuous. But I don't know enough to say whether that would be unacceptably slow in a wasm jit. I'd be curious to know what maintainers think about this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant