Overhaul API

[sfackler]

• Remove MmapView and MmapViewSync types
  • While it is very common to need something like these types when working with memory maps, the exact semantics tend to differ enough that you can't actually use these types. In addition, they're simple enough that removing them won't have too much of a negative impact.
• Move the "unsafety location" from access to construction
  • Define two types - Mmap and MmapMut for readable and readable/writable mappings respectively. They implement Deref<Target= [u8]>/DerefMut as appropriate. In contexts where access to the mapped region could cause faults (e.g. file backed maps on Unix) make the constructor unsafe.
• Keep a couple of convenience constructors on Mmap and MmapMut, but move advanced configuration to an OpenOptions-style builder. The builder will provide a cross-platform API directly with OS specific extension traits.

[brson]

This looks like something somebody new to the memmap crate could pick off, but maybe needs some guidance from @danburkert.

[jonas-schievink]

Note that a way of converting Mmap <=> MmapMut is required in order to not lose features like the ability to write a safe JIT. Also I think someone requested a way to make unreadable but executable mappings, how would those be integrated?

[sfackler]

We discussed adding a third MmapUnsafe type for more "exotic" cases that wouldn't implement Deref but would just expose the pointer and len.

[ghost]

Hey hi! II'm new to memmap as @brson said, but I'd like to help here :)

[brson]

@z1mvader it looks like @bugaevc has a [PR] opened on this subject already.

[danburkert]

@bugaevc landed the first cleanup patch, thanks!

I think the next major piece that needs to be overhauled is the Protection system. Ideally, I think we could get rid of it and replace it with more descriptive methods. For instance:

impl<'a> FileMmapOptions<'a> {
    /// mmap(ptr, len, PROT_READ, MAP_SHARED, fd, offset)
    pub fn map(&self) -> Result<Mmap> { ...  }
    /// mmap(ptr, len, PROT_READ | PROT_EXECUTE, MAP_SHARED, fd, offset)
    pub fn map_execute(&self) -> Result<Mmap> { ... }
    /// mmap(ptr, len, PROT_READ | PROT_WRITE, MAP_SHARED, fd, offset)
    pub fn map_mut(&self) -> Result<MmapMut> { ... }
    /// mmap(ptr, len, PROT_READ | PROT_WRITE, MAP_PRIVATE, fd, offset)
    pub fn map_copy(&self) -> Result<MmapMut> { ...  }
}

impl AnonymousMmapOptions {
    /// mmap(ptr, len, PROT_READ | PROT_WRITE, MAP_SHARED | MAP_ANON, fd, offset)
    pub fn map_mut(&self) -> Result<MmapMut> { ...  }
}

impl Mmap {
    // mprotect(ptr, len, PROT_READ | PROT_WRITE)
    pub fn make_mut(mut self, protection: Protection) -> Result<MmapMut> { ... }
}

impl MmapMut {
    // mprotect(ptr, len, PROT_READ)
    pub fn make_read_only(mut self) -> Result<Mmap> { ...  }
    // mprotect(ptr, len, PROT_READ | PROT_EXEC)
    pub fn make_read_exec(mut self) -> Result<Mmap> { ...  }
    ...
    // flush methods
}

Open questions:

• Does this provide the full range of flexibility that people need? Obviously not every option provided by the mmap system call is reproduced, but ideally we can cover all common cross-platform usecases with this API.
• Will this map well to Windows, especially in light of #31
• Naming

[retep998]

Note that a way of converting Mmap <=> MmapMut is required in order to not lose features like the ability to write a safe JIT. Also I think someone requested a way to make unreadable but executable mappings, how would those be integrated?

What you really want here is an API for managing virtual memory, not an API for anonymous memory mapped files. While they may both use mmap on linux, on Windows they are extremely different.

In my opinion, the needs and use cases that people express for memory maps are probably quite skewed by the fact that people abuse memory maps for virtual memory management due to the a lack of good alternative. As a result, the sooner we get a dedicated API for virtual memory management, the sooner we'd be able to know what we actually need from regular memory maps.

[brson]

I see all the op checkboxes are checked but that @danburkert has a linked PR open. So I take it that there's one more thing to do here.

Closing this will unblock #32 and #37

[Lapin0t]

I would love to have a higher-level API which would much more look like an allocator, ie something that return something more or less like Box<T> to the user. The new placement protocol might be interesting to look at too. I'm currently experimenting with things somewhat like this in a personal project (but in a specific case and without seeking to write generic code), but i would be interested to contribute back to this crate afterwards. Does this sound interesting or is it out of scope for this crate?

[danburkert]

I think higher level abstractions are going to be out of scope for this crate, we used to have some in crate but they've since been taken out. I think it would be interesting to do experiments with the new Allocator API and memory maps, but ideally it can be done outside this crate (if not, we should fix that).

[danburkert]

Woops, there are definitely still refactors coming (see #46).