ringbuffer: direct mmap consumers

[lmb]

This PR will address the ring buffer portion of #4163

Based on #4265 by @mikeagun

Updates: #4163

[github-actions]

This pull request isn't linked to any GitHub issue. Please reference an issue with a keyword such as Fixes #123, Closes #456, etc., so the work can be tracked.

[lmb]

The code I've pushed works insofar as I can run the ringbuf reader implementation of ebpf-go against it and it seems to mostly work. I've been bashing my head against how to correctly map the ring buffer to user space, and need some help.

To recap:

 ┌────────┌──────────┌──────────┌─────────┐     ┌─────────┐      
 │ Kernel │ Consumer │ Producer │ Data #1 │ ... │ Data #1 │ ...  
 └────────└──────────└──────────└─────────┘     └─────────┘      

• The ring buffer consists of four distinct sections: kernel (which contains wait event), consumer and producer pages, data pages
• The data pages need to be mapped twice, consecutively.
• In the driver, we expect the the mapping of the ringbuffer into virtual memory to be contiguous. This is so that _ring_buffer_notify_consumer can retrieve the wait_handle from the kernel page. The entire buffer is read / write.
• In user space, we don't want to allow access to the kernel page. So the mapping starts at the consumer page. However, only the consumer page is read/write, all other pages are read-only.
• User space is supposed to be able to create multiple mappings.

Creating the double mapping is a bit awkward. Combining it with different page protections make it really difficult.

My current approach is to create an MDL for the kernel space, which just covers all of the buffer. That is mapped as usual, and not a big problem (besides playing fast and loose with page locking).
For user space, I create two partial MDL which cover (1) the consumer page (2) the rest of the buffer. As discussed in the office hours, the problems start when trying to unmap the user space mapping. (Right now we simply leak the virtual address space.)

I first tried to use MmUnmapLockedPages, but that requires access to the original MDL used to create the mapping. How would I use that when there are potentially multiple mappings of the same MDL? Can I simply pass different addresses? It also has the slightly unfortunate side effect that freeing the mappings requires access to the ringbuffer handle. What I mean by this is that on Linux the flow is roughly this:

fd = create_ringbuffer
consumer = mmap(fd, 0, PAGE_SIZE, READ_WRITE)
producer = mmap(fd, PAGE_SIZE, size, READ_ONLY)
close(fd) // consumer and producer are still valid!
munmap(consumer, PAGE_SIZE)
munmap(producer, size)

So I tried to figure out another way to free this and looked into memory sections as suggested by @Alan-Jowett. I can't figure out how to make these work either:

• I don't see an API which reliably allows double mapping of a buffer. ZwMapViewOfSection aligns the base address to 64 kb, and doesn't seem to support MEM_RESERVE_PLACEHOLDER, etc. which we have in user space.
• Also tried to find a way to change page protection for individual mappings, to no avail.

Am I just missing a trick? Why is this so hard on Windows?

cc @mikeagun

[lmb]

As per the discussion in the office hours I just went for the simplest thing and added an API which allows unmapping given an fd.

[github-actions]

This pull request isn't linked to any GitHub issue. Please reference an issue with a keyword such as Fixes #123, Closes #456, etc., so the work can be tracked.

[github-actions]

This pull request isn't linked to any GitHub issue. Please reference an issue with a keyword such as Fixes #123, Closes #456, etc., so the work can be tracked.