MPF VTP Virtual to Physical
The base CCI-P interface expects physical line addresses from the AFU. This makes it difficult to share data structures containing virtually addressed pointers with a CPU. Furthermore, when the kernel allocates shared memory for use by the FPGA, memory is allocated at page granularity. Allocating large buffers requires tracking the base pointers of a large collection of individual physical pages.
On CPUs, the kernel-managed page table and hardware TLB (Translation Lookaside Buffer) combine to support large buffers with contiguous virtual addresses. MPF VTP implements an analogous structure within CCI-P.
MPF VTP provides a pair of functions to allocate and free virtually addressable memory: mpfVtpBufferAllocate() and mpfVtpBufferFree(). These functions are layered on top of the base OPAE functions for memory management, adding the side effect of updating a private page table. While this page table is structurally similar to the standard kernel page table, it is private to MPF and stored in user-space memory that is shared with the FPGA. MPF VTP implements an FPGA-side TLB that walks and caches translations from this private page table. The TLB contains private 512-entry direct-mapped caches on both the read request and write request channels. A shared set associative cache services misses in the direct-mapped caches. A hardware page table walker is also implemented.
VTP supports both standard 4KB and large 2MB pages. AFUs with large memory footprints may perform better when memory is allocated as multiples of 2MB pages. To request 2MB pages:
- Ensure that Linux huge page support is enabled and that sufficient huge pages have been reserved. Pages can be reserved at boot time by adding "hugepagesz=2M hugepages=4096" to the Linux kernel arguments, changing hugepages to a value that balances the number required against the number of physical pages available.
- Request multiples of the huge page size when calling mpfVtpBufferAllocate().
VTP memory allocation is more like mmap than malloc. To manage small chunks of memory, consider layering a heap manager on top of VTP's allocator.
MPF exposes a large set of compile-time preprocessor variables to configure the sizes of TLB caches. See the VTP section of cci_mpf_config.vh.