Optimize EOS VM OC memory slice count for parallel read-only transactions

[bhazzard]

Description
Reduce the threshold where EOS VM OC transitions from mirroring to mprotect() to conserve virtual memory, in order to enable parallel read-only transactions in a resource-efficient manner.

Summary
The current implementation of EOS VM OC uses a memory mirroring technique that requires a significant amount of virtual memory to support larger WASM memory sizes. With the recent modifications to support up to 4GiB of WASM memory, EOS VM OC now requires about 4.2TiB of virtual memory to support 16 parallel read-only transactions. This poses a significant challenge as it exceeds the virtual memory limit on most processors.

To address this challenge, the team needs to reduce the threshold at which EOS VM OC transitions from memory mirroring to mprotect(). This will help conserve virtual memory and enable parallel read-only transactions in a more resource-efficient manner. The team plans to gather data points from existing contract usage to determine a suitable threshold and add a compile knob to define the threshold number of pages where the transition occurs.

This work will contribute to the larger initiative of taking further advantage of parallel processing in LEAP v5.0.0. The EOS and Antelope community will benefit from improved performance and scalability of the platform.

Tasks

• Research existing contract usage for memory page statistics
• Determine a suitable threshold for transitioning from mirroring to mprotect()
• Add a compile knob to define the transition threshold
• Test the new threshold on a development environment
• Measure performance impact of the new threshold
• Document the new threshold and its impact
• Review and merge the changes into the main branch
• Update relevant documentation and communicate the change to the community

Original Issue
This was originally reported by @spoonincode. Here is the text of Matt's original issue:

This effort will be required to allow EOS VM OC to reasonably be used for parallel read only transactions.

EOS VM OC uses a memory mirroring technique so that WASM linear memory can be both protected via page access permissions and be resized without usage of mprotect(). EOS VM OC refers to these mirrors as "slices".

Prior to 2.1/3.1, Antelope's WASM memory could never exceed 33MiB. EOS VM OC would set up 33MiB/64KiB+1=529 slices
each with approximately 4GiB+33MiB of virtual memory. This meant that EOS VM OC required approximately 
529*(4GiB+33MiB) of virtual memory; about 2.1TiB.

In 2.1+/3.1+ Antelope technically (but Leap does not reliably) supports WASM memory up to full 4GiB (Leap's supported 
WASM limits are only those as [defined in the reference contracts](https://github.com/AntelopeIO/reference-contracts/blob/efff1254806aa2df68d06f5d5e4b5c122297a857/contracts/eosio.system/src/eosio.system.cpp#L176) which
remain 33MiB). EOS VM OC was modified so that any growth beyond 33MiB is handled via mprotect(). This allows the
optimization to remain in replace for all supported usages of Leap, while still allowing Leap to technically support the full 
Antelope protocol which allows any size up to 4GiB. However, this support still required increasing the size of a slice to a full 
8GiB of virtual memory, meaning that EOS VM OC now requires 529*8GiB of virtual memory; about 4.2TiB.

Executing parallel read only transactions via EOS VM OC will require a set of slices for each executing thread. If 16 parallel 
threads are allowed with the current strategy of requiring 529 slices per set, that would require 16*4.2TiB of virtual memory: 
more virtual memory than allowed on most processors. Future efforts, like sync calls and background memory scrubbing, will 
also increase the need for more active slice sets.

We need to reduce the threshold where EOS VM OC transitions from mirroring to mprotect() to conserve virtual memory. Ideally 
we gather some data points from existing contract usage to know what's a good cut off. It wouldn't surprise me if most contracts 
use less than 1MiB, but I would be curious to see some statistics and measurements. It should be fairly simple to add a compile 
knob (not "public" in cmake or such) defining the threshold number of pages where the transition between the two approaches 
occurs.

[bhazzard]

@spoonincode can you confirm that this captures the spirit of your original issue, and that the task list seems accurate?

[stephenpdeos]

@bhazzard to consolidate into #148

[greg7mdp]

EOS VM OC uses a memory mirroring technique so that WASM linear memory can be both protected via page access permissions and be resized without usage of mprotect().

@spoonincode , what is the benefit of using this mirroring technique vs mprotect. I assume it is for performance. Have we measured the difference?

[spoonincode]

Right, it's a performance improvement. Unfortunately today the improvement can't fully be realized because,

• a syscall is still being used to change the segment register (not required as of Linux 5.9+), and
• we don't pre-clear the memory

but if those improvements were to be made, growing memory would literally be changing the value of a register, full stop. That's obviously significantly faster than mprotect which requires a kernel call (possibly quite expensive on older CPUs with MDS problems), and which (historically at least) may need to take a global process lock to invalid the TLB, and likely needs brand new pages faulted in for the new mapping.

[greg7mdp]

Thanks Matt. Still, how often are we growing memory when executing an action? Maybe we could have a higher initial memory, and larger increments, so that we wouldn't have to grow memory often, and the cost of the mprotect as a percentage of the contact execution time would be negligible?
Do you think this could be an option? And if we did that, maybe this issue would go away?
I would not think it would be much of an issue allocating more virtual memory, since only what we actually use would be page-faulted and mapped to actual memory.

[bhazzard]

Closing this one. Will track AntelopeIO/leap#645 as part of #149

Not deleting, for conversation history.