Turn off EPC paging

[az-hasan]

Hello,

I am running a program assigning a buffer inside the enclave using malloc, at runtime. I then perform some read/write operations on the buffer. I can see performance slow down and then drop once the EPC limit is reached, with a large buffer. I understand that the EPC size is set to 128MB maximum.

1. Is there a way to turn off EPC paging in SGX, to ensure the active pages are not swapped out?

2. I was told there was no way to turn off the swap feature without changing the driver code. I am looking at sgx_page_cache.c, at a function called "sgx_swap_pages" - what must I change to turn off the swap feature?

3. If I stick to sizes small enough that I know will not trigger paging, will I be able to load the enclave, if I turn off swap?

Thank you

[sergeay]

The EPC area contains both your data (the buffer but also others such as stack, global variables your binary code...). When the size of your "working set" passes the 96MB limit (and not 128MB since 32MB are reserved), then the sub system (the driver) is starting to swap. This is not a "nice to have" feature but a necessary feature in order to have your running environment up and running. If swapping was disabled, either the allocation or your buffer access would fail and your program would crash / exit much before.

[az-hasan]

Got it. I did notice the drop between 90-100MB - the 32 MB reserved explains it.
So the pages of the EPC are managed by the Linux memory management subsystem, and if I want to ensure that for the entire run of the program, the EPC pages are not swapped out, I need to turn off the Linux swap feature (with the risk of program crashing + not being able to load an Enclave bigger than the EPC). I don't have to worry about any SGX settings.

Thank you for the clear explanation.

[sergeay]

This is almost true:

So the pages of the EPC are managed by the Linux memory management subsystem
Not exactly but managed by the SGX driver.
...I need to turn off the Linux swap feature...
No, this will not help and I did not know that such an option exists. You could cripple the driver code. As you said you would expose yourself to crashes

[az-hasan]

Ok, so the EPC pages are managed by the SGX driver, not the Linux memory management subsystem. And modifying the driver code is not trivial, and there are no settings I can change anywhere to prevent paging out of the EPC pages?

[sergeay]

At your own risk...
You probably can comment out the kernel swapper thread creation.
Easy to find, the main method is: ksgxswapd.
Your system will probably become unstable.

[rriley]

Hi, I'm working with the original question asker. In our case, we're doing some performance testing and want to make sure that our tests are not triggering swapping without us realizing it. The thought with turning it off was that we'd rather have the enclave crash due to running out of memory. (Because then we know we screwed up and should lower our memory usage.)

Thanks for your feedback and help!