It's not possible to get more than one page fault on a single page on an EDMM system

[arai-fortanix]

On an SGX2/EDMM system, it appears that it's not possible to get
SIGSEGV/SIGBUS signals for multiple faults on the same enclave page,
unless you do an EACCEPT on the page and then remove the page from
the enclave. When the first page fault happens, my application
gets a SIGSEGV, which I can forward to the enclave and handle
inside of the enclave. Subsequent faults to the same page
inside the enclave seem to be swallowed by the kernel, and I don't
get a SIGSEGV delivered.

My application wants to repeatedly fault on this page. It never
wants to add a mapping. This works exactly as I expect on
a non-EDMM system. Repeated accesses to the unmapped page result
in a user-level SIGSEGV (or SIGBUS) for each access. These
faults can be handled by my program.

If I EACCEPT the page and then remove the page from the enclave
(via ioctl(SGX_IOC_ENCLAVE_TRIM)), then I can get one more
page fault on the address, but I have to do the EACCEPT/ioctl
every time I get a page fault. This is both slow and has incorrect
semantics. The page will be briefly accessible, which I don't want.
I could have multiple threads intentionally faulting on the same
address simultaneously. One thread might not fault while the page
is briefly mapped in the address space.

I don't see a way to indicate to the driver that i don't want this
page to be populated on fault. Maybe I'm missing something. For our
purposes, some page faults should result in pages being added to
the enclave, but some should not. Only the enclave knows which faults
are one type or the other. It would be better for us if we had
a way to turn off automatic page provisioning and only add pages
to the enclave when the driver has been specifically requested to,
presumably through an mmap or ioctl call.

Alternatively, is there some way I can reject the page from being
added to the enclave? I tried just calling ioct(SGX_IOC_ENCLAVE_TRIM)
on the page, but this seems to fail unless I've done an EACCEPT()
on the page first.

[arai-fortanix]

After changing our code to make an mprotect(PROT_NONE) call on the addresses that we want to continue to fault, the driver no longer commits the page, and we are able to receive multiple page faults on the same page. So this was user error.

For anyone else who encounters the same issue: if the pages have PROT_READ | PROT_WRITE protections, but are not present in the enclave, the driver will automatically attempt to add a page to the enclave when the enclave induces a fault on one of these pages. If the page protection is PROT_NONE, the driver will not attempt to add a page to the enclave, and instead will just deliver the SIGBUS or SIGSEGV to the appplication.