Multiprocess encryption can sometimes read stale values

[sync-by-unito]

There are at least two scenarios where multiprocess encryption can incorrectly consider stale values to be up to date. This one is hit by our tests once the shared mapping is removed and they're able to test the multiprocess code paths within one process:

1. Process 1 reads page X
2. Process 2 writes to one byte range in page X
3. Process 1 refreshes the reader mapping and marks the page as StaleIV
4. Process 1 writes to a different byte range in page X
5. This byte range is copied to the read mapping and the page is marked as UpToDate
6. Process 1 reads from the byte range written by process 2 and gets garbage data

This one is more theoretical and it's unlikely anyone has actually hit it:

1. Process 1 reads page X from the Realm on threads A and B.
2. Process 2 writes to page X and also grows the Realm file.
3. Process 1 refreshes the Realm on thread A. Extending the existing reader mapping fails and it creates a new one. This marks all mappings as StaleIV.
4. Process 1 reads page X on thread B without refreshing. This rereads the IV block and then rereads page X because the IV has changed.
5. Process 1 closes the Realm on thread B.
6. More stuff happens and the old reader mapping gets clean up, but importantly no more writes to page X happen.
7. Thread A reads page X. It first checks if any other mappings have an up-to-date copy of the page, and none do as the mapping with it has been discarded. The IV recheck reports no change as it's the same as when thread B checked, so the page is marked up-to-date and Process 2's write is discarded.

[sync-by-unito]

➤ PM Bot commented:

Jira ticket: RCORE-2141

[ironage]

I don't want to undermine the existence of a bug in this complicated code. But could you clarify a few things for the sake of my understanding?

In scenario 1, between steps 3-4, the entire page (all byte ranges) should have been refreshed from disk due to advancing versions and the read barrier that happens before writing.

[tgoyne]

Reading and writing is always done via different mappings. Step 3 marks all of the pages as StaleIV, but doesn't reread anything. The read barrier on the write mapping brings that mapping fully up to date, but doesn't update the reader mapping. The write barrier on the write mapping copies just the modified bytes over to the read mapping, but not the rest of the page, and then clears StaleIV.

[ironage]

Got it, thanks for this analysis. Having multiple mappings of the same data doesn't make things simple.