Improve how page handling is done

[chotchki]

Currently all pages in the system are being handled via Tokio's BytesMut struct and related traits. This struct is awesome for giving me easy read/write methods on an underlying set of pointers.

However I have discovered a critical issue that I am uncertain how to solve effectively.

So right now I have a combination of features in my lock/cache system:

• Reader/Writer locks for a given page
• A least recently used cache of pages
• A way to check out pages and if discarded a way to purge the cache.

The issue I am facing is as follows:

1. A page is is checked out for read or write.
2. Read/writes are done to it but are discarded for any number of reasons.
3. The next process checks out the same page.
4. The process will not see the entire page since it has been consumed partially. This leads to interesting and painful bugs as I discovered in my index work.

[chotchki]

Possible Fix:

• Right now the lock/cache layer provides RwLocks over a BytesMut based on a requested (PageId, PageOffset). This does cause pain due to a split between reads and writes. I did this because its trivial to replace the contents of a page without extra allocations. However I don't think I can get away with that any more.
• Instead I think I'll provide a RwLock over (PageId, PageOffset). That lock will determine if you can update the underlying buffer.

However I don't know how to make sure other queued users get the updated page.