IStorage: Require lastTransaction() to invalidate DB before returning

[navytux]

Because if lastTransaction() returns tid, for which local database
handle has not yet been updated with invalidations, it could lead to
data corruption due to concurrency issues similar to
#290:

• DB refreshes a Connection for new transaction;

• zstor.lastTransaction() is called to obtain database view for this connection.

• objects in live-cache for this Connection are invalidated with
  invalidations that were queued through DB.invalidate() calls from
  storage.

• if lastTransaction does not guarantee that all DB invalidations for
  transactions with ID ≤ returned tid have been completed, it can be
  that:

  incomplete set of objects are invalidated in live cache

  i.e. data corruption.

This particular data corruption has been hit when working on core of
ZODB and was not immediately noticed:

#307 (review)

this fact justifies the importance of explicitly stating what IStorage
guarantees are / must be in the interface.

This guarantee

• already holds for FileStorage (no database mutations from outside of
  single process);

• is already true for ZEO4 and ZEO5
  mvccadapter: check if the last TID changed without invalidation #307 (review)
  mvccadapter: check if the last TID changed without invalidation #307 (comment)

• holds for RelStorage because it implements IMVCCStorage natively;

• is not currently true for NEO because NEO sets zstor.last_tid before
  calling DB.invalidate:

  https://lab.nexedi.com/nexedi/neoppod/blob/fc58c089/neo/client/handlers/master.py#L109-124

However NEO is willing to change and already prepared the fix to provide
this guarantee because of data corruption scenario that can happen
without it:

#307 (comment)
https://lab.nexedi.com/nexedi/neoppod/commit/5870c5d7

In other words all storages that, to my knowledge, are in current use
are either already providing specified semantic, or will be shortly
fixed to provide it.

This way we can fix up the interface and make the semantic clear.

/cc @jamadden, @vpelletier, @arnaud-fontaine, @jwolf083, @klawlf82, @gidzit, @jimfulton

[jmuchemb]

I want to close it. I see no reason to work on this separately to #307.

[navytux]

Updated patch with the following interdiff due to @jmuchemb notice:

--- a/src/ZODB/interfaces.py
+++ b/src/ZODB/interfaces.py
@@ -687,7 +687,7 @@ def lastTransaction():
         """Return the id of the last committed transaction.

         Returned tid is ID of last committed transaction as observed from
-        some time _before_ lastTransaction call was made. In particular for
+        some time _before_ lastTransaction call returns. In particular for
         client-sever case, lastTransaction can return cached view of storage
         that was learned some time ago.

[navytux]

The interface was changed via different text as part of #307 (4a6b028#diff-881ceb274f9e538d4144950eefce8682R685-R695). That change was discussed here: #307 (comment). Closing.