Use finer-grained locking during polling.

Add a base MVCCDatabaseCoordinator that does a better job tracking the minimum required TID.

Misc cleanups.

CHANGES.rst

@@ -24,37 +24,62 @@
 - Fix ``undo`` to purge the objects whose transaction was revoked from
   the cache.
 
-- History-free databases remove objects from the local cache when they
-  are replaced by a newer revision. This helps the cache size stay
-  in check. This partly rolls back the conflict resolution
-  enhancements in 3.0a7 when the updater and the conflicting updater
-  are in the same process. That will be improved.
-
-- Make new connections automatically stay up-to-date with the most
-  recent polling that's been done. In environments with a large amount
-  of write activity, this makes them much more useful immediately,
-  without having to perform large polls. Older connections, though,
-  continue to need large polls if they're used again for the first
-  time in a long time. That is expected to be improved. For now,
-  reducing the ZODB connection pool size, or enabling its idle
-  timeout, may help.
-
-- Only one connection at a time for a given database will ever perform
-  a "Using new checkpoints" poll query, which can be very expensive
-  given large cache-delta-size values. Other connections will use the
-  results of this query and make minor updates as needed.
-
-- Stop reading the current checkpoints from memcache, if one is
-  configured. Memcache integration has been discouraged since the
-  introduction of RelStorage persistent caches, which are much
-  improved in 3.0. Having checkpoint data come from there is
-  inconsistent with several of the new features that let the local
-  cache be smarter and more efficient.
-
 - Make historical storages read-only, raising
   ``ReadOnlyHistoryError``, during the commit process. Previously this
   was only enforced at the ``Connection`` level.
 
+- Rewrite the cache to understand the MVCC nature of the connections
+  that use it.
+
+  This eliminates the use of "checkpoints." Checkpoints established a
+  sort of index for objects to allow them to be found in the cache
+  without necessarily knowing their ``_p_serial`` value. To achieve
+  good hit rates in large databases, large values for the
+  ``cache-delta-size-limit`` were needed, but if there were lots of
+  writes, polling to update those large checkpoints could become very
+  expensive. Because checkpoints were separate in each ZODB connection
+  in a process, and because when one connection changed its
+  checkpoints every other connection would also change its checkpoints
+  on the next access, this could quickly become a problem in highly
+  concurrent environments (many connections making many large database
+  queries at the same time). See :issue:`311`.
+
+  The new system uses a series of chained maps representing polling
+  points to build the same index data. All connections can share all
+  the maps for their view of the database and earlier. New polls add
+  new maps to the front of the list as needed, and old mapps are
+  removed once they are no longer needed by any active transaction.
+  This simulates the underlying database's MVCC approach.
+
+  Other benefits of this approach include:
+
+  - No more large polls. While each connection still polls for each
+    transaction it enters, they now share state and only poll against
+    the last time a poll occurred, not the last time they were used.
+    The result should be smaller, more predictable polling.
+
+  - Having a model of object visibility allows the cache to use more
+    efficient data structures: it can now use the smaller LOBTree to
+    reduce the memory occupied by the cache. It also requires
+    fewer cache entries overall to store multiple revisions of an
+    object, reducing the overhead. And there are no more key copies
+    required after a checkpoint change, again reducing overhead and
+    making the LRU algorithm more efficient.
+
+  - The cache's LRU algorithm is now at the object level, not the
+    object/serial pair.
+
+  - Objects that are known to have been changed but whose old revision
+    is still in the cache are preemptively removed when no references
+    to them are possible, reducing cache memory usage.
+
+  - The persistent cache can now guarantee not to write out data that
+    it knows to be stale.
+
+  Dropping checkpoints probably makes memcache less effective, but
+  memcache hasn't been recommended for awhile.
+
+
 3.0a8 (2019-08-13)
 ==================
 

src/relstorage/_mvcc.py

@@ -0,0 +1,210 @@
+# -*- coding: utf-8 -*-
+"""
+Helper implementations for MVCC.
+"""
+from __future__ import absolute_import
+from __future__ import division
+from __future__ import print_function
+
+import threading
+
+from zope import interface
+
+from BTrees import family64
+
+from .interfaces import IMVCCDatabaseCoordinator
+from .interfaces import IDetachableMVCCDatabaseViewer
+
+@interface.implementer(IDetachableMVCCDatabaseViewer)
+class DetachableMVCCDatabaseViewer(object):
+    __slots__ = (
+        'highest_visible_tid',
+        'detached',
+    )
+
+    def __init__(self):
+        self.highest_visible_tid = None
+        self.detached = False
+
+
+@interface.implementer(IMVCCDatabaseCoordinator)
+class DetachableMVCCDatabaseCoordinator(object):
+    """
+    Simple implementation of :class:`IMVCCDatabaseCoordinator`
+    that works with :class:`relstorage.interfaces.IDetachableMVCCDatabaseViewer`
+    objects.
+
+    We keep hard references to our viewers, so if they reference us there
+    could be a cycle. Viewers must be hashable.
+
+    The ``highest_visible_tid`` and ``detached`` values of the viewer
+    must *only* be managed through this object.
+    """
+
+    maximum_highest_visible_tid = None
+    minimum_highest_visible_tid = None
+
+    def __init__(self):
+        # Manipulations of metadata must be locked.
+        # We don't always hold the lock; we rely on primitive operations of
+        # the set() in _registered_viewers to be atomic.
+        self._lock = threading.RLock()
+        # {tid: {viewer, ...}} of objects not detached and not None
+        self._by_tid = family64.IO.Bucket()
+        self._registered_viewers = set()
+        self.is_registered = self._registered_viewers.__contains__
+
+    @property
+    def _viewer_count_at_min(self):
+        # Testing.
+        if not self._registered_viewers or not self.minimum_highest_visible_tid:
+            return 0
+        viewers = self._by_tid.values()[0]
+        return len(viewers)
+
+    def register(self, viewer):
+        with self._lock:
+            if self.is_registered(viewer):
+                return
+            self._registered_viewers.add(viewer)
+            if viewer.detached:
+                return
+
+            hvt = viewer.highest_visible_tid
+            if hvt is None:
+                return
+
+            __traceback_info__ = hvt, viewer
+            self._by_tid.setdefault(hvt, set()).add(viewer)
+            self.minimum_highest_visible_tid = self._by_tid.minKey()
+            self.maximum_highest_visible_tid = self._by_tid.maxKey()
+
+    def unregister(self, viewer):
+        with self._lock:
+            if not self.is_registered(viewer):
+                return
+
+            self._registered_viewers.remove(viewer)
+            if not self._registered_viewers:
+                self.minimum_highest_visible_tid = None
+                self.maximum_highest_visible_tid = None
+                self._by_tid.clear()
+                return
+
+            self.__viewer_does_not_matter(viewer)
+
+    def __set_tids(self):
+        by_tid = self._by_tid
+        if by_tid:
+            self.minimum_highest_visible_tid = by_tid.minKey()
+            self.maximum_highest_visible_tid = by_tid.maxKey()
+        else:
+            self.minimum_highest_visible_tid = None
+            self.maximum_highest_visible_tid = None
+
+
+    def __viewer_does_not_matter(self, viewer):
+        # Because it was unregistered or because it
+        # was detached.
+        hvt = viewer.highest_visible_tid
+        by_tid = self._by_tid
+        if by_tid and hvt:
+            viewers = by_tid.get(hvt)
+            if viewers:
+                viewers.discard(viewer)
+                if not viewers:
+                    del by_tid[hvt]
+                self.__set_tids()
+
+    def clear(self):
+        with self._lock:
+            self._registered_viewers.clear()
+            self._by_tid.clear()
+            self.__set_tids()
+
+    def detach(self, viewer):
+        """
+        Cause the viewer to become detached.
+        """
+        with self._lock:
+            if not self.is_registered(viewer):
+                return
+
+            viewer.detached = True
+            self.__viewer_does_not_matter(viewer)
+
+    def detach_all(self):
+        with self._lock:
+            for viewer in self._registered_viewers:
+                viewer.detached = True
+            self._by_tid.clear()
+            self.__set_tids()
+
+    def change(self, viewer, new_hvt):
+        """
+        Cause the viewer to have a new ``highest_visible_tid``,
+        which can be greater, less, or equal to the current HVT,
+        or None.
+
+        If the *viewer* was previously detached, it is now attached.
+        """
+        with self._lock:
+            if not self.is_registered(viewer):
+                return
+
+            viewer.detached = False
+
+            old_hvt = viewer.highest_visible_tid
+            viewer.highest_visible_tid = new_hvt
+            by_tid = self._by_tid
+            if old_hvt:
+                viewers = by_tid.get(old_hvt)
+                if viewers:
+                    viewers.discard(viewer)
+                    if not viewers:
+                        del by_tid[old_hvt]
+            if new_hvt:
+                by_tid.setdefault(new_hvt, set()).add(viewer)
+
+            self.__set_tids()
+
+    def viewers_at_or_before(self, tid):
+        """
+        Return all the viewers with tids at least as old as the
+        given tid.
+
+        Passing the value from ``minumum_highest_visible_tid`` is always safe,
+        even if that value is None. If that value is None, it's because we
+        have no viewers, or the viewers we do have haven't looked at the
+        database; they'll be ignored.
+
+        Viewers that are already explicitly detached are also ignored.
+        """
+        with self._lock:
+            by_tid = self._by_tid
+            if not by_tid:
+                return ()
+            sets_before = by_tid.values(max=tid, excludemax=False)
+            return set().union(*sets_before) if sets_before else ()
+
+    def viewers_at_minimum(self):
+        """
+        Return all the viewers viewing the ``minimum_highest_visible_tid``.
+
+        If that is None, this is the empty set.
+        """
+        with self._lock:
+            if self._by_tid:
+                return self._by_tid.values()[0]
+            return ()
+
+    def detach_viewers_at_minimum(self):
+        """
+        Cause all the viewers at the minimum, if any, to be detached.
+        """
+        with self._lock:
+            if self._by_tid:
+                at_min = self._by_tid.pop(self._by_tid.minKey())
+                for viewer in at_min:
+                    viewer.detached = True
+                self.__set_tids()

src/relstorage/cache/interfaces.py

@@ -22,7 +22,7 @@
 from transaction.interfaces import TransientError
 from ZODB.POSException import StorageError
 
-from relstorage.interfaces import IMVCCDatabaseViewer
+from relstorage.interfaces import IDetachableMVCCDatabaseViewer
 from relstorage.interfaces import IMVCCDatabaseCoordinator
 
 # Export
@@ -32,7 +32,7 @@
 # pylint:disable=unexpected-special-method-signature
 # pylint:disable=signature-differs
 
-class IStorageCache(IMVCCDatabaseViewer):
+class IStorageCache(IDetachableMVCCDatabaseViewer):
     """
     A cache, as used by :class:`relstorage.interfaces.IRelStorage`.