Lucene FDBDirectory should cache file counter #1577
Assignees
Labels
performance
Performance issues
Comments
alecgrieser
added a commit
to alecgrieser/fdb-record-layer
that referenced
this issue
Mar 30, 2022
…nter This adds a new `AtomicLong` to the `FDBDirectory` where the file sequence counter is cached. The first time a user tries to get an increment, the value is loaded from the database and put into the cache. All subsequent accesses can use that cache, and the `AtomicLong` guarantees thread safety. Note that the FDB read-your-writes cache was already keeping a cached copy, but this removes the extra JNI hop and requirement to go through the FDB network thread in order to get a new increment.
alecgrieser
added a commit
to alecgrieser/fdb-record-layer
that referenced
this issue
Apr 5, 2022
…nter This adds a new `AtomicLong` to the `FDBDirectory` where the file sequence counter is cached. The first time a user tries to get an increment, the value is loaded from the database and put into the cache. All subsequent accesses can use that cache, and the `AtomicLong` guarantees thread safety. Note that the FDB read-your-writes cache was already keeping a cached copy, but this removes the extra JNI hop and requirement to go through the FDB network thread in order to get a new increment.
alecgrieser
added a commit
to alecgrieser/fdb-record-layer
that referenced
this issue
Apr 9, 2022
…nter This adds a new `AtomicLong` to the `FDBDirectory` where the file sequence counter is cached. The first time a user tries to get an increment, the value is loaded from the database and put into the cache. All subsequent accesses can use that cache, and the `AtomicLong` guarantees thread safety. Note that the FDB read-your-writes cache was already keeping a cached copy, but this removes the extra JNI hop and requirement to go through the FDB network thread in order to get a new increment.
alecgrieser
added a commit
to alecgrieser/fdb-record-layer
that referenced
this issue
Apr 9, 2022
…nter This adds a new `AtomicLong` to the `FDBDirectory` where the file sequence counter is cached. The first time a user tries to get an increment, the value is loaded from the database and put into the cache. All subsequent accesses can use that cache, and the `AtomicLong` guarantees thread safety. Note that the FDB read-your-writes cache was already keeping a cached copy, but this removes the extra JNI hop and requirement to go through the FDB network thread in order to get a new increment.
tian-yizuo
added a commit
that referenced
this issue
Apr 21, 2022
Resolves #1577: Lucene FDBDirectory should cache file counter
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
The
FDBDirectoryimplementation keeps track of a counter that is used to assign files an immutable internal ID, and then file names are linked to that ID via the "meta-data subspace". The current implementation re-reads that counter each time the counter is incremented. This should be served from the FDB-maintained RYW cache, so it shouldn't need to go all the way to a remote FDB cluster, but it could still be lifted up into the JVM and stored as anAtomicLongon the directory.The text was updated successfully, but these errors were encountered: