Skip to content

Conversation

alecgrieser
Copy link
Collaborator

@alecgrieser alecgrieser commented Sep 26, 2025

This refactors the way that index match candidates are generated so that individual index maintainer factory implementations can override how this is done for the index types they manage. This includes plumbing through the store's IndexMaintainerRegistry to the CascadesPlanner so that it can ask the right logic for the match candidate, as well as some refactoring done to the MatchCandidate logic so that, if all goes well, index implementors can rely on library code as much as possible.

As part of this approach, I modified the IndexMaintainerRegistry interface somewhat. That interface is backed by a map from index types to IndexMaintainerFactory objects. The registry implementation is typically created once at JVM startup, using the service loader, though in theory someone could override that. This adds two new related interfaces, IndesMaintainerFactoryRegistry, which returns access to the index maintainer factory directly, and IndexMatchCandidateRegistry, which can create a match candidate for a given index. The IndexMaintainerFactory class is modified to have a new method to create match candidates, and the IndexMatchCandidateRegistry will forward that request to the appropriate IndexMaintainerFactory.

This was done instead of putting a new method on the index maintainer itself. The idea there is to keep separate the concept of actually maintaining the index, which requires (for example) having a transaction opened, with the construction of artifacts that are related to the index. The index maintainer factory already had the ability to create both the index maintainer, as you might expect, but also the index validator. So this should be a natural extension to also create the index match candidate.

One other benefit: the index maintainer is bound to a single transaction. That would mean that if we wanted to separate out planning from the context of a transaction (e.g., offline planning), we'd have trouble if we were to do match candidate generation in the index maintainer itself. This aims to allow us to modify that in the future.

There's more refactoring of the MatchCandidateInterface which is designed to make it easy for use cases to create value-like match candidate definitions. There's a test class that shows that.

@alecgrieser alecgrieser added the enhancement New feature or request label Sep 26, 2025
@alecgrieser alecgrieser marked this pull request as ready for review September 29, 2025 18:08
* @return a collection of {@link MatchCandidate}s to match the index against
*/
@Nonnull
default Iterable<MatchCandidate> createMatchCandidates(@Nonnull RecordMetaData metaData, @Nonnull Index index, boolean reverse) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I cannot remember the specifics but the reverse here seems odd.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I looked it up and it's only the planning of RecordQuerys that need this. Annoying...

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I did notice that, but I didn't look too deeply into it. I'm not actually sure why we can't rely on the sort order on the sort descriptor. If we could, I'd be happy to remove this parameter

* Moved the initiliazer of `CascadesPlanner` that took an `FDBRecordStoreBase` to a method on `FDBRecordStoreBase`
* Expanded a comment that trailed off
@alecgrieser alecgrieser merged commit 97b61cb into FoundationDB:main Sep 30, 2025
7 of 8 checks passed
@alecgrieser alecgrieser deleted the refactor-match-candidate-creation branch September 30, 2025 17:35
arnaud-lacurie pushed a commit that referenced this pull request Oct 2, 2025
…#3653)

This updates the minor version to 4.7 to account for the API changes
introduced by #3640. Users who override the index maintainer registry in
some way will need to respond to those changes. This updates the minor
version to 4.7 to signal that to our consumers.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants