[KDB-788] Add secondary indexing plugin and support virtual readers from plugins #5026
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Qodana for .NETIt seems all right 👌 No new problems were found according to the checks applied 💡 Qodana analysis was run in the pull request mode: only the changed files were checked Contact Qodana teamContact us at qodana-support@jetbrains.com
|
oskardudycz
force-pushed
the
indexing/duck_db_plugin
branch
2 times, most recently
from
fc4e437 to
7432e12
Compare
oskardudycz
force-pushed
the
indexing/duck_db_plugin
branch
3 times, most recently
from
ca6793b to
61015a5
Compare
oskardudycz
force-pushed
the
indexing/duck_db_plugin
branch
from
61015a5 to
9018451
Compare
oskardudycz
force-pushed
the
indexing/duck_db_plugin
branch
2 times, most recently
from
331b8ec to
086a737
Compare
oskardudycz
force-pushed
the
indexing/duck_db_plugin
branch
from
086a737 to
cf32288
Compare
timothycoleman
previously approved these changes
May 7, 2025
alexeyzimarev
previously approved these changes
May 7, 2025
timothycoleman
previously approved these changes
May 7, 2025
oskardudycz
dismissed stale reviews from timothycoleman and alexeyzimarev
via
a0a8aae
oskardudycz
force-pushed
the
indexing/duck_db_plugin
branch
from
2229f13 to
a0a8aae
Compare
…rify pluggability
…tup to allow passing VirtualStreamReaders
…ader to test project
…ugin loader in ClusterVNodeHostedService
…erVNodeHostedService Changed also plugin registration in MiniNode to be passed as subsystem to ensure that config is called.
timothycoleman
force-pushed
the
indexing/duck_db_plugin
branch
from
a0a8aae to
05f23ef
Compare
timothycoleman
approved these changes
May 8, 2025
oskardudycz
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We want the indexing functionality isolated in a plugin so it can be easily feature-flagged and potentially licensed.
Added a basic structure for the Second Level Indexing plugin and registered it. For now, it's only enabled by default in the dev mode (it can be enabled for testing by using
KurrentDB:SecondLevelIndexing:Enabled). It's to be discussed whether and when it'll be enabled by default, but definitely it won't be enabled by default sooner than when it's ready to be released.We added a dummy indexing reader to enable testing whether registration and setup work correctly. It will be removed when a real DuckDB one is provided.
The list of virtual stream readers is now populated in the ClusterVNode constructor. However, we need some readers provided by the plugin, so we allowed providing virtual stream readers through ClusterVNode.Create and let the list be combined with default virtual streams (node state and gossip) inside the ClusterVNode constructor.