Implement edge type indices #1542
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Currently there are two different kinds of indices in the database indices for label and indiced for label+property based lookups on vertices. With this feature it should be possible to create and deal with indices on edge types in the same way as with indices on vertices. The already existing implementations of indices are good as a start for indices for edge-types, however they are not mappable one to one. Issue with edge-type based indices is that it is not always clearly defines what en edge is. If the flag --storage-properties-enabled-on-edges is False, then there is no singular datastrucrture that is the collection of all the edges present in the database. The only way to get the edges is to gather all the information runtime using the source and destination vertices of the given edge, and compute everything we need to be able to construct the EdgeAccessor objects. This might be computationally to expensive, especially in the presence of supernodes. If the datastructure that is the collection of all the edges is enabled by setting the --storage-properties-enabled-on-edges to True, it is still not trivial to get all the data that is needed to create the EdgeAccessor objects. Solution for these issues should come in subsequent commits.
The indexing datastructures that are holding the relevant information about the edge-type indices are in place. This commit adds the functionality to be able to CREATE and DROP edge-type indices.
|
Kudos, SonarCloud Quality Gate passed!
|
Implement the planned side of edge-type indexing by creating a new operator that represents the edge-type indexing operation itself and update the rewriter logic to make it understand when the new operator should replace the scanall + expand operations.
Previously abstract base classes were used to implement the on-disk related stubs as we do not plan on implementing this functionality for now for the on-disk storage mode. With this commit the relevant virtual functions have been made pure virtual and the stuv implementations have been moved to dedicated subclasses.
The edge-type index creation itself is scanning through the skiplist and building up the indices based on that, however it does not take into account those edges that are being created after the index itself is created. With this commit the index is updated upon adding edges to the database if needed.
The complexity brought by storage::View::NEW and OLD are making some edgecases hard to deal with. Add neccessary metadata, needed by the edge indexing process to work correctly in these edge-cases. Currently that is an additional pointer to the Edge constructs in the Edges skiplist. Add more comprehensive tests for the indexing and planning.
This may be a leaksanitizer fluke. If the loop is rewrtitten this way, the sanitizer stops reporting leaked memory. Pushing it now to see if that is the case on CI as well. Further investigation is needed for this case.
|
Can you link #662 to development if this PR is fixing that issue? |
|
@katarinasupe |
|
Okay, thanks, good to know! |
|
@gvolfing, I realized that the linked issue automatically closes when the pull request merges, so it's enough that we mentioned it in the comment since it's visible on both PR and issue. |
Darych
reviewed
Feb 14, 2024
antoniofilipovic
requested changes
Feb 16, 2024
antoniofilipovic
requested changes
Feb 16, 2024
Darych
reviewed
Feb 18, 2024
Darych
reviewed
Feb 18, 2024
antoniofilipovic
approved these changes
Feb 23, 2024
10 tasks
Darych
reviewed
Feb 27, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Currently there are two different kinds of indices in the database indices for label and indices for label+property based look-ups on
vertices. With the edge-type indexing capabilities it should be possible to create and deal with indices on edge types in the same way as with indices on vertex labels and properties. The case for edge-type indexes is, that in the case if the user would like to query all the edges based on a specific edge-type, the created query-plan has to traverse the entire graph, looking for the specific edge-type. The edge-type indexing feature solves this issue, however the flag,
--storage-properties-on-edgeshas to be enabled.