This repository has been archived by the owner on Aug 23, 2023. It is now read-only.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Related to the issue #1539
This splits the main index lock up by introducing 3 new RW locks.
metaTagIdx
which manages the meta tag data structures on a per org basis. This write lock only needs to be acquired when a new org starts using the meta tag index, so very rarely.metaTagIdx
also implements all the exported index methods related to the meta tag index, it is embedded into the main index.metaTagIndex
has been renamed tometaTagHierarchy
to prevent confusion with the main meta tag index struct, and it has it's own RW lock now. It also has a few new accessor methods, because previously some of it's properties were directly accessed by other methods of the index package, but now all interactions go through the new accessors to make sure the lock is used correctly.metaTagRecords
also has its own RW lock now, and a few other methods which previously accessed its properties directly are now using its accessor methods to ensure correct locking.The method
idsByTagQuery
had to be refactored because when we add/delete meta records from the enricher we need to be able to execute a query on the tag index and pass the result into the enricher without releasing the read lock on the main index in-between, otherwise there would be a race condition. So the enricher now has a callback using which it can callidsByTagQuery
on the main index. When executing such queries we now instantiate the result chan into which the query results will get pushed, pass it intoidsByTagQuery
which acquires the read lock on the main index and starts the query execution in a new routine, then we push the result chan into the enricher's event queue, and only release the read lock on the main index once the query has completed. The enricher's event handler then only needs to consume that result chan. That way we can still ensure consistency in the enricher.