Track search work by container; purge on delete#6872
Track search work by container; purge on delete#6872labkey-jeckels wants to merge 9 commits intorelease25.7-SNAPSHOTfrom
Conversation
| public boolean drainQueue(PRIORITY priority, long timeout, TimeUnit unit) throws InterruptedException | ||
| public void purgeForContainer(Container container) | ||
| { | ||
| _runQueue.removeIf(i -> container.getEntityId().equals(i._containerId)); |
There was a problem hiding this comment.
Shouldn't the task owning this item be notified? E.g. with TaskListener.indexError() or new method TaskListener.cancel(). Will the task ever know it is done?
There was a problem hiding this comment.
Good point. Added in my most recent commit.
| /** Lower priority indexing work. Intended for work placed in the queue as part of a call to DocumentProvider.enumerateDocuments() */ | ||
| crawl, | ||
| /** Intended for content that needed indexing or reindexing because it was modified or created */ | ||
| modified, |
There was a problem hiding this comment.
I'm not convinced just having Runnables < Item is sufficient? Runnables can create Runnables, and you really want those to be run at higher pri. I suppose that there is lower risk of this exploding the queue (compared to the Item explosion), but it still does sort of lead to doing a "breadth-first" expansion of the tasks instead of "depth-first".
There was a problem hiding this comment.
Maybe that could be fixed by having "child" runnables be added to the front of the queue, effectively taking the place of the "parent" (expand in place).
There was a problem hiding this comment.
I can revert the change if it's too simplistic (I intentionally did that work as a separate commit). My overall takeaway from this work is that we were wildly inconsistent with prioritization and that other people (including myself) didn't understand how the priorities were supposed to be used.
The one place that I found where Runnables beget Runnables is ExperimentServiceImpl.indexMaterials() and its data counterpart. There, a bulk Runnable creates item Resources and another bulk Runnable. I think that's consistent with the flattened set of priorities that sort Resources first. Have I gotten myself confused?
|
Replaced by #6897 |
1 similar comment
|
Replaced by #6897 |
2 participants
Rationale
Instead of waiting for the indexer to drain its queue before deleting a container, we can purge that work from the queue.
Changes
Tasks 📍