LUCENE-9406: Add IndexWriterEventListener to track events in IndexWriter - #2342
Conversation
| MergePolicy.OneMerge nextMerge = null; | ||
|
|
||
| if (pendingMerges.size() > 0) { | ||
| // nocommit getting OneMerge instance here via mergeSource.getNextMerge() will |
There was a problem hiding this comment.
How about passing the pendingMerges to the event listener instead? (Sorry if I asked for OneMerge on the issue! MergeSpecification is better since it can hold multiple merges, allows event listener to log things like how many merges were requested during commit, etc.).
There was a problem hiding this comment.
No problem! From the context I assume you are actually meaning to pass pointInTimeMerges into event listener, since pendingMerges is of type Deque<MergePolicy.OneMerge>? I've pushed a commit to use pointInTimeMerges for now.
| } | ||
|
|
||
| // Test basic semantics of merge on commit and events recording invocation | ||
| public void testMergeOnCommitWithEventListener() throws IOException { |
There was a problem hiding this comment.
You might also edit LuceneTestCase.newIndexWriterConfig to randomly swap in a MockIndexWriterEventListener just to exercise this listener in any tests using that API, which is quite a few. It could uncover times when we accidentally break something when this listener is invoked ...
There was a problem hiding this comment.
Makes sense. Added.
…r randomly into LuceneTestCase.newIndexConfig
|
I think it's good overall but I'm wondering whether it makes sense to make that field volatile... do we want to allow changing listeners over index writer lifecycle? I think it should be a regular field and IW should just read it once (and set forever). |
Thanks for the feedback Dawid! I thought about this a bit as well when I noticed most of the other fields are volatile, but I can't decide for sure which direction to go as I don't have enough context information about the different use cases IW may need to support in the wild, so I ended up following the existing pattern here. However, I do feel that it may not hurt for application to have the freedom to switch event listener in the middle if situation or need dictates (i.e. a service application containing Lucene may need to switch event listener if it exposes an API for its client to choose how the IW event stream to be sent and stored) ? |
|
If such a situation arises folks will have the ability to write a custom proxy listener with the ability to switch, add and/or remove delegates. This keeps the freedom to do anything you said and at the same time simplifies implementation (and concurrency-issues) on IW side? |
Ah that's very true! I've pushed a commit to remove volatile then. Please let me know if it looks good. |
|
Circling back to this PR as I just realized I have forgotten about it.. :D Thanks @dweiss for the review! @mikemccand do you think this PR is ready now? Anything else you would like me to improve on? |
|
Looks good to me. I've merged this in, sorry for the delay. |
|
No worry! Thanks for the review and merge! |
|
Woops, sorry for the belated response, and thank you @zacharymorn for creating this and @dweiss for merging -- it looks great! We can now add other events to track incrementally over time ... |
|
No problem, and thanks for the review feedback as well Michael! |
3 participants
Description / Solution
Add IndexWriterEventListener interface to track events in IndexWriter
Tests
Note: precommit is currently failing due to nocommit comment
Checklist
Please review the following and check all that apply:
masterbranch../gradlew check.