Resolve 'collection modified' exception when logging results #2549
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
michaelcfanning
requested review from
EasyRhinoMSFT,
marmegh and
cfaucon
as code owners
michaelcfanning
commented
Oct 6, 2022
| SingleWriter = true, | ||
| SingleReader = true, | ||
| }; | ||
| readyToHashChannel = Channel.CreateBounded<int>(channelOptions); |
marmegh
reviewed
Oct 6, 2022
| /// Gets or sets a boolean value that indicates whether the Results | ||
| /// object associated with this logger is fixed and ready to replay. | ||
| /// </summary> | ||
| public bool CacheFinalized { get; private set; } |
marmegh
reviewed
Oct 6, 2022
| SingleWriter = true, | ||
| SingleReader = true, | ||
| }; | ||
| readyToHashChannel = Channel.CreateBounded<int>(channelOptions); |
There was a problem hiding this comment.
Should we have a line break here and six lines down to improve readability?
marmegh
reviewed
Oct 6, 2022
| findFiles.Wait(); | ||
| hashFiles.Wait(); | ||
| // 4: A single-threaded consumer watches for completed scans | ||
| // and logs results, if any. This operation is singlle-threaded |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This change introduces thread-safe utilization of a single CachingLogger instance in cases when multiple files with the same file hash are analyzed.
In these circumstances, the scan should not occur again, rather, the cached results (if any) should be updated (to reflect the current file path of the duplicated file) and 'replayed' into the log.
This is tricky because the scan pipeline design is intended for every context object (per scan target) to be accessed only on a single thread.
To make this work, we introduce a slim semaphore in the caching logger, which allows access to only a single thread at a time. This semaphore's
Waitmethod is invoked on receiving the notice that a new scan target is being analyzed.We also have a new ReleaseLock method, which is called after scan completes. This method releases the semaphore and also puts the caching logger into a 'committed' state, meaning that the first copy, by file hash, of a set of duplicated targets has been analyzed and we now have a fixed set of results that can be 'replayed' into the log.
@suvamM