-
Notifications
You must be signed in to change notification settings - Fork 431
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Concurrency issue at analyze #3907
Conversation
d26335e
to
311ae37
Compare
- produce close the engineInput instead of close producerFinished, in this way the engine knows when the events finishes - add a close in the function consumeSources, in this way the consumer knows when Finding finishes
311ae37
to
5b5426e
Compare
Going to review this now. 👍🏼 |
The issues are reproducible indeed. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM.
Tested with a huge json (100000000 entries / 109G) as input and analyze processed all events accordingly.
Signalling Tracee to finish during the analyze process worked smoothly as well.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Small thing to add on the same note.
Overall LGTM
50d48d2
to
e879b11
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Many thanks @rscampos for your help on this. |
1. Explain what the PR does
fix: #3903
fix: #3905
Instead of terminating the program upon reaching the EOF (after all events have been read), ensure the program concludes only after all events have been fully processed.
2. Explain how to test it
3. Other comments