Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Change read mode to immediately stop consuming lines when shutting down #322

Merged
merged 2 commits into from Dec 13, 2023
Merged

Change read mode to immediately stop consuming lines when shutting down #322

merged 2 commits into from Dec 13, 2023

Conversation

edmocosta
Copy link
Contributor

@edmocosta edmocosta commented Dec 5, 2023 •
edited

This PR changes the read mode to immediately stop consuming buffered lines when the pipeline shutdown is requested.

Closes: #323

@edmocosta edmocosta changed the title [WIP][Test] Change read mode to immediately stop consuming lines when shutting down [WIP] Change read mode to immediately stop consuming lines when shutting down Dec 5, 2023
@edmocosta edmocosta changed the title [WIP] Change read mode to immediately stop consuming lines when shutting down Change read mode to immediately stop consuming lines when shutting down Dec 7, 2023
@edmocosta edmocosta marked this pull request as ready for review December 7, 2023 09:11
@edmocosta edmocosta requested a review from jsvd December 12, 2023 09:56
jsvd
jsvd approved these changes Dec 13, 2023
View reviewed changes
Copy link
Contributor

@jsvd jsvd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.
I've confirmed that this change allows Logstash to terminate much sooner, on each line instead on each loop (which can be hundreds of lines).
I've also tested with multiple reads and multiple shutdowns/restarts that no data was lost or duplicated.

@edmocosta edmocosta merged commit 70aadf2 into logstash-plugins:main Dec 13, 2023
2 checks passed
@edmocosta edmocosta deleted the change-read-mode-shutdown branch December 13, 2023 13:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jsvd jsvd jsvd approved these changes

Assignees

@edmocosta edmocosta

Labels
int-shortlist
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Read mode is not immediately stop consuming lines when shutting down is requested
3 participants
@edmocosta @jsvd @roaksoax