Skip to content

Conversation

andrewvc
Copy link
Contributor

@andrewvc andrewvc commented Jul 5, 2017

Fixes elastic/logstash#7588 (comment) by correctly ordering calls to manticore.

Also removes a dead function that used to be the thing that sent requests.

andrewvc added 3 commits July 5, 2017 13:34
This plugin could sometimes deadlock when the downstream server responded quickly.
This patch fixes this by making sure the callbacks are declared before the request is sent.

Fixes elastic/logstash#7588 (comment)
Copy link
Member

@jsvd jsvd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

amazing job hunting this bug. I think it's worth documenting a bit more the rationale for the changes.

CHANGELOG.md Outdated
@@ -1,3 +1,6 @@
## 4.3.1
- Fix deadlock that could occur in certain odd situations. If you noticed this plugin stalling this should fix it.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we describe this a bit more or just say it fixes a deadlock? it seems the deadlock would occur pretty much every time in any situation?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jsvd well, I actually couldn't repo it against a slow endpoint. I think from a user's perspective this is in the category of must upgrade. I think it makes sense to modify the message to say "All users should upgrade, even if they aren't experiencing problems".

I don't know if the changelog is the place to go deep into code however.

end
end

request.call # Actually invoke the request in the background
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is it worth describing a bit more why this fixes the issue? I'm ok with putting that explanation in the commit, but here would prevent future errors. if someone decides to "refactor" the order of these things

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jsvd Sure, I'm glad to put that message in here

@andrewvc
Copy link
Contributor Author

andrewvc commented Jul 5, 2017

Testing this against nginx, which would see this request every ~ 50 requests, I now find that things work:

bin/logstash -w 1 --log.level=info -b 5 -e "input { generator { count => 100000} } output { http { url => 'http://localhost:8080' http_method => post ignorable_codes => [405] } stdout { codec => dots }  }"

wc -l  /usr/local/var/log/nginx/access.log
  100001 /usr/local/var/log/nginx/access.log

The extra 1 line is accounted for by some whitespace I made when clearing the log file with echo "" > access.log.

Copy link
Member

@jsvd jsvd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, please squash, merge and publish. Great job hunting this down, @andrewvc and @original-brownbear <3

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants