Update ElasticSearchDao to support LogStash #22
Conversation
|
A colleague of mine actually just implemented this exact same thing for the exact same reason; in a local SNAPSHOT build. Good timing +1 |
|
What is the benefit of sending to LogStash? Does LogStash do something specific? |
|
LogStash is Elastic's configurable data-processing and queuing software. Having LogStash feeding logs into Elasticsearch is a very common pattern, it adds another layer of protection in case of failure or overload and it can preprocess data before it is stored. LogStash presents the same interface as Elasticsearch, but it returns 200 instead of 201, hence this change. |
|
@rebnridgway I see. Thank you very much for the explanation. Our volume is very low. I guess we will continue to hit ES directly. |
|
@jesusaurus @R-Gerard Is this project dead? I see last PR was integrated almost a year ago.. We'd prefer to pull a new version from Jenkins update center, instead of building a fork locally.. |
|
@jakub-bochenski I marked this plugin with the |
|
@R-Gerard I can see it now, please excuse my banner-blindness :) |
|
@oleg-nenashev I'm new to jenkins plugin maintenance: is there anything preventing us from merging this? |
|
@jakub-bochenski No special organization-wide process. Plugin maintainers are eligible to define the process they prefer. |
This allows ElasticSearchDao to push data to a LogStash instance set up with the http input plugin instead of an ElasticSearch instance. They expose a similiar API.
|
@oleg-nenashev I've changed the code to simply accept any success (2xx) code, this should be more robust. |
|
Can we get this merged? |
|
Has anyone besides the committer tested a snapshot version of 9ce0c79 ? The change happened after the code approval. VJM, I'm curious too, do we need a (foster/permanent) maintainer to get this merged or are there special cases when CloudBees merges unofficial plugin PRs due to popular demand :) |
|
@diginc Jenkins project is independent from CloudBees. There is no official/unofficial PRs as well there is no official/unofficial plugins in the community. Sometimes Jenkins "core team" goes forward and releases changes in non-maintained without taking ownership , but it happens only in the case of severe regressions and 0-day security issues. For all other changes there is a "adopt a plugin" community process: https://wiki.jenkins-ci.org/display/JENKINS/Adopt+a+Plugin |
|
@diginc I was planning to merge this soon as we have been using this version for a few months now. I'm not sure what exactly is your concern? Do you see some issues with the recent update -- it's functionally the same as far as 200 and 201 codes are concerned, the only difference is for status codes from 202-299 range. |
|
On the surface the latest refactored code looks good but I haven't actually tested so I was just asking who had tested/ran the refactored commit in production |
|
@jakub-bochenski @hkbao when do we expect to see this new plugin version out here: https://wiki.jenkins-ci.org/display/JENKINS/Logstash+Plugin ? |
|
@vjm I was intending to try and do a release by the end of this month, depending on the status of other PRs currently active. In the meantime I'd like to encourage you to try out the master build and test if it works for you before the release. |
|
@jakub-bochenski we will try it out and let you know! thanks! |
|
@jakub-bochenski I am not seeing the option from the dropdown after installing the master build. I am able to see version 1.2.1 in the plugins page. Any suggestions? |
|
@vjm there are no new options introduced here, this just changes the message-posting logic to accept any 2xx codes instead of only 200 (you should be able to use ES backend to post to Logstash now) |
|
Ok, trying that but not getting any feedback in the UI. @jakub-bochenski I am not seeing anything come through in logstash. Is there a way to validate that this output is actually getting sent somewhere out of Jenkins? |
|
@vjm I'm using master build now to push to logstash, this is the config I'm using The plugin is throwing exceptions in case of errors and those should be visible in Jenkins log |
|
Is this wrapper compatible with Jenkins file yet?
…On Tue, Jun 27, 2017 at 4:53 AM jakub-bochenski ***@***.***> wrote:
@vjm <https://github.com/vjm> I'm using 6d3a8f7b now to push to logstash,
this is the config I'm using
[image: screenshot_20170627_114909]
<http://url>
Remember you need to add either a wrapper or publisher to the job you are
interested in.
The plugin is throwing exceptions in case of errors and those should be
visible in Jenkins log
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#22 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AA0IRvqF7Jd-nlB0Js61AhqbJiJCDQM5ks5sINEagaJpZM4Lm-xC>
.
|
|
@jakub-bochenski any idea when that will be in the master build I can test? |
|
I will grab this today
…On Tue, Jun 27, 2017 at 8:25 AM jakub-bochenski ***@***.***> wrote:
@vjm <https://github.com/vjm> I've rebased the PR from master, you can
grab the .hpi file from CI build, but be quick as the retention is only a
few days
Please let me know about the results, I'm also testing this on our staging
Jenkins instance, but the more eyes the better
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#22 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AA0IRsMldyuZic_r2tEqR-vQyds5MCi9ks5sIQKvgaJpZM4Lm-xC>
.
|
|
any suggestions? i can properly send data to this endpoint in other methods. |
|
@jakub-bochenski I opened #29 to help address these issues by actually creating a separate Logstash provider DAO. |
|
I'm soon going on vacation for next two weeks and I won't be able to push a new release with this changes -- sorry about that, but some other last minute tasks got in the way. I will try to do a release the first thing when I come back. |
This allows ElasticSearchDao to push data to a LogStash instance set up with the http input plugin instead of an ElasticSearch instance. They expose the same API, but LogStash returns 200 while ElasticSearch returns 201.