Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Doc]Add tips for troubleshooting a pipeline #11545

Closed
wants to merge 4 commits into from

Conversation

karenzone
Copy link
Contributor

@karenzone karenzone commented Jan 27, 2020

Pipelines vary wildly, and it's nearly impossible to offer prescriptive steps for troubleshooting. Perhaps we can help by offering some general guidelines, things to try, etc.

This is a basic start. I'm hoping that we can expand with more tips and guidelines now and over time.

PREVIEW: http://logstash_11545.docs-preview.app.elstc.co/guide/en/logstash/master/troubleshooting.html#ts-pipeline

@karenzone karenzone added the docs label Jan 27, 2020
@karenzone
Copy link
Contributor Author

@bczifra Thanks for the suggestion. @andsel Thanks for the tips so far.
I welcome ideas from you and from others to expand this section.


To better identify the offending pipeline, {ls} could separate its logs by pipeline.
To switch on the log per pipeline feature is sufficient to enable `pipeline.separate_logs` in your `logstash.yml`
or passing the `-Dls.pipeline.separate_logs=true` on command line.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure this is quite right, or if we even can change this setting from the command line.

Although we use the Java System Property ls.pipeline.separate_logs in the log4j templates (and one would ordinarily set Java System Properties with -Dproperty=value), it looks like on startup we stomp on the existing value of that property without first reading it, stetting to to be the value held in pipeline.separate_logs from logstash.yml:

    java.lang.System.setProperty("ls.pipeline.separate_logs", setting("pipeline.separate_logs").to_s)

-- logstash-core/lib/logstash/runner.rb:257@2925b491

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@yaauie you are right, many thanks to point it out. Removed and rephrased a little bit

@karenzone
Copy link
Contributor Author

Any further comments or additions? We can expand this question later if needed.

@karenzone
Copy link
Contributor Author

@andsel Any further comments or additions? We can expand this question later if needed.

@andsel
Copy link
Contributor

andsel commented Mar 11, 2020

LGTM

@elasticsearch-bot
Copy link

Karen Metts merged this into the following branches!

Branch Commits
6.8 8b77363, b074533, 2f2b6a7, 4dba018
7.6 1a7936d, adc527a, 7932c61, 9af6889
7.x 7549824, d057e8b, 0a88578, 8c11a95
master edc5915, 8502224, 64330ab, 3ac8aa2

elasticsearch-bot pushed a commit that referenced this pull request Mar 11, 2020
elasticsearch-bot pushed a commit that referenced this pull request Mar 11, 2020
elasticsearch-bot pushed a commit that referenced this pull request Mar 11, 2020
elasticsearch-bot pushed a commit that referenced this pull request Mar 11, 2020
elasticsearch-bot pushed a commit that referenced this pull request Mar 11, 2020
elasticsearch-bot pushed a commit that referenced this pull request Mar 11, 2020
elasticsearch-bot pushed a commit that referenced this pull request Mar 11, 2020
elasticsearch-bot pushed a commit that referenced this pull request Mar 11, 2020
elasticsearch-bot pushed a commit that referenced this pull request Mar 11, 2020
elasticsearch-bot pushed a commit that referenced this pull request Mar 11, 2020
elasticsearch-bot pushed a commit that referenced this pull request Mar 11, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants