Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Doc]Add tips for troubleshooting a pipeline #11545

Closed
wants to merge 4 commits into from
Closed

[Doc]Add tips for troubleshooting a pipeline #11545

Changes from 2 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
f6deb1c
Add tips for troubleshooting a pipeline
karenzone Jan 27, 2020
fc62390
Minor, added suggestion to switch on the log per pipeline
andsel Jan 28, 2020
7306e1f
Removed errored CLI flag description
andsel Jan 29, 2020
9c3ff17
Update troubleshooting.asciidoc
karenzone Feb 12, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
31 changes: 29 additions & 2 deletions docs/static/troubleshooting.asciidoc
View file
Open in desktop
Expand Up @@ -8,8 +8,6 @@ have something to add, please:
https://github.com/elastic/logstash/issues, or
* create a pull request with your proposed changes at https://github.com/elastic/logstash.

// After merge, update PR link to link directly to this topic in GH

Also check out the https://discuss.elastic.co/c/logstash[Logstash discussion
forum].

Expand Down Expand Up @@ -139,9 +137,38 @@ For general performance tuning tips and guidelines, see <<performance-tuning>>.



[float]
[[ts-pipeline]]
== Troubleshooting a pipeline

Pipelines, by definition, are unique. Here are some guidelines to help you get
started.

* Identify the offending pipeline.
* Start small. Create a minimum pipeline that manifests the problem.


For basic pipelines, this configuration could be enough to make the problem show itself.

[source,ruby]
-----
input {stdin{}} output {stdout{}}
-----

To better identify the offending pipeline, {ls} could separate its logs by pipeline.
To switch on the log per pipeline feature is sufficient to enable `pipeline.separate_logs` in your `logstash.yml`
or passing the `-Dls.pipeline.separate_logs=true` on command line.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure this is quite right, or if we even can change this setting from the command line.

Although we use the Java System Property ls.pipeline.separate_logs in the log4j templates (and one would ordinarily set Java System Properties with -Dproperty=value), it looks like on startup we stomp on the existing value of that property without first reading it, stetting to to be the value held in pipeline.separate_logs from logstash.yml:

    java.lang.System.setProperty("ls.pipeline.separate_logs", setting("pipeline.separate_logs").to_s)

-- logstash-core/lib/logstash/runner.rb:257@2925b491

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@yaauie you are right, many thanks to point it out. Removed and rephrased a little bit


For more complex pipelines, the problem could be caused by a series of plugins in
a specific order. Troubleshooting these pipelines usually requires trial and error.
Start by systematically removing input and output plugins until you're left with
the minimum set that manifest the issue.

We want to expand this section to make it more helpful. If you have
troubleshooting tips to share, please:

* create an issue at https://github.com/elastic/logstash/issues, or
* create a pull request with your proposed changes at https://github.com/elastic/logstash.

[float]
[[ts-kafka]]
Expand Down