Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

securityonion-elastic: improve support for custom ingest parsers #1671

Closed
dougburks opened this issue Dec 4, 2019 · 2 comments
Closed

securityonion-elastic: improve support for custom ingest parsers #1671

dougburks opened this issue Dec 4, 2019 · 2 comments
Assignees
Projects

Comments

@dougburks
Copy link
Contributor

@dougburks dougburks commented Dec 4, 2019

From Slack discussion with Jim Hranicky:

there at least 2 ways to add custom Elasticsearch ingest pipelines today:
(1) Simply add your new pipelines to the existing directory /etc/elasticsearch/ingest/. The caveat here is that if you change our existing parsers your changes will be overwritten on next upgrade.
OR
(2) Copy the existing directory /etc/elasticsearch/ingest to a new directory (for example /etc/elasticsearch/ingest-custom/) and then add a new setting to /etc/nsm/securityonion.conf:
ELASTICSEARCH_INGEST_PIPELINES="/etc/elasticsearch/ingest-custom"
The caveat here is that your new directory wouldn't automatically get any improvements that we make to the upstream parsers.
In either case, you would then need to run sudo so-elasticsearch-pipelines for the changes to take effect.

Jim's response:

So I created /etc/elasticsearch/ingest-custom , symlinked the files in /etc/elasticsearch/ingest there, added my pipeline, and set the variable in /etc/nsm/securityonion.conf . That seems to work. For custom pipelines there does still need to be a way to add an output to /etc/logstash/conf.d.minimal, I'm doing that by hand for now. FYI.
Also added /etc/elasticsearch/ingest-custom to salt

@dougburks dougburks self-assigned this Dec 4, 2019
@dougburks dougburks added this to To do in 16.04.6.4 via automation Dec 4, 2019
dougburks added a commit to Security-Onion-Solutions/securityonion-elastic that referenced this issue Dec 27, 2019
@dougburks dougburks moved this from To do to In progress in 16.04.6.4 Dec 27, 2019
@dougburks dougburks moved this from In progress to In Testing in 16.04.6.4 Jan 4, 2020
@defensivedepth

This comment has been minimized.

Copy link
Collaborator

@defensivedepth defensivedepth commented Jan 14, 2020

Looks good to me

@defensivedepth defensivedepth moved this from In Testing to Tested in 16.04.6.4 Jan 14, 2020
@dougburks

This comment has been minimized.

@dougburks dougburks closed this Feb 5, 2020
16.04.6.4 automation moved this from Tested to Done Feb 5, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
16.04.6.4
  
Done
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
You can’t perform that action at this time.