Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow per-pipeline config of ECS Compatibility mode via Central Management #12861

Merged
merged 3 commits into from
Jun 17, 2021

Conversation

yaauie
Copy link
Member

@yaauie yaauie commented Apr 29, 2021

Release notes

When using Kibana's Central Management for Logstash, values for pipeline.ordered and pipeline.ecs_compatibility settings are now supported.

What does this PR do?

Brings pipeline.ordered and pipeline.ecs_compatibility settings into the supported set here on the Logstash side so that they will be respected when provided by a pipeline that is managed by Kibana's Central Management for Logstash. Currently, the Kibana UI does not support setting these in the pipeline editor, but allows unvalidated k/v settings through pipelines managed through its API.

This PR does NOT include the necessary changes to Kibana's pipeline editor to make these UI-accessible (hint: here?) or Kibana's API documentation (hint: here).

Why is it important/What is the impact to the user?

Allows a user to configure the ECS Compatibility or Strict Ordering settings of a pipeline when that pipeline is managed by Kibana's Central Management for Logstash.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files (and/or docker env variables)
  • I have added tests that prove my fix is effective or that my feature works

Author's Checklist

How to test this PR locally

Using Kibana's API, persist a pipeline with the settings pipeline.ordered and/or pipeline.ecs_compatibility. Observe that without this PR they are ignored, but with this PR they are respected.

@@ -281,6 +281,10 @@ def to_hash
}
end

def inspect
Copy link
Member Author

@yaauie yaauie Apr 29, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Context: this makes failing expectations involving the Setting object a lot easier to understand.

before:

expected `#<LogStash::Setting::PositiveInteger:0x1e48ac75 @name="pipeline.workers", @value=nil, @value_is_set=f...tor_proc=#<Proc:0x624fe3ee@/Users/yaauie/src/elastic/ls/logstash-core/lib/logstash/settings.rb:398>>.set?` to be truthy, got false

after:

expected `<LogStash::Setting::PositiveInteger(pipeline.workers): 16 (DEFAULT)>.set?` to be truthy, got false

Copy link
Member

@robbavey robbavey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@yaauie yaauie force-pushed the ecs-compatibility-vs-central-management branch from 387da82 to a59f141 Compare June 17, 2021 20:01
@yaauie
Copy link
Member Author

yaauie commented Jun 17, 2021

Clean rebase, re-runinng jenkins before merging.

@yaauie yaauie merged commit 49e6b0e into elastic:master Jun 17, 2021
yaauie added a commit that referenced this pull request Jun 17, 2021
…ement (#12861)

* spec: noop refactor of xpack central management

* spec: validate central management settings loading

* central management: allow pipeline.ordered and pipeline.ecs_compatibility settings

(cherry picked from commit 49e6b0e)
@yaauie yaauie added the v7.14.0 label Jun 17, 2021
kares added a commit to kares/logstash that referenced this pull request Jul 1, 2021
* master: (41 commits)
  Test: resolve integration failure due ECS mode (elastic#13044)
  Feat: event factory support (elastic#13017)
  Doc: Add geoip database API to node stats (elastic#13019)
  Add geoip database metrics to /node/stats API (elastic#13004)
  ecs: on-by-default plus docs (elastic#12830)
  ispec: fix cross-spec leak from fatal error integration specs (elastic#13002)
  Fix UBI source URL (elastic#13008)
  update fpm to allow pkg creation on jdk11+jruby 9.2 (elastic#13005)
  Add unit test to grant that production aliases correspond to a published RubyGem (elastic#12993)
  Fix logstash.bat not setting exit code (elastic#12948)
  Use the OS separator to invoke gradlew from Rake script (elastic#13000)
  Allow per-pipeline config of ECS Compatibility mode via Central Management (elastic#12861)
  Update jinja2 dependency in docker build (elastic#12994)
  fix database manager with multiple pipelines (elastic#12862)
  Fix Reflections stack traces when process yml files in classpath and debug is enabled (elastic#12991)
  Fix/log4j routing to avoid create spurious file (elastic#12965)
  Deps: update JRuby to 9.2.19.0 (elastic#12989)
  Doc: Add tip for checking for existing field (elastic#12899)
  Added test to cover the installation of aliased plugins (elastic#12967)
  CI: Update logstash_release.json after 7.3.12 (elastic#12986)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants