Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow underlying beats module configuration overrides in Elastic Agent from Fleet integration configuration page #155339

Closed
geekpete opened this issue Apr 20, 2023 · 2 comments
Labels
Team:Fleet Team label for Observability Data Collection Fleet team Team:Integrations Team Label for Observability Integrations team

Comments

@geekpete
Copy link
Member

Describe the feature:

Currently unless a particular configuration input is surfaced in the Add Integration or specific Integration's settings page, then the configuration cannot be manually set in the resulting configuration files for particular Beats/Modules within the running Agent on the host.
Even if a configuration CAN be set, it's only set in one specific way when generated, even if there's multiple ways to set the same configuration in a resulting YAML configuration.

There's no way to provide overrides to the configuration that's generated from the available inputs or to set any other configuration that isn't handled by the available inputs in the Fleet/Integrations UI.

Describe a specific use case for the feature:

The feature is to provide a way to manually manipulate the resulting config files as an override mechanism, set from within a section in the Integration configuration page.

The purpose can be for a variety of reasons, such as workarounds or troubleshooting/testing that involve custom edits to the resulting yaml file that are not possible today with the restriction on only using values from the available configuration inputs and only in the specific resulting format that is generated even if different configuration formats might be possible for the same input data (eg single dot notation line vs nested).

Compared to Beats generally, the config files can be directly manipulated much more easily as a best practice to allow for workarounds to solve a particular requirement until a fix might be released in later stack versions, which isn't as workable when using Agent.

Such a capability might look like the way we allow overrides to the Elasticsearch output via the Advanced YAML configuration option under Fleet Management settings:
https://www.elastic.co/guide/en/fleet/8.6/fleet-agent-proxy-support.html#_set_the_proxy_for_communicating_with_elasticsearch

but will instead be specific to the config of particular Beats modules related to the particular Integration.

@botelastic botelastic bot added the needs-team Issues missing a team label label Apr 20, 2023
@geekpete geekpete added the Team:Agent Agent Team label Apr 20, 2023
@botelastic botelastic bot removed the needs-team Issues missing a team label label Apr 20, 2023
@geekpete geekpete added Team:Integrations Team Label for Observability Integrations team and removed Team:Agent Agent Team labels Apr 20, 2023
@joshdover joshdover added the Team:Fleet Team label for Observability Data Collection Fleet team label May 25, 2023
@elasticmachine
Copy link
Contributor

Pinging @elastic/fleet (Team:Fleet)

@cmacknz
Copy link
Member

cmacknz commented Jun 21, 2023

Done in #158699 for 8.9.0.

@cmacknz cmacknz closed this as completed Jun 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Team:Fleet Team label for Observability Data Collection Fleet team Team:Integrations Team Label for Observability Integrations team
Projects
None yet
Development

No branches or pull requests

4 participants