Enable independent Elastic Agent if root privileges is set to true #1815

mrodm · 2024-05-03T08:34:09Z

Relates #787
Closes #1586

This PR allows to spin up new Elastic agents independents from the Elastic Agent run in the stack (from elastic-package stack up command) for testing, if the packages defined in their manifest agent.privileges.root key as true.

That action can be overwritten if it is needed by setting the ELASTIC_PACKAGE_TEST_ENABLE_INDEPENDENT_AGENT environment variable. This variable has preference over the manifest file.

NOTE: if packages contain a folder to deploy a previous "custom agent" (_dev/deploy/agent), elastic-package will not create an independent Elastic Agent with User root. In that scenario, elastic-package will still create the previous custom agents.

…gent.privileges.root is true

scripts/test-check-packages.sh

mrodm · 2024-05-03T10:06:06Z

test/packages/with-custom-agent/auditd_manager/manifest.yml

@@ -1,15 +1,17 @@
-format_version: 1.0.0
+format_version: 2.12.0


Update manifest to add agent.privileges.root key. This package keeps the custom agent (from servicedeployer).

Ok, looking at this, we can probably remove the custom agent, and the auditd_manager_independent_agent test package.

I was wondering if we should keep that package to test the scenario of the custom agents. Although, oracle package is still there in this test folder.

If we remove the custom-agent folder of auditd_manager, we should add the settings into the system configuration file at least to add the capabilities required for the process.

Ok, I guess we can do that. But as you prefer, having both packages also makes sense.

jsoriano

Nice!

jsoriano · 2024-05-03T10:14:31Z

cmd/testrunner.go

 		}

-		runIndependentElasticAgent := false
+		// Enable independent agents if package defines that requires root privileges


Maybe we can mention here that this is temporary till independent agents are enabled by default.

jsoriano · 2024-05-03T10:14:57Z

test/packages/with-custom-agent/auditd_manager/manifest.yml

@@ -1,15 +1,17 @@
-format_version: 1.0.0
+format_version: 2.12.0


Ok, looking at this, we can probably remove the custom agent, and the auditd_manager_independent_agent test package.

mrodm · 2024-05-03T11:54:35Z

It needs to take into account that if any package contains a folder to deploy a previous "custom agent" (_dev/deploy/agent), elastic-package will not create an independent Elastic Agent with User root. In that scenario, elastic-package will still create the previous custom agents. cc @jsoriano

Updated the description also with this note.

jsoriano

🎉

jsoriano · 2024-05-03T11:57:08Z

scripts/test-check-packages.sh

    elif [ "${PACKAGE_TEST_TYPE:-other}" == "with-logstash" ] && [ "${package_to_test}" == "system_benchmark" ]; then
        elastic-package benchmark system --benchmark logs-benchmark -v --defer-cleanup 1s
    else
-      if [[ "${ELASTIC_PACKAGE_TEST_ENABLE_INDEPENDENT_AGENT}" == "false" && "${package_to_test}" == "auditd_manager_independent_agent" ]]; then


Umm, this auditd_manager_independent_agent package could be moved out of the with-custom-agent directory, right? And then maybe renamed back to auditd_manager.

Ok, moving this package out of this folder 👍

jsoriano · 2024-05-03T12:01:35Z

It needs to take into account that if any package contains a folder to deploy a previous "custom agent" (_dev/deploy/agent), elastic-package will not create an independent Elastic Agent with User root. In that scenario, elastic-package will still create the previous custom agents. cc @jsoriano

The only custom agent that requires root is the one for auditd_manager, and it correctly defines the user and capabilities it requires, this will continue to be honored, right?

mrodm · 2024-05-03T12:13:17Z

The only custom agent that requires root is the one for auditd_manager, and it correctly defines the user and capabilities it requires, this will continue to be honored, right?

Yes, that will be honored. elastic-package will create a custom agent (same as before) for that package with the capabilities and user defined in that YAML file.

elasticmachine · 2024-05-03T12:56:50Z

💚 Build Succeeded

Buildkite Build
Commit: e08ad61

History

💚 Build #3018 succeeded ba2fe78
💔 Build #3016 failed fd243fd
💔 Build #3015 failed 515b65a
💚 Build #3014 succeeded b289532

cc @mrodm

mrodm · 2024-05-03T13:14:00Z

test integrations

elasticmachine · 2024-05-03T13:19:49Z

Created or updated PR in integrations repository to test this version. Check elastic/integrations#9783

mrodm · 2024-05-03T13:45:28Z

The only custom agent that requires root is the one for auditd_manager, and it correctly defines the user and capabilities it requires, this will continue to be honored, right?

Yes, that will be honored. elastic-package will create a custom agent (same as before) for that package with the capabilities and user defined in that YAML file.

Just tested with this PR elastic/integrations#9783
Among the packages that define agent.privileges.root as true:

the following packages create independent Elastic Agent containers:
- network_traffic
- system_auditd
- fim
Package auditd_manager keeps creating the previous custom Elastic Agent container (from servicedeployer).
Package universal_profiling_agent does not have any system tests.

mrodm added 2 commits May 3, 2024 10:30

Enable independent agents if agent.privileges.root is true

4498438

Update trap function name

b289532

mrodm self-assigned this May 3, 2024

mrodm mentioned this pull request May 3, 2024

Add ability to deploy custom elastic-agents on different OS or runtimes #787

Open

mrodm changed the title ~~Enable independent agent if root privileges is set to true~~ Enable independent Elastic Agent if root privileges is set to true May 3, 2024

mrodm added 2 commits May 3, 2024 12:02

Remove exception - elastic-package should use independent agents as a…

4b14073

…gent.privileges.root is true

Update auditd_manager manifest

515b65a

mrodm commented May 3, 2024

View reviewed changes

scripts/test-check-packages.sh Show resolved Hide resolved

mrodm commented May 3, 2024

View reviewed changes

jsoriano reviewed May 3, 2024

View reviewed changes

mrodm added 4 commits May 3, 2024 12:39

Just set env. variable to enable independent agents if true

2dfbedd

Update comment

fd243fd

Remove default initialization of env. var

ba2fe78

Remove global default initialization in script

74343ce

mrodm marked this pull request as ready for review May 3, 2024 11:46

mrodm requested a review from jsoriano May 3, 2024 11:46

jsoriano approved these changes May 3, 2024

View reviewed changes

Moving package to parallel folder

e08ad61

elasticmachine mentioned this pull request May 3, 2024

Test elastic-package#1815 - DO NOT MERGE elastic/integrations#9783

Closed

jsoriano approved these changes May 3, 2024

View reviewed changes

mrodm mentioned this pull request May 3, 2024

[System tests] Run system tests for integrations with a non root Elastic Agent #1586

Closed

mrodm merged commit 48a4da9 into elastic:main May 3, 2024

mrodm deleted the enable-independent-agent-root-privileges branch May 3, 2024 14:42

		@@ -1,15 +1,17 @@
		format_version: 1.0.0
		format_version: 2.12.0

Enable independent Elastic Agent if root privileges is set to true #1815

Enable independent Elastic Agent if root privileges is set to true #1815

Uh oh!

Conversation

mrodm commented May 3, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jsoriano left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mrodm commented May 3, 2024

Uh oh!

jsoriano left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jsoriano commented May 3, 2024

Uh oh!

mrodm commented May 3, 2024

Uh oh!

elasticmachine commented May 3, 2024

💚 Build Succeeded

History

Uh oh!

mrodm commented May 3, 2024

Uh oh!

elasticmachine commented May 3, 2024

Uh oh!

mrodm commented May 3, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

mrodm commented May 3, 2024 •

edited

Loading