Handle new action for switching Agent from privileged to unprivileged mode

[ycombinator]

Describe the enhancement:

Elastic Agents have the ability to run in either privileged mode, i.e. with a privileged user like root on Linux systems, or unprivileged mode, i.e. with an unprivileged user.

Moreover, Agents running in privileged mode have the ability to switch themselves to unprivileged mode. This ability is being exposed via the Agent CLI. We now wish to expose this same ability using the Fleet UI.

⚠️ Important note: Fleet UI users should only be switch Agents from privileged to unprivileged mode, not the other way around.

For this, Agent will need to handle a new action from Fleet.

Describe a specific use case for the feature:

Security: Allowing users to reduce the privileges required to run Elastic Agent.

What is the definition of done?

• A new action is defined for switching Agent from privileged to unprivileged mode
• When agent receives said action, it switches from privileged to unprivileged mode
• When the switch succeeds, Agent communicates this success to Fleet
• If the switch fails, Agent communicates this failure to Fleet
• Test exercising the success scenario
• Test exercising the failure scenario

[elasticmachine]

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

[ycombinator]

I think it would be useful to do a bit of technical definition for this feature, covering not just the responsibilities of Agent but also the associated responsibilities of Fleet UI and Fleet Server so we have a holistic design in place before starting to implement this feature.

[pierrehilbert]

⚠️ Important note: Fleet UI users should only be switch Agents from privileged to unprivileged mode, not the other way around.

Switching the other way will technically be impossible but I agree we should make it clear from the UI when we will add this feature there.

[blakerouse]

To add some details on the technical implementation for this work. The flow of this should work as the following:

1. Upon receiving the action to switch to unprivileged mode the Elastic Agent should store that action into the state store, but NOT ACK it.
2. Then it should perform the elastic-agent unprivileged as a sub-process, ensuring to create the process in a way where when the daemon process is stopped that it will not stop or kill the spawned elastic-agent unprivileged process.
3. elastic-agent unprivileged should then perform the work (understand here there is a chance that if something goes wrong the process is not coming back without manual intervention) this needs to be made clear in the UI
4. Elastic Agent should then restart, read the state store, determine that it has a unprivileged mode action and ACK the action if it is now unprivileged.

[ycombinator]

@blakerouse WDYT about replacing the ack'ing mechanism with reporting as part of the check-in payload whether the Agent is running as privileged or not, perhaps as part of the local_metadata field? I'm suggesting this because we've found ack'ing to be unreliable in the past when it came to upgrades and when we implemented upgrade details, we decided to communicate them through the check-in payload and it seems to be working well.

[blakerouse]

@ycombinator Actually that would be better.