Running custom inputs under Elastic Agent

[ruflin]

Elastic Agent is a supervisor and can run any binary supporting the elastic-agent-client. This allows Elastic Agent to be extended with additional binaries. At the moment these binaries must be built by Elastic for signing. One of the ideas discussed is, that at one point this can also be used by non Elastic binaries.

But the above still puts a burden on running custom inputs / collectors built by users. The collectors might already exist and adding support for the elastic-agent-client is not feasible. Instead these collectors / scripts could be executed by an input similar to the old exectbeat.

This exec input could be configured to run any command on a predefined schedule and would read the data from stdout. Different output formats like prometheus, json and others would be supported.

The configuration could look something like:

inputs:
  - type: command
    schedule: 10s
    format: json
    run: /foo/bar/my-collector.py

This would run my-collector.py every 10s.

There are security concerns around being able to just execute commands in an Elastic Agent especially in the context of Fleet. I would expect this input to be in a separate binary potentially not shipped by default. In addition, it could be blocked in capabilities.yml by default. A user would have to enable it for every Elastic Agent where this should be available.

[elasticmachine]

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

[ruflin]

Lots of previous technical discussions on this happened already in elastic/beats#18323

[jlind23]

@ruflin could you keep only one issue to track the work down? :-)

[ruflin]

I don't think it is the same issue. The previous one is around Beats, this one here is for Elastic Agent. But the security challenges are the same. My assumption with Elastic Agent around is that it is less likely that this will still happen in Beats so we could close the other one but make sure we look up the technical discussion there.

[jlind23]

@kvch could you consider it for the input manager proposal?

[ericleong86]

+1 to this.

From security perspective, most established EDR vendors have ways to run custom scripts on adhoc or upon certain event basis (some even have remote access capabilities). Of course, all of these functionality comes with privilege management on the console and proper audit reporting.

The use case is mainly to be used as incident response tool when a breach is detected. Some customers leverage this function as additional forensics collection after the infected endpoint/server has been isolated (quarantined).

[yvbondarenko]

+1

[nithril]

👍

[RichardJECooke]

A workaround to having no custom script module is to run MetricBeat with the HTTP module, which can connect to anything, and connect it to another container. This other container runs a little Node Express web server that can collect whatever type of metric you want from wherever you want, and expose it as an HTTP GET.

Haven't tried this yet, but I think this will work, won't it?

[zez3]

A workaround to having no custom script module is to run MetricBeat with the HTTP module, which can connect to anything,

For that you could use the custom integrations(tcp, udp, file stream or http) all using a beat underneath. You could also use the EDR https://www.elastic.co/docs/current/serverless/security/response-actions
This issue is about running your custom Binary(own beat if you will)
From the security stand this new binary should be whitelisted beforehand

[Erikg346]

+1