Demo script used for Elastic Daily Bytes S04E03 - Cloud - Elastic Agent Ingestion

Setup

Start a GCP VM.

On the machine, download the elastic-agent and auditbeat:

curl -L -O https://artifacts.elastic.co/downloads/beats/elastic-agent/elastic-agent-8.1.3-amd64.deb
curl -L -O https://artifacts.elastic.co/downloads/beats/auditbeat/auditbeat-8.1.3-amd64.deb

If already installed, remove them:

sudo systemctl stop elastic-agent
sudo systemctl stop auditbeat
sudo dpkg --purge elastic-agent
sudo dpkg --purge auditbeat

Demo part

Create a cluster

Open your cloud deployment in the cloud console. Create a new cluster.

Make sure to create a cluster with the integrations server and the enterprise search. It's proposed by default.

Highlight the fact that everything is deployed and managed for us within few clicks. So we can easily start to ingest new data as we need to.

Add new integrations

Show the system integration:

Monitor the system

Add it:

Configure it:

Install the agent

And follow the guide:

# curl -L -O https://artifacts.elastic.co/downloads/beats/elastic-agent/elastic-agent-8.1.3-amd64.deb
sudo dpkg -i elastic-agent-8.1.3-amd64.deb

It gives:

Selecting previously unselected package elastic-agent.
(Reading database ... 43909 files and directories currently installed.)
Preparing to unpack elastic-agent-8.1.3-amd64.deb ...
Unpacking elastic-agent (8.1.3) ...
Setting up elastic-agent (8.1.3) ...
Processing triggers for systemd (232-25+deb9u13) ...

Enroll the agent

Copy from the Linux / macOS tab the url and the token:

• URL: https://ed32a7ba14f1459fa2359402f293c600.fleet.europe-west1.gcp.cloud.es.io:443
• TOKEN: VnBISlg0QUJESjZONVQxcXUzUkg6Z2VHcTlHTmxSd1dWaHgxZW5BUkcyZw==

sudo elastic-agent enroll --url=https://ed32a7ba14f1459fa2359402f293c600.fleet.europe-west1.gcp.cloud.es.io:443 --enrollment-token=VnBISlg0QUJESjZONVQxcXUzUkg6Z2VHcTlHTmxSd1dWaHgxZW5BUkcyZw==

It gives:

{"log.level":"info","@timestamp":"2022-04-25T08:30:44.320Z","log.origin":{"file.name":"cmd/enroll_cmd.go","file.line":455},"message":"Starting enrollment to URL: https://ed32a7ba14f1459fa2359402f293c600.fleet.europe-west1.gcp.cloud.es.io:443/","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-25T08:30:45.593Z","log.origin":{"file.name":"cmd/enroll_cmd.go","file.line":253},"message":"Elastic Agent might not be running; unable to trigger restart","ecs.version":"1.6.0"}
Successfully enrolled the Elastic Agent.

Enable and start the agent:

sudo systemctl enable elastic-agent 
sudo systemctl start elastic-agent

Open the "[Metrics System] Overview" asset.

And click on the Host one:

And open the Observability Application:

Start the injector:

./injector.sh

And look at the data.

Add OsQuery

When deployed to the agents, you can start running commands on the host machines:

Run:

select * from etc_hosts
select * from os_version

Add Auditbeat

That's the old way for doing this. Some of the beats components are not yet deployable through the elastic agent.

#curl -L -O https://artifacts.elastic.co/downloads/beats/auditbeat/auditbeat-8.1.3-amd64.deb
sudo dpkg -i auditbeat-8.1.3-amd64.deb

sudo vi /etc/auditbeat/auditbeat.yml

cloud.id: "Elastic_Daily_Bytes_S04:ZXVyb3BlLXdlc3QxLmdjcC5jbG91ZC5lcy5pbyQ0ODdjZTRlMDk2ZDI0NTEzYWMwNTkwNGU2ZWU2MjdhYyRkMTk0YTI1MTczYTc0YjQ5OWUzZDE2NTkzNzQ2ZTAzMg=="
cloud.auth: "elastic:<password>"

sudo auditbeat setup
sudo service auditbeat start

Check the data coming.

Open the Security App to show that data from observability is also used for security, out of the box.

Webcrawler

If time allows.

Because we also deployed by default the enterprise search service, we can crawl a website like my personal blog post https://david.pilato.fr.

Conclusion

Just start a cluster and you can immediately start to get data to it. It could be as you seen previously by adding sample data or by deploying the agent on the hosts you want to monitor or secure. It could be by crawling also the website.

The next session will be about making all those deployments even more automatic by using Terraform. Alex will cover this tomorrow.