[Fleet] Fleet project

[nchaulet]

Summary

Meta issue for the fleet project, to regroup the issues related to this project.
Currently the project live under the feature-make-it-ingest branch

Glossary

Agent: an agent installed on a system, an agent could be permanent (agent permanently installed on a server), ephemeral (agent in container), or temporary.

Open issues/questions

• TLS cert Trust kibana and pass cert of ES to agent via now trusted kibana?

Scope

The fleet project entails the items listed under the Fleet tab in the new Ingest app OR Kibana management section.
Fleet will:

• Enroll agents, and ensure their status / events API is returning data that is actionable & meaningful to users.
• Assign agents to policies (1 - to - many)
• Allow for actions to be sent to agents such as pausing or resuming an agent.
• Manage the protocols & APIs for Agent <> Kibana communications

Blockers

• Functional programatic and REST API for policies [EPM] Ingest - Policy and Datasources API #48028
• Need to get full install instructions working with the Agent (templating system already implemented) https://github.com/elastic/fleet/issues/140
• Kibana user gets added privileges? Or new Ingest user (email thread)

MVP functionality:

• Screen designs

Agents tab

• List all agents
• Filter by policy ID dropdown
• Filter by policy name vs ID [Fleet] Filter by policy name not ID #48029 (BLOCKED: [EPM] Ingest - Policy and Datasources API #48028)
• Page via API
• Support KQL query bar for filtering
• Support select all selecting all agents matching the query
• Support filter out inactive agents toggle
• Allow unenroll of an agent or agents
• Display agent status calculated by last checkin date by type
• Enrollment API Key CRUD UI [fleet] Enrollment API Key management UI #49207

Agent details screen

• Display agent metadata
• Allow users to add/edit meta data of an agent in the UI
• Allow unenroll of an agent
• Display agent event logs
• Display data about host

Policy List Tab (partially BLOCKED: #48028)

• Display a list of Policies [Fleet] Display a list of Policies #49227
• Support removing a selected policy [Fleet] Support removing a selected policy #49228
• Support filters on the list of policies [Fleet] Support filters on the list of policies #49229

Policy Details (partially BLOCKED: #48028)

• Display metrics of aggregated agent statuses on a policy [Fleet] Display metrics of agrigated agent statuses on a policy #49230
• List data sources on a policy with links to view the data source in EPM [Fleet] List data sources on a policy with links to view the data source in EPM #49231
• Support removing the policy [Fleet] Support removing a selected policy #49228

Other UI

• Create "Ingest UI" framework in the style of management UI [Fleet] Create new parent "Ingest Management" app #49234

API

• Define and document agent API endpoints
• Move to ES API Tokens for enroll and access tokens [Fleet] Use Elasticsearch API Keys instead of custom JWT tokens #48968
• Enhance kibana user or add ingest user for Fleets internal actions
• use .fleet indexes [Ingest/Fleet] Use .fleet and .ingest index #46373
• Connect agent checking to policy APIs [Fleet] get policy from EPM policy api and format for agent #48448 (BLOCKED: [EPM] Ingest - Policy and Datasources API #48028)

enrollment /install scripts (BLOCKED https://github.com/elastic/fleet/issues/140)

Note: code is in place, just need to add instructions

• Bash install script for *nix
• install script for windows
• config instructions for Docker
• config instructions for k8
• config instructions for Terraform
• config instructions for Puppet
• config instructions for Ansible
• config instructions for Chief

Testing

• Work with Endgame to setup load testing of agent + fleet [Fleet] Setup load testing / benchmarks #49765
• Setup UI integration tests
• Setup Agent <> Fleet integration tests (CI server setup needed)
• Get ride of memory repository [Fleet] Refactorize server test: remove in memory repository #50172

[nchaulet]

@mattapperson the APIs for policy, datasource, ... are going to be exposed by the ingest plugin right?

[mattapperson]

correct, but please count them here as Ingest plugin is part of the Fleet "project"

[elasticmachine]

Pinging @elastic/ingest (Feature:Fleet)

[ruflin]

Closing this issue as we now track the overall Ingest Manager release.