Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fleet Server package #544

Merged
merged 9 commits into from
Jan 20, 2021
Merged

Fleet Server package #544

merged 9 commits into from
Jan 20, 2021

Conversation

aleksmaus
Copy link
Member

Bootstraps all the indexes needed for the Fleet Server once the integration is installed.

Screen Shot 2021-01-19 at 5 05 05 PM

@ruflin @blakerouse
Since it creates only ILM policies and templates, will have another PR for the Fleet Server to make it resilient to the situation where the indexes do not exists yet.

Closing the initial draft #540 that @ruflin started.

@elasticmachine
Copy link

elasticmachine commented Jan 19, 2021

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: Pull request #544 updated

    • Start Time: 2021-01-20T16:19:22.222+0000
  • Duration: 7 min 11 sec

  • Commit: 202c4bc

Test stats 🧪

Test Results
Failed 0
Passed 1
Skipped 0
Total 1

@andresrc andresrc added the Team:Elastic-Agent Label for the Agent team label Jan 20, 2021
@elasticmachine
Copy link

Pinging @elastic/agent (Team:Agent)

@ruflin
Copy link
Member

ruflin commented Jan 20, 2021

@ycombinator This is a package without a data_stream which I think is valid?

Update: change would need to happen here? https://github.com/elastic/package-spec/blob/master/versions/1/spec.yml#L25

@@ -0,0 +1,23 @@
{
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aleksmaus Very interesting that it just works with the dot prefix. It will make it fun to develop as some OS hide these files :-)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you mean windowz? :-)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OS X ...

"index_patterns": [
".fleet-servers"
],
"template": {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder how restrictive we should be in these templates. Should we allow additional fields or not? If yes, keyword as default?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For upgrades would that be good to add to all the templates? Just incase that Fleet Server itself is upgraded before the package?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably. In the ideal scenario the template would of course always be first and we could even use strict mappings, but not the case for now.

@ruflin
Copy link
Member

ruflin commented Jan 20, 2021

@aleksmaus What I suggest to keep moving here is adding a "fake" data_stream to pass validation, merge PR, promote it to snapshot so we can start using this for testing in Kibana / fleet-server.

I think we will have follow up discussions around more details in the template and ILM policy but this can come later.

Copy link

@blakerouse blakerouse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good, some questions and comments.

title: Fleet Server
description: Fleet Server setup
inputs:
- type: fleet_server

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change type to fleet-server. Fleet Server already expects the configuration file format to be:

inputs:
  - type: fleet-server
    host: 0.0.0.0
    port: 8000

inputs:
- type: fleet_server
vars:
# TODO: Mainly for testing, probably no config options at first

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should do host binding and port binding.

Which are the fields host and port.

required: true
show_user: true
default:
- 127.0.0.1

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should default to 0.0.0.0.

default:
- 127.0.0.1
title: "Fleet Server parts"
description: "Hello World Fleet Server"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should add port with default set to 8000 (as it's the default now). We should probably change that.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aleksmaus
Copy link
Member Author

Screen Shot 2021-01-20 at 10 16 31 AM

Copy link
Member

@ruflin ruflin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. It doesn't make it into the policy yet but lets get it in so we have all the assets.

title: Fleet Server
description: Fleet Server setup
inputs:
- type: fleet-server
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This does not end up in the policy currently as we don't use it in any of the data streams and have a template for it. Maybe a good use for our fake data stream?

We had a discussion around this for pure input packages. I'm trying to find the issue.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aleksmaus aleksmaus merged commit c8027b0 into elastic:master Jan 20, 2021
@ycombinator
Copy link
Contributor

ycombinator commented Jan 25, 2021

Hi @aleksmaus, @ruflin, I was on PTO all last week and am catching up on PRs today.

I see that this PR was merged with a fake data_stream folder to keep moving forward (👍 ). But now that we have elastic/package-spec#113 merged (along with elastic/elastic-package#232 and #563), could we get a follow up PR to remove the fake data_stream folder from the fleet_server package in the integrations repo? Thanks!

eyalkraft pushed a commit to build-security/integrations that referenced this pull request Mar 30, 2022
* Add fleet-server package

* add ilm policy examples

* fix template formatting

* fix priority so it does not conflict

* apply proper formatting

* Functional indices boostrap

* Make lint happier

* Address review feedback. Add fake datastream, to pass linter

* Better Fleet Server configuration description

Co-authored-by: ruflin <spam@ruflin.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Team:Elastic-Agent Label for the Agent team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants