Doc: Add more configurable attributes

[karenzone]

Adds more attributes to allow for more content differentiation between Beats and Elastic Agent

Related: #422

[karenzone]

@andsel Let's review content together to make sure that content and context is correct for both beats and Elastic Agent.

[andsel]

At https://github.com/logstash-plugins/logstash-input-beats/pull/423/files#diff-cae5619b3d18ec99c5ccd0a9f6de0c6d3f53343c64692444551a7d29da6863e7R82 we have

...of the name to the Beat version...

maybe, for cases that are not covered by :plugin-uc: Beats, we should define another uppercase not plural placeholder

[andsel]

A similar problem, with singular beat is also present at https://github.com/logstash-plugins/logstash-input-beats/pull/423/files#diff-cae5619b3d18ec99c5ccd0a9f6de0c6d3f53343c64692444551a7d29da6863e7R154

[karenzone]

Carry over note from #417: #417 (review)

[karenzone]

@andsel Those deeplinks aren't resolving correctly for me, so I made some guesses and changed more instances.

Other place to discuss/resolve:

• config sample: lines 47, 51, 52, 54
• ILM: line 59
• multiline: line 64 (see https://www.elastic.co/guide/en/fleet/7.x/fleet-limitations.html)
• versioned indices: lines 81, 84, 92

Noting here that you've already explained that the ECS section should use the [beats] namespace.
I'll set up some time for us to run through the remaining instances and resolve them.

[karenzone]

@andsel Will this work as documented? We're relying on metadata, so I think it might not.

[andsel]

@karenzone we should remove any reference to [@metadata][beat] and [@metadata][version] because the agent doesn't send that information

[karenzone]

@andsel Ready for review. Also, I left a question inline. Thanks!

[karenzone]

@andsel Do we have the info we need to wrap this up?

[andsel]

HI @karenzone as described in the comment left inline, we should remove reference to metadata sent by beat in the case of agent

[andsel]

@karenzone we should remove any reference to [@metadata][beat] and [@metadata][version] because the agent doesn't send that information

[karenzone]

@andsel Thanks for this update.

we should remove any reference to [@metadata][beat] and [@metadata][version] because the agent doesn't send that information

What should we replace it with in the examples, ILM, and versioned indices sections?

[andsel]

In that case, I think we could cut that part of ILM warning and Versioned indices, for the example we could replace it with a datastream enabled one, like:

output {
  elasticsearch {
    hosts => ["http://localhost:9200"]
    data_stream => "true"
  }
}

WDYT?

[karenzone]

@andsel This approach is working so far. For the Versioned indices info, are you proposing that we remove that whole section for Agent for now?

[karenzone]

@andsel Should this reflect the beats version (instead of the stack version)?

[andsel]

[@metadata][version] field read from the remote beats is the version of the remote beat and not of the overall stack

[karenzone]

Thanks for validating. I will fix.

[karenzone]

@andsel Thanks for clarifying some of the fine details. I've introduced conditional processing to handle the differences. Please let me know what you think.

[andsel]

From a content perspective it LGTM!

Maybe we should check how it renders

[karenzone]

@andsel

Maybe we should check how it renders

I always do that locally as part of content development, but it would be nice for you to see it, too.

We can't generate docs until this PR is merged and the gem is published. However, I can do a close approximation. I will put that content into a PR with "do not merge" warnings so you can see it, too. Stay tuned!

UPDATE:
elastic/logstash-docs#1102

[andsel]

LGTM

[karenzone]

Published at v6.1.6: https://rubygems.org/gems/logstash-input-beats/versions/6.1.6-java
Thanks for publishing, @kares ❤️