[Filebeat] Add support for specifying AWS cred file #15656
Conversation
leehinman
added
bug
Filebeat
|
Pinging @elastic/siem (Team:SIEM) |
leehinman
changed the title
leehinman
force-pushed
the
15652_cloudtrail_fix
branch
from
81d7a5b
to
9f98663
Compare
| #var.credential_profile_name: fb-aws | ||
|
|
||
| # filename of shared credential file | ||
| # Use "" to specify default |
There was a problem hiding this comment.
Do we have to speicify "" to get default? If var.shared_credential_file is not specified in the aws.yml config, it should goes to default ~/.aws/credentials right?
There was a problem hiding this comment.
@kaiyan-sheng When I was testing if var.shared_credential_file wasn't set at all, I would get an error template map has no entry for key. I tried testing for the existence of the key (if, with & index) but got the same error. If we could conditionally test for the presence of the key that would be ideal. Any ideas?
There was a problem hiding this comment.
In config template you can check if the variables is defined and handle it however you want. Just make sure you don't have a default value in the module manifest. For example:
shared_credential_file: {{ if .shared_credential_file }}{{ .shared_credential_file }}{{ end }}
or
shared_credential_file: {{ if .shared_credential_file }}{{ .shared_credential_file }}{{ else }}/some/file{{ end }}
and this is my preferred way because you are not duplicating default values in several places.
{{ if .shared_credential_file }}
shared_credential_file: {{ .shared_credential_file }}
{{ end }}
There was a problem hiding this comment.
@andrewkroh It's likely I'm making a stupid mistake, but I tried the last one and got:
2020-01-20T20:47:59.256-0600 ERROR instance/beat.go:921 Exiting: Error getting config for fileset aws/cloudtrail: Error interpreting the template of the input: template: text:6:6: executing "text" at <.shared_credential_file>: map has no entry for key "shared_credential_file"
Exiting: Error getting config for fileset aws/cloudtrail: Error interpreting the template of the input: template: text:6:6: executing "text" at <.shared_credential_file>: map has no entry for key "shared_credential_file"
I'm wondering if this is because of
tpl := template.New("text").Option("missingkey=error")
line 263 in fileset.go . The missingkey=error got added in November.
There was a problem hiding this comment.
yep, changing to missingkey=zero made it so that
{{ if .shared_credential_file }}
shared_credential_file: {{ .shared_credential_file }}
{{ end }}
worked.
There was a problem hiding this comment.
The variable isn't declared in the manifest. Try adding it to this file, but without a default. Then when its not set a zero value will be added to the template data. It should behave like this.
There was a problem hiding this comment.
That worked. Thanks. Knew I was missing something.
Sure. |
- add "shared_credential_file" to cloudtrail config Fixes elastic#15652
- also make credential_profile_name optional
leehinman
force-pushed
the
15652_cloudtrail_fix
branch
from
9f98663
to
bee24fd
Compare
* Add optional AWS shared_credential_file to all s3 input modules * Made AWS credential_profile_name optional for all s3 input modules Fixes elastic#15652 (cherry picked from commit 005f474)
leehinman
added
v7.7.0
and removed
needs_backport
* Add optional AWS shared_credential_file to all s3 input modules * Made AWS credential_profile_name optional for all s3 input modules Fixes elastic#15652 (cherry picked from commit 005f474)
…cred file (#15909) * [Filebeat] Add support for specifying AWS cred file (#15656) * Add optional AWS shared_credential_file to all s3 input modules * Made AWS credential_profile_name optional for all s3 input modules Fixes #15652 (cherry picked from commit 005f474) * update aws.asciidoc * update variables with default for 7.5 only Co-authored-by: Lee Hinman <57081003+leehinman@users.noreply.github.com>
…ng AWS cred file (elastic#15909) * [Filebeat] Add support for specifying AWS cred file (elastic#15656) * Add optional AWS shared_credential_file to all s3 input modules * Made AWS credential_profile_name optional for all s3 input modules Fixes elastic#15652 (cherry picked from commit a6a9c37) * update aws.asciidoc * update variables with default for 7.5 only Co-authored-by: Lee Hinman <57081003+leehinman@users.noreply.github.com>
Fixes #15652