-
Notifications
You must be signed in to change notification settings - Fork 3.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support env variable in condition #13608
Support env variable in condition #13608
Conversation
…ted BooleanExpression cache. The cache is referenced after compiling to graph.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Overall LGTM, with one concern that is out of scope for this PR: it is quite easy to use a secret from they keystore in a place that is not a password property. In this case it's likely that the value will be logged raw.
On one hand we want users to make their own decisions, but we should guide them towards best practices. We need to track this concern and think of ways to improve on it, maybe by having the cve tell us where the value came from (env or keystore) and log a warning if the element isn't a password setting. Another alternative is to only allow retrieving from the keystore on password setting elements (more intrusive).
import org.logstash.config.ir.graph.QueueVertex; | ||
import org.logstash.config.ir.graph.Vertex; | ||
import org.logstash.config.ir.graph.SeparatorVertex; | ||
import org.logstash.config.ir.graph.*; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We tend do avoid wildcard imports, and you can make intellij not do them automatically, see: https://sergiodelamo.com/blog/intellij-idea-disable-wildcard-imports.html
This PR substitutes ${VAR} in Expression, except RegexValueExpression, with the value in secret store, env. The substitution happens after syntax parsing and before graph execution. Fixed: elastic#5115
This PR substitutes ${VAR} in Expression, except RegexValueExpression, with the value in secret store, env. The substitution happens after syntax parsing and before graph execution. Fixed: elastic#5115
@jsvd Regarding showing log with secret in non password property, I am wondering in what situation we show such log. |
This is the major one. it means that any value in the secret store is dumped raw to disk when someone starts Logstash in debug mode unless it's in a password setting. Historically we've tried to gate this exposure behind the extra "config.debug" flag:
Again, this should not be an issue and users should understand that if it's in keystore, it should be put into a password setting that mascarades itself when logged, but we can also expect that someone will do something like:
Then at some point run the pipeline in debug mode. |
A new feature simplifies the logstash configuration file: elastic/logstash#13608
Release notes
Support environment variable in condition.
Motivation
Two substitutions in ruby side cause inconsistent precedence when resolving the variable.
In the past,
${VAR:default_value}
is resolved in the following precedence: secret store -> environment variable -> default value.After two substitutions, (first substitute) environment variable -> default value -> (second substitute) secret store
The inconsistency could confuse users.
To truly support
if
with value in secret store, this commit implement substitution in java sideWhat does this PR do?
This PR substitutes ${VAR} in Expression, except RegexValueExpression, with the value in secret store, env.
The substitution happens after syntax parsing and before graph execution.
Why is it important/What is the impact to the user?
This PR allows users to use environment variable in condition, eg.
if [app] == "${VAR}"
Checklist
Author's Checklist
()
,!()
,>
,>=
,<
,<=
,==
,!=
,in
,not in
,=~
,!~
,and
,or
,nand
,xor
/${VAR}/
is not expected to do substitutionand
. () and () and ()Known Limitation
Substitution only evaluates ${VAR} to string.
if [app] == "${VAR}"
is a valid statement. VAR is resolved as a string.if [app] == 1
is a valid statement, butif [app] == ${VAR}
is a wrong syntax. VAR cannot be evaluated to a number, hence cannot do number comparison.This is the same behavior as of now. The following count is a string and cannot be a number
How to test this PR locally
run logstash with above config should see
tag
$ VAR=foobar bin/logstash
Related issues
Fixed: #5115
Use cases
Screenshots
Logs