-
Notifications
You must be signed in to change notification settings - Fork 24.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[ML][Data Frames] Fix null aggregation handling in indexer #45061
[ML][Data Frames] Fix null aggregation handling in indexer #45061
Conversation
Pinging @elastic/ml-core |
5be3009
to
f4fd537
Compare
"Data frame transform unexpectedly stopped seeing data. " + | ||
"Please verify that these indices exist and are open [" + | ||
Strings.arrayToCommaDelimitedString(getConfig().getSource().getIndex()) + | ||
"]."); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I waffled a bit on how to word this. From my understanding, aggregations
should only ever be null here iff the underlying indices are all closed/deleted
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yes, it's only null
if source became empty.
With the given wording, it sounds more like a warning. So I would either change the log/audit level or the wording.
run elasticsearch-ci/packaging-sample |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What about other places in the code? My gut feeling about #44906: it's more than this, see the dev notes about _preview
and deferred validation.
"Data frame transform unexpectedly stopped seeing data. " + | ||
"Please verify that these indices exist and are open [" + | ||
Strings.arrayToCommaDelimitedString(getConfig().getSource().getIndex()) + | ||
"]."); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yes, it's only null
if source became empty.
With the given wording, it sounds more like a warning. So I would either change the log/audit level or the wording.
@hendrikmuhs we already validate for concrete index existence on |
@elasticmachine update branch |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think we need the words "Concrete" or "Underlying" in the messages, but apart from that I'm happy with this change
.../java/org/elasticsearch/xpack/dataframe/action/TransportPreviewDataFrameTransformAction.java
Outdated
Show resolved
Hide resolved
.../data-frame/src/main/java/org/elasticsearch/xpack/dataframe/transforms/DataFrameIndexer.java
Outdated
Show resolved
Hide resolved
.../data-frame/src/main/java/org/elasticsearch/xpack/dataframe/transforms/DataFrameIndexer.java
Outdated
Show resolved
Hide resolved
…wtrent/elasticsearch into bug/ml-df-fix-npe-when-aggs-are-null
@elasticmachine update branch |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
I am not sure about the logging/audit w.r.t. a PR I am working on, but this can be revisited as a follow up.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
run elasticsearch-ci/2 |
…5061) * [ML][Data Frames] Fix null aggregation handling in indexer * addressing PR comments * adjusting error messages
…5061) * [ML][Data Frames] Fix null aggregation handling in indexer * addressing PR comments * adjusting error messages
Since we are more lenient now when expanding index patterns, it could be that there are no open concrete indices contained within the pattern. When this is the case
SearchResponse#getAggregations()
returnsnull
.This adds a
null
check for that method, audits that possibly all the underlying indices are closed/deleted, and returns that the current checkpoint is done.closes #44906