Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for histogram fields to rate aggregation #63289

Conversation

imotov
Copy link
Contributor

@imotov imotov commented Oct 5, 2020

The rate aggregation now supports histogram fields. At the moment only sum
is supported. The value count mode will be added as a follow up.

Closes #62939

The rate aggregation now supports histogram fields. At the moment only sum
is supported. The value count mode will be added as a follow up.

Closes elastic#60674
@imotov imotov marked this pull request as ready for review October 5, 2020 22:50
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-analytics-geo (:Analytics/Aggregations)

@elasticmachine elasticmachine added the Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) label Oct 5, 2020
@imotov imotov added v7.10.0 and removed WIP labels Oct 5, 2020
@imotov
Copy link
Contributor Author

imotov commented Oct 5, 2020

Not sure if it will make it before the feature freeze for 7.10.0, but we can try.

Copy link
Member

@not-napoleon not-napoleon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Needs to be a new aggregator and VSRegistry entry.

for (int i = 0; i < valuesCount; i++) {
double value = values.nextValue();
kahanSummation.add(value);
if (valuesSource instanceof HistogramValuesSource.Histogram) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rather than using an instanceof check and having one aggregator that supports both ValuesSourceTypes, we should create a new aggregator for histogram types, and add a second mapping in the register aggregators method on the factory.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, didn't think about that! Great idea.

@imotov imotov removed the v7.10.0 label Oct 6, 2020
Copy link
Member

@not-napoleon not-napoleon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for making the changes!

@@ -46,7 +40,7 @@ public RateAggregator(
Map<String, Object> metadata
) throws IOException {
super(name, context, parent, metadata);
this.valuesSource = (ValuesSource.Numeric) valuesSourceConfig.getValuesSource();
this.valuesSource = valuesSourceConfig.getValuesSource();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is fine, but just for the sake of mentioning options, we could push getting the values source into the concrete subclasses and do the cast in the constructor, rather than at access time in the leaf reader.

@@ -44,15 +45,21 @@
static void registerAggregators(ValuesSourceRegistry.Builder builder) {
builder.register(
RateAggregationBuilder.REGISTRY_KEY,
List.of(CoreValuesSourceType.NUMERIC, CoreValuesSourceType.BOOLEAN),
RateAggregator::new,
Collections.singletonList(CoreValuesSourceType.NUMERIC),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it a BWC break that we're dropping boolean here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Technically, yes. But this behavior was never documented, and it doesn't make any sense in this context. I am not even sure what would be an interpretation of this number in case of boolean field.

@imotov imotov merged commit 34bff3f into elastic:master Oct 8, 2020
imotov added a commit to imotov/elasticsearch that referenced this pull request Oct 8, 2020
The rate aggregation now supports histogram fields. At the moment only sum
is supported.

Closes elastic#62939
imotov added a commit that referenced this pull request Oct 8, 2020
The rate aggregation now supports histogram fields. At the moment only sum
is supported.

Closes #62939
@imotov imotov deleted the issue-62939-add-rate-support-for-histogram-fields branch October 8, 2020 23:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Analytics/Aggregations Aggregations >enhancement Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) v7.11.0 v8.0.0-alpha1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support histogram fields in rate aggregation
4 participants