Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Maps] add Top term aggregation #57875

Merged
merged 40 commits into from
Feb 26, 2020
Merged
Show file tree
Hide file tree
Changes from 34 commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
30b03a9
[Maps] add Top term aggregation
nreese Feb 18, 2020
127b381
update pew-pew source to handle terms agg
nreese Feb 18, 2020
9e31333
make helper function for pulling values from bucket
nreese Feb 18, 2020
402358e
update terms source
nreese Feb 18, 2020
66967ab
better join labels
nreese Feb 18, 2020
1dff40c
categoricla meta
nreese Feb 18, 2020
19d3720
remove unused constant
nreese Feb 18, 2020
b99138a
remove unused changes
nreese Feb 18, 2020
8aceadb
remove unused constant METRIC_SCHEMA_CONFIG
nreese Feb 18, 2020
b90233d
Merge branch 'master' of github.com:elastic/kibana into top_terms
nreese Feb 18, 2020
6011af3
update jest expect
nreese Feb 18, 2020
90aeae2
fix auto complete suggestions for top term
nreese Feb 18, 2020
28a854f
get category autocomplete working with style props from joins
nreese Feb 18, 2020
46af667
pluck categorical style meta with real field name
nreese Feb 18, 2020
66c5f2b
mock MetricsEditor to fix jest test
nreese Feb 18, 2020
2facf0a
Merge branch 'master' of github.com:elastic/kibana into top_terms
nreese Feb 19, 2020
ecc64b0
review feedback
nreese Feb 19, 2020
dd6f7a8
es_agg_utils.js to es_agg_utils.ts
nreese Feb 19, 2020
9c3bec2
Merge branch 'master' into top_terms
elasticmachine Feb 20, 2020
71d93c1
typing updates
nreese Feb 20, 2020
7991da2
Merge branch 'top_terms' of github.com:nreese/kibana into top_terms
nreese Feb 20, 2020
e99485d
merge with master
nreese Feb 20, 2020
b7747d5
Merge branch 'master' into top_terms
elasticmachine Feb 20, 2020
9b9d9e3
Merge branch 'master' into top_terms
elasticmachine Feb 20, 2020
adddefe
use composit agg to avoid search.buckets limit
nreese Feb 21, 2020
6f11d57
Merge branch 'top_terms' of github.com:nreese/kibana into top_terms
nreese Feb 21, 2020
17bf805
i18n update and functional test fix
nreese Feb 21, 2020
c8762eb
stop paging through results when request is aborted
nreese Feb 21, 2020
9be819e
remove unused file
nreese Feb 21, 2020
4fb0a09
Merge branch 'master' into top_terms
elasticmachine Feb 24, 2020
c2c4feb
Merge branch 'master' into top_terms
elasticmachine Feb 24, 2020
572a97f
do not use composite agg when no terms sub-aggregations
nreese Feb 24, 2020
1aae2ea
clean up
nreese Feb 24, 2020
a126388
pass indexPattern to getValueAggsDsl
nreese Feb 25, 2020
4de141a
Merge branch 'master' of github.com:elastic/kibana into top_terms
nreese Feb 25, 2020
7a0ffa8
review feedback
nreese Feb 25, 2020
79b5a3a
more review feedback
nreese Feb 25, 2020
ac8c04b
ts-ignore for untyped imports in tests
nreese Feb 25, 2020
122491a
more review feedback
nreese Feb 26, 2020
104acf6
add bucket.hasOwnProperty check
nreese Feb 26, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions x-pack/legacy/plugins/maps/common/constants.ts
Original file line number Diff line number Diff line change
Expand Up @@ -117,16 +117,16 @@ export const DRAW_TYPE = {
POLYGON: 'POLYGON',
};

export const METRIC_TYPE = {
export const AGG_TYPE = {
nreese marked this conversation as resolved.
Show resolved Hide resolved
AVG: 'avg',
COUNT: 'count',
MAX: 'max',
MIN: 'min',
SUM: 'sum',
TERMS: 'terms',
UNIQUE_COUNT: 'cardinality',
};

export const COUNT_AGG_TYPE = METRIC_TYPE.COUNT;
export const COUNT_PROP_LABEL = i18n.translate('xpack.maps.aggs.defaultCountLabel', {
defaultMessage: 'count',
});
Expand Down
22 changes: 15 additions & 7 deletions x-pack/legacy/plugins/maps/public/actions/map_actions.js
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ import {
getTransientLayerId,
getOpenTooltips,
getQuery,
getDataRequest,
} from '../selectors/map_selectors';
import { FLYOUT_STATE } from '../reducers/ui';
import {
Expand Down Expand Up @@ -76,7 +77,7 @@ export const HIDE_LAYER_CONTROL = 'HIDE_LAYER_CONTROL';
export const HIDE_VIEW_CONTROL = 'HIDE_VIEW_CONTROL';
export const SET_WAITING_FOR_READY_HIDDEN_LAYERS = 'SET_WAITING_FOR_READY_HIDDEN_LAYERS';

function getLayerLoadingCallbacks(dispatch, layerId) {
function getLayerLoadingCallbacks(dispatch, getState, layerId) {
return {
startLoading: (dataId, requestToken, meta) =>
dispatch(startDataLoad(layerId, dataId, requestToken, meta)),
Expand All @@ -87,6 +88,13 @@ function getLayerLoadingCallbacks(dispatch, layerId) {
updateSourceData: newData => {
dispatch(updateSourceDataRequest(layerId, newData));
},
isRequestStillActive: (dataId, requestToken) => {
const dataRequest = getDataRequest(getState(), layerId, dataId);
if (!dataRequest) {
return false;
}
return dataRequest.dataRequestToken === requestToken;
},
registerCancelCallback: (requestToken, callback) =>
dispatch(registerCancelCallback(requestToken, callback)),
};
Expand All @@ -98,11 +106,11 @@ function getLayerById(layerId, state) {
});
}

async function syncDataForAllLayers(getState, dispatch, dataFilters) {
async function syncDataForAllLayers(dispatch, getState, dataFilters) {
const state = getState();
const layerList = getLayerList(state);
const syncs = layerList.map(layer => {
const loadingFunctions = getLayerLoadingCallbacks(dispatch, layer.getId());
const loadingFunctions = getLayerLoadingCallbacks(dispatch, getState, layer.getId());
return layer.syncData({ ...loadingFunctions, dataFilters });
});
await Promise.all(syncs);
Expand Down Expand Up @@ -412,7 +420,7 @@ export function mapExtentChanged(newMapConstants) {
},
});
const newDataFilters = { ...dataFilters, ...newMapConstants };
await syncDataForAllLayers(getState, dispatch, newDataFilters);
await syncDataForAllLayers(dispatch, getState, newDataFilters);
};
}

Expand Down Expand Up @@ -653,7 +661,7 @@ export function syncDataForLayer(layerId) {
const targetLayer = getLayerById(layerId, getState());
if (targetLayer) {
const dataFilters = getDataFilters(getState());
const loadingFunctions = getLayerLoadingCallbacks(dispatch, layerId);
const loadingFunctions = getLayerLoadingCallbacks(dispatch, getState, layerId);
await targetLayer.syncData({
...loadingFunctions,
dataFilters,
Expand Down Expand Up @@ -773,7 +781,7 @@ export function setQuery({ query, timeFilters, filters = [], refresh = false })
});

const dataFilters = getDataFilters(getState());
await syncDataForAllLayers(getState, dispatch, dataFilters);
await syncDataForAllLayers(dispatch, getState, dataFilters);
};
}

Expand All @@ -792,7 +800,7 @@ export function triggerRefreshTimer() {
});

const dataFilters = getDataFilters(getState());
await syncDataForAllLayers(getState, dispatch, dataFilters);
await syncDataForAllLayers(dispatch, getState, dataFilters);
};
}

Expand Down
13 changes: 6 additions & 7 deletions x-pack/legacy/plugins/maps/public/components/metric_editor.js
Original file line number Diff line number Diff line change
Expand Up @@ -12,17 +12,16 @@ import { EuiFieldText, EuiFormRow } from '@elastic/eui';

import { MetricSelect, METRIC_AGGREGATION_VALUES } from './metric_select';
import { SingleFieldSelect } from './single_field_select';
import { METRIC_TYPE } from '../../common/constants';
import { AGG_TYPE } from '../../common/constants';
import { getTermsFields } from '../index_pattern_util';

function filterFieldsForAgg(fields, aggType) {
if (!fields) {
return [];
}

if (aggType === METRIC_TYPE.UNIQUE_COUNT) {
return fields.filter(field => {
return field.aggregatable;
});
if (aggType === AGG_TYPE.UNIQUE_COUNT || aggType === AGG_TYPE.TERMS) {
return getTermsFields(fields);
}

return fields.filter(field => {
Expand All @@ -38,7 +37,7 @@ export function MetricEditor({ fields, metricsFilter, metric, onChange, removeBu
};

// unset field when new agg type does not support currently selected field.
if (metric.field && metricAggregationType !== METRIC_TYPE.COUNT) {
if (metric.field && metricAggregationType !== AGG_TYPE.COUNT) {
const fieldsForNewAggType = filterFieldsForAgg(fields, metricAggregationType);
const found = fieldsForNewAggType.find(field => {
return field.name === metric.field;
Expand All @@ -64,7 +63,7 @@ export function MetricEditor({ fields, metricsFilter, metric, onChange, removeBu
};

let fieldSelect;
if (metric.type && metric.type !== METRIC_TYPE.COUNT) {
if (metric.type && metric.type !== AGG_TYPE.COUNT) {
fieldSelect = (
<EuiFormRow
label={i18n.translate('xpack.maps.metricsEditor.selectFieldLabel', {
Expand Down
20 changes: 13 additions & 7 deletions x-pack/legacy/plugins/maps/public/components/metric_select.js
Original file line number Diff line number Diff line change
Expand Up @@ -8,44 +8,50 @@ import React from 'react';
import PropTypes from 'prop-types';
import { i18n } from '@kbn/i18n';
import { EuiComboBox } from '@elastic/eui';
import { METRIC_TYPE } from '../../common/constants';
import { AGG_TYPE } from '../../common/constants';

const AGG_OPTIONS = [
{
label: i18n.translate('xpack.maps.metricSelect.averageDropDownOptionLabel', {
defaultMessage: 'Average',
}),
value: METRIC_TYPE.AVG,
value: AGG_TYPE.AVG,
},
{
label: i18n.translate('xpack.maps.metricSelect.countDropDownOptionLabel', {
defaultMessage: 'Count',
}),
value: METRIC_TYPE.COUNT,
value: AGG_TYPE.COUNT,
},
{
label: i18n.translate('xpack.maps.metricSelect.maxDropDownOptionLabel', {
defaultMessage: 'Max',
}),
value: METRIC_TYPE.MAX,
value: AGG_TYPE.MAX,
},
{
label: i18n.translate('xpack.maps.metricSelect.minDropDownOptionLabel', {
defaultMessage: 'Min',
}),
value: METRIC_TYPE.MIN,
value: AGG_TYPE.MIN,
},
{
label: i18n.translate('xpack.maps.metricSelect.sumDropDownOptionLabel', {
defaultMessage: 'Sum',
}),
value: METRIC_TYPE.SUM,
value: AGG_TYPE.SUM,
},
{
label: i18n.translate('xpack.maps.metricSelect.termsDropDownOptionLabel', {
defaultMessage: 'Top term',
}),
value: AGG_TYPE.TERMS,
},
{
label: i18n.translate('xpack.maps.metricSelect.cardinalityDropDownOptionLabel', {
defaultMessage: 'Unique count',
}),
value: METRIC_TYPE.UNIQUE_COUNT,
value: AGG_TYPE.UNIQUE_COUNT,
},
];

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ import { i18n } from '@kbn/i18n';
import { FormattedMessage } from '@kbn/i18n/react';
import { EuiButtonEmpty, EuiSpacer, EuiTextAlign } from '@elastic/eui';
import { MetricEditor } from './metric_editor';
import { METRIC_TYPE } from '../../common/constants';
import { AGG_TYPE } from '../../common/constants';

export function MetricsEditor({ fields, metrics, onChange, allowMultipleMetrics, metricsFilter }) {
function renderMetrics() {
Expand Down Expand Up @@ -100,6 +100,6 @@ MetricsEditor.propTypes = {
};

MetricsEditor.defaultProps = {
metrics: [{ type: METRIC_TYPE.COUNT }],
metrics: [{ type: AGG_TYPE.COUNT }],
allowMultipleMetrics: true,
};
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ import {
} from '@elastic/eui';
import { MetricsEditor } from '../../../../components/metrics_editor';
import { FormattedMessage } from '@kbn/i18n/react';
import { METRIC_TYPE } from '../../../../../common/constants';
import { AGG_TYPE } from '../../../../../common/constants';

export class MetricsExpression extends Component {
state = {
Expand Down Expand Up @@ -59,7 +59,7 @@ export class MetricsExpression extends Component {
render() {
const metricExpressions = this.props.metrics
.filter(({ type, field }) => {
if (type === METRIC_TYPE.COUNT) {
if (type === AGG_TYPE.COUNT) {
return true;
}

Expand All @@ -70,7 +70,7 @@ export class MetricsExpression extends Component {
})
.map(({ type, field }) => {
// do not use metric label so field and aggregation are not obscured.
if (type === METRIC_TYPE.COUNT) {
if (type === AGG_TYPE.COUNT) {
return 'count';
}

Expand Down Expand Up @@ -130,5 +130,5 @@ MetricsExpression.propTypes = {
};

MetricsExpression.defaultProps = {
metrics: [{ type: METRIC_TYPE.COUNT }],
metrics: [{ type: AGG_TYPE.COUNT }],
};
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,12 @@
* you may not use this file except in compliance with the Elastic License.
*/

jest.mock('../../../../components/metrics_editor', () => ({
MetricsEditor: () => {
return <div>mockMetricsEditor</div>;
},
}));

import React from 'react';
import { shallow } from 'enzyme';
import { MetricsExpression } from './metrics_expression';
Expand Down
33 changes: 16 additions & 17 deletions x-pack/legacy/plugins/maps/public/layers/fields/es_agg_field.js
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,10 @@
*/

import { AbstractField } from './field';
import { COUNT_AGG_TYPE } from '../../../common/constants';
import { AGG_TYPE } from '../../../common/constants';
import { isMetricCountable } from '../util/is_metric_countable';
import { ESAggMetricTooltipProperty } from '../tooltips/es_aggmetric_tooltip_property';
import { getField, addFieldToDSL } from '../util/es_agg_utils';

export class ESAggMetricField extends AbstractField {
static type = 'ES_AGG';
Expand All @@ -34,22 +35,21 @@ export class ESAggMetricField extends AbstractField {
}

isValid() {
return this.getAggType() === COUNT_AGG_TYPE ? true : !!this._esDocField;
return this.getAggType() === AGG_TYPE.COUNT ? true : !!this._esDocField;
}

async getDataType() {
// aggregations only provide numerical data
return 'number';
return this.getAggType() === AGG_TYPE.TERMS ? 'string' : 'number';
}

getESDocFieldName() {
return this._esDocField ? this._esDocField.getName() : '';
}

getRequestDescription() {
return this.getAggType() !== COUNT_AGG_TYPE
return this.getAggType() !== AGG_TYPE.COUNT
? `${this.getAggType()} ${this.getESDocFieldName()}`
: COUNT_AGG_TYPE;
: AGG_TYPE.COUNT;
}

async createTooltipProperty(value) {
Expand All @@ -63,18 +63,13 @@ export class ESAggMetricField extends AbstractField {
);
}

makeMetricAggConfig() {
const metricAggConfig = {
id: this.getName(),
enabled: true,
type: this.getAggType(),
schema: 'metric',
params: {},
getValueAggDsl(indexPattern) {
const field = getField(indexPattern, this.getESDocFieldName());
const aggType = this.getAggType();
const aggBody = aggType === AGG_TYPE.TERMS ? { size: 1, shard_size: 1 } : {};
return {
[aggType]: addFieldToDSL(aggBody, field),
};
if (this.getAggType() !== COUNT_AGG_TYPE) {
metricAggConfig.params = { field: this.getESDocFieldName() };
}
return metricAggConfig;
}

supportsFieldMeta() {
Expand All @@ -85,4 +80,8 @@ export class ESAggMetricField extends AbstractField {
async getOrdinalFieldMetaRequest(config) {
return this._esDocField.getOrdinalFieldMetaRequest(config);
}

async getCategoricalFieldMetaRequest() {
return this._esDocField.getCategoricalFieldMetaRequest();
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -5,24 +5,24 @@
*/

import { ESAggMetricField } from './es_agg_field';
import { METRIC_TYPE } from '../../../common/constants';
import { AGG_TYPE } from '../../../common/constants';

describe('supportsFieldMeta', () => {
test('Non-counting aggregations should support field meta', () => {
const avgMetric = new ESAggMetricField({ aggType: METRIC_TYPE.AVG });
const avgMetric = new ESAggMetricField({ aggType: AGG_TYPE.AVG });
expect(avgMetric.supportsFieldMeta()).toBe(true);
const maxMetric = new ESAggMetricField({ aggType: METRIC_TYPE.MAX });
const maxMetric = new ESAggMetricField({ aggType: AGG_TYPE.MAX });
expect(maxMetric.supportsFieldMeta()).toBe(true);
const minMetric = new ESAggMetricField({ aggType: METRIC_TYPE.MIN });
const minMetric = new ESAggMetricField({ aggType: AGG_TYPE.MIN });
expect(minMetric.supportsFieldMeta()).toBe(true);
});

test('Counting aggregations should not support field meta', () => {
const countMetric = new ESAggMetricField({ aggType: METRIC_TYPE.COUNT });
const countMetric = new ESAggMetricField({ aggType: AGG_TYPE.COUNT });
expect(countMetric.supportsFieldMeta()).toBe(false);
const sumMetric = new ESAggMetricField({ aggType: METRIC_TYPE.SUM });
const sumMetric = new ESAggMetricField({ aggType: AGG_TYPE.SUM });
expect(sumMetric.supportsFieldMeta()).toBe(false);
const uniqueCountMetric = new ESAggMetricField({ aggType: METRIC_TYPE.UNIQUE_COUNT });
const uniqueCountMetric = new ESAggMetricField({ aggType: AGG_TYPE.UNIQUE_COUNT });
expect(uniqueCountMetric.supportsFieldMeta()).toBe(false);
});
});
Loading