optimize elasticsearch metrics default search

[gabrielsky]

our company use skywalking for several months
we change metricsDataTTL to 30 days
elasticsearch server ：16core 32G

GET _cat/indices?v&s=docs.count:desc

green  open   skywalking_metrics-longavg-20211111                       bI-5N45nQaOIS71QyMD8ow   1   0   12180869      2414800      2.1gb          2.1gb
green  open   skywalking_metrics-longavg-20211103                       YLNAZbHJT_G53m6hPnfCrA   1   0   12109565      3121639      2.2gb          2.2gb
green  open   skywalking_metrics-longavg-20211112                       7fQIs4IKTt6532IOgFifBg   1   0   11965210      2381682      2.1gb          2.1gb
green  open   skywalking_metrics-longavg-20211104                       IfAXOD92TrWck7JS3TXzgw   1   0   11775141      2493038      2.1gb          2.1gb
green  open   skywalking_metrics-longavg-20211110                       9lQ9Rc5ATkqLplcUbO0zgA   1   0   11747089      2812463      2.1gb          2.1gb
green  open   skywalking_metrics-longavg-20211109                       nzNA4y46RZ-aMjWwDNCF4A   1   0   11645584      2251116        2gb            2gb
green  open   skywalking_metrics-longavg-20211108                       h9EKicUXSOqxHRiTDSdkxw   1   0   11642035      2134679        2gb            2gb
green  open   skywalking_metrics-longavg-20211105                       v6I-7viAQfCLX4uSX3ByhA   1   0   11635243      3686740      2.2gb          2.2gb
green  open   skywalking_metrics-longavg-20211106                       U3gvuz2_SzGI6Oe4p2oH-g   1   0   11393479      3565332      2.1gb          2.1gb

elasticsearch will generate index “skywalking_metrics-longavg-yyyyMMdd” day by day ，and every index size more than 2gb
every “skywalking_metrics-longavg-yyyyMMdd” has a alias “skywalking_metrics-longavg"
so when open skywalking-ui home page or refresh ,it will search by alias of skywalking_metrics-longavg，actually search 30 indexes ,it's easy timeout
my question is why not search the actually index ，most of time default search date range is nearest 15 minutes

I modified some code like this , the default search become fast
AggregationQueryEs7DAO::sortMetrics

...
SearchResponse response = getClient().search(sourceBuilder, tableNames(duration, tableName));
...

    public String[] tableNames(Duration duration, String tableName) {
        List<LocalDate> days = Lists.newArrayList();
        LocalDate start = LocalDate.parse(StringUtils.left(duration.getStart(), 10));
        LocalDate end = LocalDate.parse(StringUtils.left(duration.getEnd(), 10));
        while (!start.isAfter(end)) {
            days.add(start);
            start = start.plusDays(1);
        }
        String[] indexNames = new String[days.size()];
        return days.stream().map(s -> tableName + "-" + s.format(DateTimeFormatter.ofPattern("yyyyMMdd"))).collect(Collectors.toList()).toArray(indexNames);
    }

[wu-sheng]

We tried, but the key issue of this proposal is, if there is one day in the duration doesn't have metric, then no index would be generated. Your query would end with error.

I remember there was one PR about this, but it doesn't end in the main stream because of this

[wu-sheng]

Why or why not is easy. You could try and make yourself understand more.
In any open source community, asking is always easy.

[gabrielsky]

or use the API to determine if index exists during a search

[gabrielsky]

The authorities seem to have a solution to the index's nonexistence

When the ignore_unavailable option is False and an index is missing the restore request will fail.
https://www.elastic.co/guide/en/elasticsearch/client/curator/current/option_ignore.html

@wu-sheng

[wu-sheng]

In a community, we have countless asking every day. We always prefer some one takes actions.
If you could make changes runnable, and send a pull request, we could follow from there.

[gabrielsky]

Thank you for your quick reply~
I will try to write some code and send a pull request In the coming days
I hope Skywalking gets better and better. I love it，It help solves a lot of our problems