Recommendations for reducing the number of indexes

Please answer these questions before submitting your issue.

• Why do you submit this issue?
• Question or discussion
• Bug
• Requirement
• Feature or performance improvement

---

Question

• What do you want to know?

Under the default parameters of SW, the number of ES indexes has reached more than 500, and the number of shards has exceeded the default maximum index number of ES shards of 1000. Under certain conditions, the larger the number of shards will affect query performance, It will also increase the burden of cluster management and increase the instability of the cluster.

---

Bug

• Which version of SkyWalking, OS and JRE?

• Which company or project?

• What happen?
  If possible, provide a way for reproducing the error. e.g. demo application, component version.

---

Requirement or improvement

• Please describe about your requirements or improvement suggestions.
  Is it possible to combine hour, day, and month indexes and distinguish them by a type field inside the index, which is expected to reduce the number of indexes by 3/4. Query performance may be partially improved due to the use of fewer shards. But this has not been actually tested.

[wu-sheng]

Your idea is not possible, those four are using different TTL, which is hard to share the same index files. The idea could be sharing index for different metrics of different metrics. Adding a new column of metrics name in the index, and then, the query could use this. But across different time levels is not a good way.

Maybe i didn't make it clear. Because I looked at the mapping of these four indexes, the structure is similar. Is it possible to add a type field to indicate the TTL type, that is, to store hour, day, and month in this field, so that all four queries query the same index name and use type = hour or day or month to filter

[wu-sheng]

But you can't delete data, which makes the TTL meaningless. You can't just make query works,

Deleting data is the same as before, but when deleting, add use type = hour or day or month to filter.This is the same as the previous logic. The only difference is the index name parameter, which is filtered by the type field.

[darcydai]

But you can't delete data, which makes the TTL meaningless. You can't just make query works,

I am more concerned skywalking support set different shards for indices,
Especially segment index ,size is very large than others.

[wu-sheng]

Deleting data is the same as before, but when deleting, add use type = hour or day or month to filter.This is the same as the previous logic. The only difference is the index name parameter, which is filtered by the type field.

No, it doesn't. Currently, from 6.2.0, deleting mean remove the whole index, rather than data, which will cause elasticsearch GC or even FGC. So if you put different precision data in the same index, this is a problem.

[wu-sheng]

I am more concerned skywalking support set different shards for indices,
Especially segment index ,size is very large than others.

I think this could be discussed and changed. After all, the segment is different from others. We could provide some config for segment only. Do you want to try to propose a pull request to do so?

[darcydai]

I am more concerned skywalking support set different shards for indices,
Especially segment index ,size is very large than others.

I think this could be discussed and changed. After all, the segment is different from others. We could provide some config for segment only. Do you want to try to propose a pull request to do so?

Yes !. I glad to it, I will open a pull request after completed.

Deleting data is the same as before, but when deleting, add use type = hour or day or month to filter.This is the same as the previous logic. The only difference is the index name parameter, which is filtered by the type field.

No, it doesn't. Currently, from 6.2.0, deleting mean remove the whole index, rather than data, which will cause elasticsearch GC or even FGC. So if you put different precision data in the same index, this is a problem.

I'm also talking about deleting the index, not deleting the data inside. Deleting data does have performance problems, which is related to the principle of inverted index. For example, all_p75_day-20191225 and all_p75_hour-20191225 are deleted on a daily basis, But they can be merged into one, and the entire index is deleted dayly when merge one index.

[wu-sheng]

For example, all_p75_day-20191225 and all_p75_hour-20191225 are deleted on a daily basis, But they can be merged into one, and the entire index is deleted dayly when merge one index.

But you break the TTL configuration. And a more simple question, how do you name this config? Could you describe it?

If it is a merge index, these may need to be modified

[wu-sheng]

If it is a merge index, these may need to be modified

Modification is not the key point. The point is how do you explain this to the user. Also, it is hard to name. Could you have an accurate and easily understandable name for this config?

[wu-sheng]

My only thought is only metricsTTL, and you have to explain, it will affect the different precision data, including month, day, hour and minute.

OK, I have n’t figured it out yet, let me think again