Index is too large #437
-
|
I have 20 million articles, I tried to index 80 thousand article titles, the index size is 215.59 MB, the indexed data looks like the following。 If all indexes will reach nearly 60GB, is this normal? It's only 5GB in my mysql,Is it the Mappings setting or something else? |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment
-
|
We us compression to reduce size of the indexes. We also use sharding to improve performance. We use 2 layers of sharding. The more shards you have, the higher the size will be. You can try reducing the size of the index by tuning the following environment variables. ZINC_SHARD_NUM=1 // For 1 GB default shard size. You can change it to 10 GB to reduce index size. |
Beta Was this translation helpful? Give feedback.
We us compression to reduce size of the indexes. We also use sharding to improve performance. We use 2 layers of sharding. The more shards you have, the higher the size will be. You can try reducing the size of the index by tuning the following environment variables.
ZINC_SHARD_NUM=1
// For 1 GB default shard size. You can change it to 10 GB to reduce index size.
ZINC_SHARD_MAX_SIZE=1073741824