Add Force Merge to hot lifecycle phase #43165
Comments
polyfractal
added
the
:Data Management/ILM+SLM
|
Pinging @elastic/es-core-features |
@mud5150 will doing a hot phase with a |
|
It could work, however in my particular instance all phases are in use. In my instance "hot" servers hold 28 days of data. Warm servers (slower disks) hold the rest of the year and still get a good amount of user queries. Cold nodes have slow disks and high ratio of storage to memory, use frozen indices, etc. It's not a problem for me to have curator force merge the index after the rollover happens, I just thought the above seems like a reasonable use case. |
|
We discussed this and decided to allow the force merge action in the hot phase, with a small caveat. We will require that the |
This commit changes the `forcemerge` action to also be allowed in the `hot` phase for policies. The forcemerge will occur after a rollover, and allows users to take advantage of higher disk speeds for performing the force merge (on a separate node type, for example). On caveat with this is that a `forcemerge` in the `hot` phase *MUST* be accompanied by a `rollover` action. ILM validates policies to ensure this is the case. Resolves elastic#43165
* Allow forcemerge in the hot phase for ILM policies This commit changes the `forcemerge` action to also be allowed in the `hot` phase for policies. The forcemerge will occur after a rollover, and allows users to take advantage of higher disk speeds for performing the force merge (on a separate node type, for example). On caveat with this is that a `forcemerge` in the `hot` phase *MUST* be accompanied by a `rollover` action. ILM validates policies to ensure this is the case. Resolves #43165 * Use anyMatch instead of findAny in validation * Make randomTimeseriesLifecyclePolicy single-pass
* Allow forcemerge in the hot phase for ILM policies This commit changes the `forcemerge` action to also be allowed in the `hot` phase for policies. The forcemerge will occur after a rollover, and allows users to take advantage of higher disk speeds for performing the force merge (on a separate node type, for example). On caveat with this is that a `forcemerge` in the `hot` phase *MUST* be accompanied by a `rollover` action. ILM validates policies to ensure this is the case. Resolves elastic#43165 * Use anyMatch instead of findAny in validation * Make randomTimeseriesLifecyclePolicy single-pass
* Allow forcemerge in the hot phase for ILM policies This commit changes the `forcemerge` action to also be allowed in the `hot` phase for policies. The forcemerge will occur after a rollover, and allows users to take advantage of higher disk speeds for performing the force merge (on a separate node type, for example). On caveat with this is that a `forcemerge` in the `hot` phase *MUST* be accompanied by a `rollover` action. ILM validates policies to ensure this is the case. Resolves #43165 * Use anyMatch instead of findAny in validation * Make randomTimeseriesLifecyclePolicy single-pass
Add Force Merge to hot lifecycle phase:
For time series data it makes sense that you may want to do a force merge on an index in the hot phase after rollover occurs. Seems pretty straight forward but if you want more information LMK.
The text was updated successfully, but these errors were encountered: