update usage formulas for Manila shares as requested by Manila team

[majewsky]

Based on a request from @chuan137. Can you guys please verify that I understood the requirement correctly?

[Carthaca]

Let's exercise some examples:

Assumption: Castellum shrinks when usage is below 50%, extends when usage above 80%. Percentage-step resizing with 10%.
Share size 10GB in manila => 20GB volume in NetApp with volume usable size 10GB and snapshot reserve 10GB.
Terms: Spillover == snapshot space is eating up into volume usable size, i.e. snapshot usage > 100%

Example A:
Volume usage 60% (6 of 10GB), snapshot usage 90% (9 of 10 GB)
-> NEW: share would be extended to 12GB (50% vol and 75% snap usage)
-> OLD: no action
-> not ideal, no spillover and volume usage was not high

Example B:
Volume usage 40%, snapshot usage 60%
-> NEW: no action
-> OLD: share would be shrinked to 8GB (50% vol and 75% snap usage)
-> not ideal, old scenario had no spillover either

So I think what we want is: never shrink lower than 100% snap usage and extend if snap usage > 100%.
So using a fixed value of 100% snap usage instead of the values set from the customers that were meant for the volume size usable in the active filesystem.

Wdyt?

[Carthaca]

@majewsky can we have it that way, that customers can set the threshold for volume usage and snapshot usage independently?

[majewsky]

In principle, we do support multi-usage resources (currently only used for server groups that scale with CPU usage and RAM usage). However, changing this on an existing asset type is a breaking change in the API and UI. We should be extremely sure that it's a good idea before proceeding down that path.

[chuan137]

Can we keep the old logic for proposing new share size, and add additional step to make sure the new share size is not smaller than snapshotUsage, sth like

newSize = proposeNewSize(volumeUsageSize, ...)
newSize = max(snapshotUsageSize, newSize)

UI can be kept as it was? Snapshot usage is handled internally and kept away from Customer's consideration.

[majewsky]

That would require adding an asset-level MinSize constraint into the core logic. After going through all the calculations again with my colleagues over here, I can see that this would be required for a full solution. However, adding this constraint requires working on the core logic and thus would be a substantial undertaking to ensure that no bugs are accidentally introduced.

I propose the following plan:

1. To avoid situations like this where we end up doing lots of roundtrips because I need to implement a formula that should fundamentally be under your control, I would like you to add metrics to the netapp-exporter that reflect the result of all those calculations, e.g.

   netapp_volume_size_bytes_for_castellum # must be equal to what the Manila API reports
   netapp_volume_usage_bytes_for_castellum
   netapp_volume_min_size_bytes_for_castellum # share size that Castellum should never resize below
   

   I'm not attached to these names, obviously; feel free to come up with better names if you like.
2. As I implement these metrics, I add support for the asset-level MinSize constraint to the Castellum scaling logic.

[majewsky]

FYI: I had some time today and implemented the option for asset-level MinSize constraints in the core logic (2fceeb6).

[chuan137]

PR for the metrics including the minimal share size

[majewsky]

We have moved to a different set of metrics that allow the Manila team to have ownership of these formulas.