Many ZfsThin Snapshots with Minimal Data Usage Inflate Reserved Capacity #438
Description
In tag 1.30.4, when calculating reserved capacity, the code at
controller/src/main/java/com/linbit/linstor/core/apicallhandler/controller/FreeCapacityAutoPoolSelectorUtils.java:232
calls the function getReservedSum with a collection of VlmProviderObject items, which are actually SnapShotVolumes.
Inside getReservedSum (around line 250), the method currently uses the AllocatedSize to determine how much capacity should be reserved. However, for snapshot volumes, it might make more sense to use the "used" size instead of the "allocated" size. This is because the "used" size represents the actual amount of storage consumed.
Here is the code
private static <RSC extends AbsResource<RSC>> long getReservedSum(
Collection<VlmProviderObject<RSC>> absVlmListRef
)
{
long ret = 0;
for (VlmProviderObject<RSC> absVlmData : absVlmListRef)
{
if (absVlmData.getVolume() instanceof SnapshotVolume)
{
ret += absVlmData.getAllocatedSize();
}
else
{
ret += absVlmData.getUsableSize();
}
}
return ret;
}In practice, if you have many snapshots with no changes (zero delta), each snapshot might show a high AllocatedSize even though they all point to the same underlying data and do not consume additional space. As a result, basing reserved capacity on AllocatedSize could overestimate the storage required, whereas using the "used" size would more accurately reflect the real consumption.
Here is an example of zfs list output that demostrates this issue
root@wood:~# zfs list -t snapshot -r pool1/volume-ec9df818-05ef-4c62-8cf4-e88b253f6e70_00000
NAME USED AVAIL REFER MOUNTPOINT
pool1/volume-ec9df818-05ef-4c62-8cf4-e88b253f6e70_00000@snapshot-6f10312e-d801-440d-8881-c009c7c025d1 84K - 587M -
pool1/volume-ec9df818-05ef-4c62-8cf4-e88b253f6e70_00000@snapshot-8017db1f-5673-4458-b54f-935699459510 0B - 587M -
pool1/volume-ec9df818-05ef-4c62-8cf4-e88b253f6e70_00000@snapshot-dfb63cf0-3bba-432e-a51d-7fe250d5a156 0B - 587M -
pool1/volume-ec9df818-05ef-4c62-8cf4-e88b253f6e70_00000@snapshot-482883d9-1ef9-4634-865b-83382d83e8d8 0B - 587M -
pool1/volume-ec9df818-05ef-4c62-8cf4-e88b253f6e70_00000@snapshot-944f599f-fd20-4af2-ab5b-b77b2afad088 0B - 587M -
pool1/volume-ec9df818-05ef-4c62-8cf4-e88b253f6e70_00000@snapshot-48eadc03-39be-412a-99b6-b3d5a00fe812 0B - 587M -
pool1/volume-ec9df818-05ef-4c62-8cf4-e88b253f6e70_00000@snapshot-3fa4ca5e-5364-47ad-9b84-9dfc0e9ba259 0B - 587M -
pool1/volume-ec9df818-05ef-4c62-8cf4-e88b253f6e70_00000@snapshot-42e31b5e-609d-4cf9-aec4-4bf8418c703b 0B - 587M -
pool1/volume-ec9df818-05ef-4c62-8cf4-e88b253f6e70_00000@snapshot-8f890cb0-54e8-4679-895b-95169e748b18 0B - 587M -
pool1/volume-ec9df818-05ef-4c62-8cf4-e88b253f6e70_00000@snapshot-db970417-383c-4006-8922-04c19ee5d191 0B - 587M -
pool1/volume-ec9df818-05ef-4c62-8cf4-e88b253f6e70_00000@snapshot-2906b6be-244a-4e39-bf6a-712781998b7c 0B - 587M -
pool1/volume-ec9df818-05ef-4c62-8cf4-e88b253f6e70_00000@snapshot-3153dc5d-1871-495c-b4fa-01486a0eae9d 0B - 587M -
pool1/volume-ec9df818-05ef-4c62-8cf4-e88b253f6e70_00000@snapshot-dcdecc65-3d71-42f1-830d-ee6d0fbf6f56 0B - 587M -
pool1/volume-ec9df818-05ef-4c62-8cf4-e88b253f6e70_00000@snapshot-0815aceb-80ef-4ed1-a0b6-ff85ef64006b 0B - 587M -
pool1/volume-ec9df818-05ef-4c62-8cf4-e88b253f6e70_00000@snapshot-872d6992-ed98-4b85-8112-f5c6ad0a38c2 0B - 587M -
pool1/volume-ec9df818-05ef-4c62-8cf4-e88b253f6e70_00000@snapshot-c0d8d781-7625-4459-b40c-e5706b7afbcb 0B - 587M -
pool1/volume-ec9df818-05ef-4c62-8cf4-e88b253f6e70_00000@snapshot-5aa48aa1-5668-4627-b8cd-aca0373f00d4 0B - 587M -
pool1/volume-ec9df818-05ef-4c62-8cf4-e88b253f6e70_00000@snapshot-479bca62-af74-43bb-9790-b8728b926aae 0B - 587M -
pool1/volume-ec9df818-05ef-4c62-8cf4-e88b253f6e70_00000@snapshot-bcdb6a6c-fe51-4d68-814d-3d4157205a7e 0B - 587M -
pool1/volume-ec9df818-05ef-4c62-8cf4-e88b253f6e70_00000@snapshot-85e12d41-ea3c-4562-8646-96eac5bc2705 0B - 587M -
pool1/volume-ec9df818-05ef-4c62-8cf4-e88b253f6e70_00000@snapshot-430c9be2-1ac0-4454-829a-89641884f3e2 0B - 587M -
pool1/volume-ec9df818-05ef-4c62-8cf4-e88b253f6e70_00000@snapshot-6af91e9e-3e58-4f5d-9c08-f5bf8296b77c 0B - 587M -
pool1/volume-ec9df818-05ef-4c62-8cf4-e88b253f6e70_00000@snapshot-830beaa3-ce32-4f08-83fa-6206e088cfe6 0B - 587M -
pool1/volume-ec9df818-05ef-4c62-8cf4-e88b253f6e70_00000@snapshot-9c6cbb9a-f214-47f9-9e42-67ca2419c368 0B - 587M -
pool1/volume-ec9df818-05ef-4c62-8cf4-e88b253f6e70_00000@snapshot-fa9d39c7-f8f7-475b-99ff-a0131ab6d0e9 0B - 587M -
pool1/volume-ec9df818-05ef-4c62-8cf4-e88b253f6e70_00000@snapshot-1600db26-958a-4a7a-964b-c5aea67cc2a5 0B - 587M -
pool1/volume-ec9df818-05ef-4c62-8cf4-e88b253f6e70_00000@snapshot-cf6b06c3-fdd4-4d72-9677-ab1f0008a568 0B - 587M -
pool1/volume-ec9df818-05ef-4c62-8cf4-e88b253f6e70_00000@snapshot-0ebdfff3-d3e4-478e-a951-70ede591ba67 0B - 587M -
pool1/volume-ec9df818-05ef-4c62-8cf4-e88b253f6e70_00000@snapshot-a08a8d66-4691-4a10-85ab-74b0fc556fff 0B - 587M -
pool1/volume-ec9df818-05ef-4c62-8cf4-e88b253f6e70_00000@snapshot-9f3d2c0b-1f79-4501-ad09-9696e388fccb 0B - 587M -
pool1/volume-ec9df818-05ef-4c62-8cf4-e88b253f6e70_00000@snapshot-0c5bbfe2-e512-487b-bc65-b4eec49dab85 0B - 587M -
pool1/volume-ec9df818-05ef-4c62-8cf4-e88b253f6e70_00000@snapshot-2f7830b8-91ef-403c-bc3d-9d8e7a3a3e69 0B - 587M -
pool1/volume-ec9df818-05ef-4c62-8cf4-e88b253f6e70_00000@snapshot-7db0ca79-4df0-48c9-b01c-8c9e9a9b9499 0B - 587M -
pool1/volume-ec9df818-05ef-4c62-8cf4-e88b253f6e70_00000@snapshot-65d270d9-3f38-4652-9612-9b5df13cf016 0B - 587M -
pool1/volume-ec9df818-05ef-4c62-8cf4-e88b253f6e70_00000@snapshot-e4e678da-ed59-4012-8560-13f51b477faa 0B - 587M -
pool1/volume-ec9df818-05ef-4c62-8cf4-e88b253f6e70_00000@snapshot-52175a57-e01a-4c03-ab73-e1c33d8a78dd 0B - 587M -
pool1/volume-ec9df818-05ef-4c62-8cf4-e88b253f6e70_00000@snapshot-91b7273f-8436-4c47-b111-49b970b30ddc 0B - 587M -
pool1/volume-ec9df818-05ef-4c62-8cf4-e88b253f6e70_00000@snapshot-7bcb4195-c630-4262-8260-546dfd68cc0f 0B - 587M -
pool1/volume-ec9df818-05ef-4c62-8cf4-e88b253f6e70_00000@snapshot-50e389b3-e493-4b65-96ef-4182547a7154 0B - 587M -
pool1/volume-ec9df818-05ef-4c62-8cf4-e88b253f6e70_00000@snapshot-cbaafdc3-c8c9-4eeb-9917-8446453e9ce1 0B - 587M -
pool1/volume-ec9df818-05ef-4c62-8cf4-e88b253f6e70_00000@snapshot-440e5a6a-5a8a-43ed-b44e-0f0b3eb3f542 0B - 587M -
pool1/volume-ec9df818-05ef-4c62-8cf4-e88b253f6e70_00000@snapshot-6d65d575-49d2-4f18-84a2-976d7ef25969 0B - 587M -