prune bookmarks ?

[devZer0]

as syncoid may use bookmarks for transfer, they pile up on the source host after a while and for example show up when doing "zfs get compression" and look exactly like a snapshot there. this may confuse users who don't know about bookmarks.

what about adding a prune config option to sanoid to also prune bookmarks, if you don't want tens of thousands of them? yes, you want to keep those, but you don't want tens of thousands to pile up until something blows/overflows

also see #618 and #544

[zig]

Could we imagine this strategy, when using --no-sync-snap, maybe a new option --create-per-destination-bookmark :

After a successful send, create on the source one bookmark which name derives from the destination. It will simply point to the latest sent snapshot to that destination. If the bookmark already exists, destroy it before creating it again (we're moving the bookmark).

The option --create-bookmark creates a bookmark with same name as the snapshot, but I find this not very useful, because it will accumulate and also because we don't know which ones can be deleted when there are several destinations.

[devZer0]

i have created a prune script which seems to work good for me.

you can pass the number of days to keep as a param. value it must be greater then some $numdayssafe which is set inside the script, so nobody would delete too many bookmarks accidentally

[rpool/vms-files-zstd]
        use_template = prod
        pruning_script = /root/prune-bookmarks.sh 30

[ssdpool2/vms-files-zstd]
        use_template = prod
        pruning_script = /root/prune-bookmarks.sh 15

[template_prod]
        frequently = 16
        hourly = 24
        daily = 1
        monthly = 0
        yearly = 0
        autosnap = yes
        autoprune = yes
        
        
# cat /root/prune-bookmarks.sh
#!/bin/bash

echo $(date) >>/tmp/prune-bookmarks.log

numdays=$1
numdayssafe=30

# test if number
if !  [[ $numdays =~ ^-?[0-9]+$ ]]
then
echo "not a number"
exit 1
fi

# test if greater minimum days to keep
if [ "$numdays" -lt $numdayssafe ]; then
echo "smaller then $numdayssafe"
exit 1
fi

# derived from https://www.jan0sch.de/post/deleting-old-zfs-snapshots/

# Get EPOCH timestamp from $numdays ago.
DATE=$(echo "$(date +"%s") - $numdays * 24 * 60 * 60" | bc)

echo "Deleting bookmarks older than $numdays days..." >>/tmp/prune-bookmarks.log

echo "target:" $SANOID_TARGET >>/tmp/prune-bookmarks.log

zfs list -p -S creation -o name,creation -t bookmark $SANOID_TARGET | awk -v DATE=$DATE '$2<DATE {print $1}' | while read bookmark;do echo $bookmark;zfs destroy -vp $bookmark;done >>/tmp/prune-bookmarks.log

[devZer0]

root@pve1:~# zfs get compression|grep "@"|wc -l
124
root@pve1:~# zfs get compression|grep "#"|wc -l
25005

[devZer0]

i'm getting a little bit nervous....but i dislike the extra prune-script approach.

could we please have a bookmark prune feature ?

# zfs list -t bookmark| wc -l
47261

[beren12]

There is no real need other than to make you feel warm and fuzzy. Bookmarks take up almost 0 space.

[devZer0]

did you ever try to delete 40k zfs booksmarks @beren12 ?

a bookmark is an object, and every object takes ressources and runtime.

# zfs get all|wc -l
219593

[jsddsfoh]

i have created a prune script which seems to work good for me.

you can pass the number of days to keep as a param. value it must be greater then some $numdayssafe which is set inside the script, so nobody would delete too many bookmarks accidentally

Do I understand correctly, this deletes bookmarks older than x days, but when the newest bookmark is older than x days, it will get deleted and there are no bookmarks at all left?

[jsddsfoh]

just tried by manually increasing date, yes it does.

[jsddsfoh]

I edited this line, which now leaves additional 25 bookmarks. Feel free to change to any number you like.
awk ouputs bookmarks older than x days, beginning with the newest
sed cuts the top 25 from awk output

zfs list -p -S creation -o name,creation -t bookmark $SANOID_TARGET | awk -v DATE=$DATE '$2<DATE {print $1}' | sed -e '1,25d' | while read bookmark;do echo $bookmark;zfs destroy -vp $bookmark;done >>/tmp/prune-bookmarks.log