Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fallocate plot files in background upon creation #46

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

mocksoul
Copy link
Contributor

@mocksoul mocksoul commented Aug 1, 2021

Once hdd is being filled up to 80%+ allocating 101.5GiB of data can take up to 30-60 extra seconds. That is usually done during plot writing stage (3, 4 and waiting for prev plot in stage 1) by filesystem itself while plot file is being written.

In this patch i'm using fallocate system call and run it in background thread after plot file creation. Linux optimizes this -- fallocate process take lowest IO prio in kernel -- so, it does not affect previous plot writing speeds at all.

This can have different impact on various filesystems, I'm using ext4 with data=writeback.

Same can be archieved by separate fallocate process on zero sized *.plot.tmp files, e.g:
while true; do find /mnt/plots_mountpoint -name '*.plot.tmp' -size 0 -print -exec fallocate -l 101G {} \;; sleep 60; done

Example bladebit output:

Generating F1...
Fallocating files...
Finished F1 generation in 2.64 seconds.
Sorting F1...
Fallocate succeeded in 15.47 secs
Finished F1 sort in 43.67 seconds.

This is newly created filesystem on 18TB disk. On 80%+ filled disks fallocate syscall takes ~1min and happens in separate thread.

@mocksoul mocksoul changed the title Fallocate plot files in background upon creation fallocate plot files in background upon creation Aug 1, 2021
@mocksoul
Copy link
Contributor Author

mocksoul commented Aug 1, 2021

I'm fallocate-ing 101GiB, because avg plot size is around 101.35GiB and smallest I have seen is 101.25GiB (10k+ plots).
It could be possible to allocate more here (e.g. 102GiB), but that will require ftruncate after all writes to plot file done.

But 101GiB works fine for me, so I didnt bothered =).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant