Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
output: Don't use nbdkit-file-plugin cache=none when writing
nbdkit-file-plugin flag cache=none is meant to ensure that the file plugin does not pollute the page cache. However it doesn't yet work very well, especially for writing. As you can see here: https://gitlab.com/nbdkit/nbdkit/-/blob/048d5b9818c88355e596824355269773e5c4f6ad/plugins/file/file.c#L594 it is currently implemented by flushing the whole file (including parallel requests), and then evicting the data from the page cache. This implementation is naive. (See nbdcopy copy/file-ops.c for a much better implementation.) This causes quite a considerable slow down when writing to a local file (eg -os local). The penalty varies between machines and possibly kernels. On my Fedora Rawhide AMD server with 12 cores and SSDs, the slow down is under 5%. But on my RHEL 9 Intel NUC server with 2 cores and a hard disk, it makes a really huge difference, nearly doubling the total time taken. I left a note in the code that we should fix this in nbdkit and re-add this option or similar when it is working. I also removed the fadvise=sequential option. This is associated with cache=none in the nbdkit documentation, and anyway we are not doing sequential writes. (This makes no measurable difference in my testing)
- Loading branch information