Skip to content

Commit

Permalink
increase zfs send prefetching (openzfs#456)
Browse files Browse the repository at this point in the history
When `zfs send` has to retrieve the data from the object store (as
opposed to zettacache), we can benefit from a larger prefetch queue
depth, similar to what we see for predictive prefetching.  This is
because we need a high GetObject queue depth (dozens of requests,
covering dozends of MB's of data) to get good throughput from S3.

This commit increases the zfs send prefetching to 64MB (from 16MB), and
clarifies the comment around it (there's really no downside to
increasing it aside from RAM usage, since this is not using ARC
prefetching any more).
  • Loading branch information
ahrens committed Jun 1, 2022
1 parent 3075380 commit d591138
Showing 1 changed file with 4 additions and 6 deletions.
10 changes: 4 additions & 6 deletions module/zfs/dmu_send.c
Expand Up @@ -69,13 +69,11 @@
/* Set this tunable to TRUE to replace corrupt data with 0x2f5baddb10c */
static int zfs_send_corrupt_data = B_FALSE;
/*
* This tunable controls the amount of data (measured in bytes) that will be
* prefetched by zfs send. If the main thread is blocking on reads that haven't
* completed, this variable might need to be increased. If instead the main
* thread is issuing new reads because the prefetches have fallen out of the
* cache, this may need to be decreased.
* This tunable controls the amount of memory (measured in bytes) that will be
* used to buffer data read for zfs send. If the main thread is blocking on
* reads that haven't completed, this variable might need to be increased.
*/
static int zfs_send_queue_length = SPA_MAXBLOCKSIZE;
static int zfs_send_queue_length = 64 * 1024 * 1024;
/*
* This tunable controls the length of the queues that zfs send worker threads
* use to communicate. If the send_main_thread is blocking on these queues,
Expand Down

0 comments on commit d591138

Please sign in to comment.