From 9646e76dda2341d1cdd3796690a1f1861ebcb1bd Mon Sep 17 00:00:00 2001 From: "Jan Alexander Steffens (heftig)" Date: Mon, 27 Jan 2020 18:16:20 +0100 Subject: [PATCH] ZEN: INTERACTIVE: Increase block layer queue depth to 512 4.9: In a surprising turn of events, while benchmarking and testing hierarchical scheduling with BFQ + writeback throttling, it turns out that raising the number of requests in queue _actually_ improves responsiveness and completely eliminates the random stalls that would normally occur without hierarchical scheduling. To make this test more intense, I used the following test: Rotational disk1: rsync -a /source/of/data /target/to/disk1 Rotational disk2: rsync -a /source/of/data /target/to/disk2 And periodically attempted to write super fast with: dd if=/dev/zero of=/target/to/disk1/block bs=4096 This wrote 10gb incredibly fast to writeback and I encountered zero stalls through this entire test of 10-15 minutes. My suspicion is that with cgroups, BFQ is more able to properly sort among multiple drives, reducing the chance of a starved process. This plus writeback throttling completely eliminate any outstanding bugs with high writeback ratios, letting the user enjoy low latency writes (application thinks they're already done), and super high throughput due to batched writes in writeback. Please note however, without the following configuration, I cannot guarantee you will not get stalls: CONFIG_BLK_CGROUP=y CONFIG_CGROUP_WRITEBACK=y CONFIG_IOSCHED_CFQ=y CONFIG_CFQ_GROUP_IOSCHED=y CONFIG_IOSCHED_BFQ=y CONFIG_BFQ_GROUP_IOSCHED=y CONFIG_DEFAULT_BFQ=y CONFIG_SCSI_MQ_DEFAULT=n Special thanks to h2, author of smxi and inxi, for providing evidence that a configuration specific to Debian did not cause stalls found the Liquorix kernels under heavy IO load. This specific configuration turned out to be hierarchical scheduling on CFQ (thus, BFQ as well). --- include/linux/blkdev.h | 4 ++++ init/Kconfig | 1 + 2 files changed, 5 insertions(+) diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index 4c636c42ad6864..eb83d68d76ebb0 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -45,7 +45,11 @@ struct blk_queue_stats; struct blk_stat_callback; #define BLKDEV_MIN_RQ 4 +#ifdef CONFIG_ZEN_INTERACTIVE +#define BLKDEV_MAX_RQ 512 +#else #define BLKDEV_MAX_RQ 128 /* Default maximum */ +#endif /* Must be consistent with blk_mq_poll_stats_bkt() */ #define BLK_MQ_POLL_STATS_BKTS 16 diff --git a/init/Kconfig b/init/Kconfig index c979c6198b1853..69796872c396fd 100644 --- a/init/Kconfig +++ b/init/Kconfig @@ -82,6 +82,7 @@ config ZEN_INTERACTIVE --- Block Layer ---------------------------------------- Default scheduler.........: mq-deadline -> bfq + Queue depth...............: 128 -> 512 config BROKEN bool