block, bfq: add Early Queue Merge (EQM) to BFQ-v7r8 for 3.0.0

A set of processes may happen  to  perform interleaved reads, i.e.,requests
whose union would give rise to a  sequential read  pattern.  There are two
typical  cases: in the first  case,   processes  read  fixed-size chunks of
data at a fixed distance from each other, while in the second case processes
may read variable-size chunks at  variable distances. The latter case occurs
for  example with  QEMU, which  splits the  I/O generated  by the  guest into
multiple chunks,  and lets these chunks  be served by a  pool of cooperating
processes,  iteratively  assigning  the  next  chunk of  I/O  to  the first
available  process. CFQ  uses actual  queue merging  for the  first type of
rocesses, whereas it  uses preemption to get a sequential  read pattern out
of the read requests  performed by the second type of  processes. In the end
it uses  two different  mechanisms to  achieve the  same goal: boosting the
throughput with interleaved I/O.

This patch introduces  Early Queue Merge (EQM), a unified mechanism to get a
sequential  read pattern  with both  types of  processes. The  main idea is
checking newly arrived requests against the next request of the active queue
both in case of actual request insert and in case of request merge. By doing
so, both the types of processes can be handled by just merging their queues.
EQM is  then simpler and  more compact than the  pair of mechanisms used in
CFQ.

Finally, EQM  also preserves the  typical low-latency properties of BFQ, by
properly restoring the weight-raising state of  a queue when it gets back to
a non-merged state.

Signed-off-by: Mauro Andreolini <mauro.andreolini@unimore.it>
Signed-off-by: Arianna Avanzini <avanzini.arianna@gmail.com>
Signed-off-by: Paolo Valente <paolo.valente@unimore.it>

block/Kconfig.iosched

@@ -63,7 +63,7 @@ config IOSCHED_BFQ
 	  It aims at distributing the bandwidth as desired, independently of
 	  the disk parameters and with any workload. It also tries to
 	  guarantee low latency to interactive and soft real-time
-	  applications.  If compiled built-in (saying Y here), BFQ can
+	  applications. If compiled built-in (saying Y here), BFQ can
 	  be configured to support hierarchical scheduling.
 
 config CGROUP_BFQIO

block/bfq-cgroup.c

@@ -322,11 +322,16 @@ static struct bfq_group *__bfq_cic_change_cgroup(struct bfq_data *bfqd,
 						 struct cfq_io_context *cic,
 						 struct cgroup *cgroup)
 {
-	struct bfq_queue *async_bfqq = cic_to_bfqq(cic, 0);
-	struct bfq_queue *sync_bfqq = cic_to_bfqq(cic, 1);
+	struct bfq_queue *async_bfqq;
+	struct bfq_queue *sync_bfqq;
 	struct bfq_entity *entity;
 	struct bfq_group *bfqg;
 
+	spin_lock(&bfqd->eqm_lock);
+
+	async_bfqq = cic_to_bfqq(cic, 0);
+	sync_bfqq = cic_to_bfqq(cic, 1);
+
 	bfqg = bfq_find_alloc_group(bfqd, cgroup);
 	if (async_bfqq != NULL) {
 		entity = &async_bfqq->entity;
@@ -346,6 +351,8 @@ static struct bfq_group *__bfq_cic_change_cgroup(struct bfq_data *bfqd,
 			bfq_bfqq_move(bfqd, sync_bfqq, entity, bfqg);
 	}
 
+	spin_unlock(&bfqd->eqm_lock);
+
 	return bfqg;
 }
 

block/bfq-ioc.c

@@ -122,10 +122,19 @@ static void __bfq_exit_single_io_context(struct bfq_data *bfqd,
 		cic->cfqq[BLK_RW_ASYNC] = NULL;
 	}
 
+	spin_lock(&bfqd->eqm_lock);
 	if (cic->cfqq[BLK_RW_SYNC] != NULL) {
+		/*
+		 * If the bic is using a shared queue, put the reference
+		 * taken on the io_context when the bic started using a
+		 * shared bfq_queue.
+		 */
+		if (bfq_bfqq_coop(cic->cfqq[BLK_RW_SYNC]))
+			put_io_context(ioc);
 		bfq_exit_bfqq(bfqd, cic->cfqq[BLK_RW_SYNC]);
 		cic->cfqq[BLK_RW_SYNC] = NULL;
 	}
+	spin_unlock(&bfqd->eqm_lock);
 }
 
 /**
@@ -172,6 +181,25 @@ static struct cfq_io_context *bfq_alloc_io_context(struct bfq_data *bfqd,
 							bfqd->queue->node);
 	if (cic != NULL) {
 		cic->last_end_request = jiffies;
+		/*
+		 * A newly created cic indicates that the process has just
+		 * started doing I/O, and is probably mapping into memory its
+		 * executable and libraries: it definitely needs weight raising.
+		 * There is however the possibility that the process performs,
+		 * for a while, I/O close to some other process. EQM intercepts
+		 * this behavior and may merge the queue corresponding to the
+		 * process  with some other queue, BEFORE the weight of the queue
+		 * is raised. Merged queues are not weight-raised (they are assumed
+		 * to belong to processes that benefit only from high throughput).
+		 * If the merge is basically the consequence of an accident, then
+		 * the queue will be split soon and will get back its old weight.
+		 * It is then important to write down somewhere that this queue
+		 * does need weight raising, even if it did not make it to get its
+		 * weight raised before being merged. To this purpose, we overload
+		 * the field raising_time_left and assign 1 to it, to mark the queue
+		 * as needing weight raising.
+		 */
+		cic->wr_time_left = 1;
 		INIT_LIST_HEAD(&cic->queue_list);
 		INIT_HLIST_NODE(&cic->cic_list);
 		cic->dtor = bfq_free_io_context;