Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
MDEV-11039 - Add new scheduling algorithm for reducing tail latencies (for 10.2) #248
This branch introduces a new scheduling algorithm (Variance-Aware-Transaction-Scheduling, VATS) for the record lock manager of InnoDB based on MariaDB 10.2. Instead of using First-Come-First-Served (FCFS), the newly introduced algorithm prefers the eldest transaction. A configuration parameter (innodb_lock_schedule_algorithm) is introduced for users to choose between VATS and FCFS (the default one). We've extensively tested this algorithm in many workloads. The algorithm is very simple, and the changes are very local, but it significantly improves performance (in terms of average latency and throughput) and predictability (in terms of reduction of tail and quantile latencies) For more details, please refer to this paper http://arxiv.org/abs/1602.01871
Changes look correct and fine. There is small differences on used style and normal InnoDB style because InnoDB style uses tab for indention not spaces. Could you please fix the indention before I will merge this. Currently on 10.2 there is no xtradb.
Thanks for your contribution. These changes were originally proposed for 10.1 in PR#245, Normally there's no need to create separate pull request for 10.2, because it will be merged from 10.1 anyway.
But I guess target version of this PR is not decided currently, so I keep both PR's open.
@svoj Hi, Sergey. I created a separate pull request for 10.2 because the locking system has undergone some significant code refactor in it, so the pull request for 10.1 would not be compatible with 10.2. Also from our information it would be easier for you to accept new features in version 10.2.
This patch contains several fixes that hasn't been adopted in the previous patch yet. Do you want me to update that patch as well?
pushed a commit
this pull request
Nov 10, 2017
In MySQL 8.0, I found the following bug fixes related to this:
It looks like MySQL 8.0 employs a threshold based on
I would appreciate it if you could comment how the implementations of the algorithm differ between MariaDB 10.2 and MySQL 8.0.
The algorithms we merged into MariaDB is actually different from what we merged into MySQL 8.0. They are based on two of our papers (VATS, CATS). VATS grants locks to transactions according to their age, while CATS does so according to each transaction's dependency set size (the number of transactions being blocked on it). Therefore, CATS's implementation is much more complicated because it also needs to maintain each transaction's dependency set size.
For VATS, the idea is very simple: it inserts transactions to the queue according to their ages so that they are granted in this order. An additional thing I did was to keep the granted locks at the front of the queue.
Is there a specific test case that I can use to reproduce the problem? I will look into this problem as well.