Please sign in to comment.
rwsem: Implement writer lock-stealing for better scalability
Commit 5a50508 ("mm/rmap: Convert the struct anon_vma::mutex to an rwsem") changed struct anon_vma::mutex to an rwsem, which caused aim7 fork_test performance to drop by 50%. Yuanhan Liu did the following excellent analysis: https://lkml.org/lkml/2013/1/29/84 and found that the regression is caused by strict, serialized, FIFO sequential write-ownership of rwsems. Ingo suggested implementing opportunistic lock-stealing for the front writer task in the waitqueue. Yuanhan Liu implemented lock-stealing for spinlock-rwsems, which indeed recovered much of the regression - confirming the analysis that the main factor in the regression was the FIFO writer-fairness of rwsems. In this patch we allow lock-stealing to happen when the first waiter is also writer. With that change in place the aim7 fork_test performance is fully recovered on my Intel NHM EP, NHM EX, SNB EP 2S and 4S test-machines. Reported-by: email@example.com Reported-by: Yuanhan Liu <firstname.lastname@example.org> Signed-off-by: Alex Shi <email@example.com> Cc: David Howells <firstname.lastname@example.org> Cc: Michel Lespinasse <email@example.com> Cc: Linus Torvalds <firstname.lastname@example.org> Cc: Andrew Morton <email@example.com> Cc: Peter Zijlstra <firstname.lastname@example.org> Cc: Anton Blanchard <email@example.com> Cc: Arjan van de Ven <firstname.lastname@example.org> Cc: email@example.com Link: https://lkml.org/lkml/2013/1/29/84 Link: http://firstname.lastname@example.org [ Small stylistic fixes, updated changelog. ] Signed-off-by: Ingo Molnar <email@example.com>
- Loading branch information...