Please sign in to comment.
futexes: Avoid taking the hb->lock if there's nothing to wake up
In futex_wake() there is clearly no point in taking the hb->lock if we know beforehand that there are no tasks to be woken. While the hash bucket's plist head is a cheap way of knowing this, we cannot rely 100% on it as there is a racy window between the futex_wait call and when the task is actually added to the plist. To this end, we couple it with the spinlock check as tasks trying to enter the critical region are most likely potential waiters that will be added to the plist, thus preventing tasks sleeping forever if wakers don't acknowledge all possible waiters. Furthermore, the futex ordering guarantees are preserved, ensuring that waiters either observe the changed user space value before blocking or is woken by a concurrent waker. For wakers, this is done by relying on the barriers in get_futex_key_refs() -- for archs that do not have implicit mb in atomic_inc(), we explicitly add them through a new futex_get_mm function. For waiters we rely on the fact that spin_lock calls already update the head counter, so spinners are visible even if the lock hasn't been acquired yet. For more details please refer to the updated comments in the code and related discussion: https://lkml.org/lkml/2013/11/26/556 Special thanks to tglx for careful review and feedback. Suggested-by: Linus Torvalds <email@example.com> Reviewed-by: Darren Hart <firstname.lastname@example.org> Reviewed-by: Thomas Gleixner <email@example.com> Reviewed-by: Peter Zijlstra <firstname.lastname@example.org> Signed-off-by: Davidlohr Bueso <email@example.com> Cc: Paul E. McKenney <firstname.lastname@example.org> Cc: Mike Galbraith <email@example.com> Cc: Jeff Mahoney <firstname.lastname@example.org> Cc: Scott Norton <email@example.com> Cc: Tom Vaden <firstname.lastname@example.org> Cc: Aswin Chandramouleeswaran <email@example.com> Cc: Waiman Long <Waiman.Long@hp.com> Cc: Jason Low <firstname.lastname@example.org> Cc: Andrew Morton <email@example.com> Link: http://firstname.lastname@example.org Signed-off-by: Ingo Molnar <email@example.com>
- Loading branch information...