Skip to content

Commit

Permalink
io_uring: optimise wq_has_sleeper's smb_mb()
Browse files Browse the repository at this point in the history
Inside __io_cq_unlock_post(), we unlock a spinlock and then are trying
to wake a CQ waitqueue, for that

Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
  • Loading branch information
isilence committed Jun 19, 2022
1 parent 243c921 commit 6224f58
Showing 1 changed file with 8 additions and 1 deletion.
9 changes: 8 additions & 1 deletion io_uring/io_uring.c
Original file line number Diff line number Diff line change
Expand Up @@ -527,6 +527,12 @@ void __io_commit_cqring_flush(struct io_ring_ctx *ctx)
io_eventfd_signal(ctx);
}

#ifdef __x86_64__
#define smp_mb__after_spin_unlock() do {} while (0)
#else
#define smp_mb__after_spin_unlock() smp_mb()
#endif

static inline void __io_cq_unlock_post(struct io_ring_ctx *ctx)
__releases(ctx->completion_lock)
{
Expand All @@ -535,7 +541,8 @@ static inline void __io_cq_unlock_post(struct io_ring_ctx *ctx)

/* former io_cqring_ev_posted */
io_commit_cqring_flush(ctx);
if (wq_has_sleeper(&ctx->cq_wait))
smp_mb__after_spin_unlock();
if (waitqueue_active(&ctx->cq_wait))
io_cqring_wake(ctx);
}

Expand Down

0 comments on commit 6224f58

Please sign in to comment.