Skip to content

Commit 114276a

Browse files
mstsirkinIngo Molnar
authored andcommitted
mm, sched: Drop voluntary schedule from might_fault()
might_fault() is called from functions like copy_to_user() which most callers expect to be very fast, like a couple of instructions. So functions like memcpy_toiovec() call them many times in a loop. But might_fault() calls might_sleep() and with CONFIG_PREEMPT_VOLUNTARY this results in a function call. Let's not do this - just call __might_sleep() that produces a diagnostic for sleep within atomic, but drop might_preempt(). Here's a test sending traffic between the VM and the host, host is built with CONFIG_PREEMPT_VOLUNTARY: before: incoming: 7122.77 Mb/s outgoing: 8480.37 Mb/s after: incoming: 8619.24 Mb/s outgoing: 9455.42 Mb/s As a side effect, this fixes an issue pointed out by Ingo: might_fault might schedule differently depending on PROVE_LOCKING. Now there's no preemption point in both cases, so it's consistent. Signed-off-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Peter Zijlstra <peterz@infradead.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Link: http://lkml.kernel.org/r/1369577426-26721-10-git-send-email-mst@redhat.com Signed-off-by: Ingo Molnar <mingo@kernel.org>
1 parent 016be2e commit 114276a

File tree

2 files changed

+3
-2
lines changed

2 files changed

+3
-2
lines changed

include/linux/kernel.h

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -198,7 +198,7 @@ void might_fault(void);
198198
#else
199199
static inline void might_fault(void)
200200
{
201-
might_sleep();
201+
__might_sleep(__FILE__, __LINE__, 0);
202202
}
203203
#endif
204204

mm/memory.c

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4222,7 +4222,8 @@ void might_fault(void)
42224222
if (segment_eq(get_fs(), KERNEL_DS))
42234223
return;
42244224

4225-
might_sleep();
4225+
__might_sleep(__FILE__, __LINE__, 0);
4226+
42264227
/*
42274228
* it would be nicer only to annotate paths which are not under
42284229
* pagefault_disable, however that requires a larger audit and

0 commit comments

Comments
 (0)