Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Signal to be delivered on the correct thread. #9

Merged
merged 24 commits into from
Sep 15, 2020

Conversation

KenGordon
Copy link
Collaborator

@KenGordon KenGordon commented Aug 28, 2020

Previously all pending signals were delivered by whichever thread noticed them. In the preemptive kernel this was ok. However in lkl this means that signals were often sent on the wrong thread.

The solution is to split the capture of the signal from the sending. They now get captured to a list hanging off the appropriate task structure and then sent by the task itself at the next opportunity.

This moves the sending of usual signals (not shutdown) outside of the cpu owning code path. Previously a signal handler that caused a task switch would trigger an lkl_bug when the lock count went above 1.

There is a new test to go into sgx-lkl/tests/basic/async_signals as part of a coming PR for that repo.

pending signals whenever they are encountered to instead placing them on
a per task list and then delivering them to the appropriate task at the
next opportunity.

Also move the signal delivery out of the code which holds the cpu lock.
Otherwise if the signal handler causes a context switch (eg printf
blocks) it tries to get the lock again and crashes.

This work is part of sgx-lkl issue 709, ex 644 and perhaps 209.
@KenGordon KenGordon changed the title Signal to delivered on the correct thread. Signal to be delivered on the correct thread. Aug 28, 2020
@davidchisnall
Copy link

Is it possible to add an LKL test suite test for this?

arch/lkl/include/asm/signal.h Outdated Show resolved Hide resolved
arch/lkl/include/asm/signal.h Outdated Show resolved Hide resolved
arch/lkl/include/asm/thread_info.h Outdated Show resolved Hide resolved
arch/lkl/include/asm/thread_info.h Outdated Show resolved Hide resolved
arch/lkl/include/asm/thread_info.h Show resolved Hide resolved
arch/lkl/kernel/signal.c Outdated Show resolved Hide resolved
arch/lkl/kernel/signal.c Outdated Show resolved Hide resolved
arch/lkl/kernel/signal.c Show resolved Hide resolved
arch/lkl/kernel/syscalls.c Outdated Show resolved Hide resolved
arch/lkl/kernel/traps.c Show resolved Hide resolved
@SeanTAllen
Copy link

This is some reasonably complicated stuff. Is there somewhere we can document how it all works? At the moment, there's a few comments spread out, but no larger overview. I think a larger overview would be helpful both for this review and for anyone who has to work on this in the future.

prp pushed a commit that referenced this pull request Sep 7, 2020
[ Upstream commit e24c644 ]

I compiled with AddressSanitizer and I had these memory leaks while I
was using the tep_parse_format function:

    Direct leak of 28 byte(s) in 4 object(s) allocated from:
        #0 0x7fb07db49ffe in __interceptor_realloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10dffe)
        #1 0x7fb07a724228 in extend_token /home/pduplessis/repo/linux/tools/lib/traceevent/event-parse.c:985
        #2 0x7fb07a724c21 in __read_token /home/pduplessis/repo/linux/tools/lib/traceevent/event-parse.c:1140
        #3 0x7fb07a724f78 in read_token /home/pduplessis/repo/linux/tools/lib/traceevent/event-parse.c:1206
        #4 0x7fb07a725191 in __read_expect_type /home/pduplessis/repo/linux/tools/lib/traceevent/event-parse.c:1291
        #5 0x7fb07a7251df in read_expect_type /home/pduplessis/repo/linux/tools/lib/traceevent/event-parse.c:1299
        #6 0x7fb07a72e6c8 in process_dynamic_array_len /home/pduplessis/repo/linux/tools/lib/traceevent/event-parse.c:2849
        #7 0x7fb07a7304b8 in process_function /home/pduplessis/repo/linux/tools/lib/traceevent/event-parse.c:3161
        #8 0x7fb07a730900 in process_arg_token /home/pduplessis/repo/linux/tools/lib/traceevent/event-parse.c:3207
        #9 0x7fb07a727c0b in process_arg /home/pduplessis/repo/linux/tools/lib/traceevent/event-parse.c:1786
        #10 0x7fb07a731080 in event_read_print_args /home/pduplessis/repo/linux/tools/lib/traceevent/event-parse.c:3285
        lkl#11 0x7fb07a731722 in event_read_print /home/pduplessis/repo/linux/tools/lib/traceevent/event-parse.c:3369
        lkl#12 0x7fb07a740054 in __tep_parse_format /home/pduplessis/repo/linux/tools/lib/traceevent/event-parse.c:6335
        lkl#13 0x7fb07a74047a in __parse_event /home/pduplessis/repo/linux/tools/lib/traceevent/event-parse.c:6389
        lkl#14 0x7fb07a740536 in tep_parse_format /home/pduplessis/repo/linux/tools/lib/traceevent/event-parse.c:6431
        lkl#15 0x7fb07a785acf in parse_event ../../../src/fs-src/fs.c:251
        lkl#16 0x7fb07a785ccd in parse_systems ../../../src/fs-src/fs.c:284
        lkl#17 0x7fb07a786fb3 in read_metadata ../../../src/fs-src/fs.c:593
        lkl#18 0x7fb07a78760e in ftrace_fs_source_init ../../../src/fs-src/fs.c:727
        lkl#19 0x7fb07d90c19c in add_component_with_init_method_data ../../../../src/lib/graph/graph.c:1048
        lkl#20 0x7fb07d90c87b in add_source_component_with_initialize_method_data ../../../../src/lib/graph/graph.c:1127
        lkl#21 0x7fb07d90c92a in bt_graph_add_source_component ../../../../src/lib/graph/graph.c:1152
        lkl#22 0x55db11aa632e in cmd_run_ctx_create_components_from_config_components ../../../src/cli/babeltrace2.c:2252
        lkl#23 0x55db11aa6fda in cmd_run_ctx_create_components ../../../src/cli/babeltrace2.c:2347
        lkl#24 0x55db11aa780c in cmd_run ../../../src/cli/babeltrace2.c:2461
        lkl#25 0x55db11aa8a7d in main ../../../src/cli/babeltrace2.c:2673
        lkl#26 0x7fb07d5460b2 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x270b2)

The token variable in the process_dynamic_array_len function is
allocated in the read_expect_type function, but is not freed before
calling the read_token function.

Free the token variable before calling read_token in order to plug the
leak.

Signed-off-by: Philippe Duplessis-Guindon <pduplessis@efficios.com>
Reviewed-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
Link: https://lore.kernel.org/linux-trace-devel/20200730150236.5392-1-pduplessis@efficios.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>
arch/lkl/configs/defconfig Outdated Show resolved Hide resolved
arch/lkl/include/asm/cpu.h Outdated Show resolved Hide resolved
arch/lkl/include/asm/cpu.h Outdated Show resolved Hide resolved
arch/lkl/include/asm/signal.h Outdated Show resolved Hide resolved
arch/lkl/include/asm/signal.h Outdated Show resolved Hide resolved
arch/lkl/kernel/syscalls.c Outdated Show resolved Hide resolved
arch/lkl/kernel/syscalls.c Outdated Show resolved Hide resolved
arch/lkl/kernel/threads.c Outdated Show resolved Hide resolved
net/Kconfig Outdated Show resolved Hide resolved
net/Makefile Outdated Show resolved Hide resolved
@vtikoo vtikoo requested a review from mikbras September 10, 2020 16:29
arch/lkl/kernel/signal.c Outdated Show resolved Hide resolved
KenGordon and others added 3 commits September 14, 2020 09:57
Typo in comment.

Co-authored-by: Vikas Amar Tikoo <vtikoo@users.noreply.github.com>
Clarification in comment.

Co-authored-by: Vikas Amar Tikoo <vtikoo@users.noreply.github.com>
Copy link

@mikbras mikbras left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you Ken for doing this work. The approach seems solid. I have a few questions, mainly for my own understanding.

I did notice that the tail does not seem to be updated when appending to the list. This is the only bug I could find.

arch/lkl/kernel/cpu.c Show resolved Hide resolved
arch/lkl/kernel/cpu.c Show resolved Hide resolved
arch/lkl/kernel/cpu.c Show resolved Hide resolved
arch/lkl/kernel/signal.c Show resolved Hide resolved
arch/lkl/kernel/signal.c Outdated Show resolved Hide resolved
arch/lkl/kernel/signal.c Show resolved Hide resolved
arch/lkl/kernel/threads.c Show resolved Hide resolved
Would have been an issue once more that one signal was in the task's
queue.
@KenGordon KenGordon merged commit e041aa7 into upstream-refactor Sep 15, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
5 participants