Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[crash] Invalid objects returned from my module function #2

Closed
sebastiencs opened this issue Apr 12, 2018 · 17 comments

Comments

Projects
None yet
4 participants
@sebastiencs
Copy link

commented Apr 12, 2018

Hi,

I'm using emacs-module-rs in some of my projects. I just made a small module that convert a json string to a lisp object.
You can see the code here, it's very small.

When a big json object is returned, it makes emacs crash. It doesn't happen with small objects.
It seems that the lisp object is not valid:

Stack traces


0  in terminate_due_to_signal of ../../src/emacs.c:369
1  in die of ../../src/alloc.c:7434
2  in XSTRING of ../../src/lisp.h:1395
3  in SDATA of ../../src/lisp.h:1446
4  in print_object of ../../src/print.c:1930
5  in print_vectorlike of ../../src/print.c:1713
6  in print_object of ../../src/print.c:2104
7  in print of ../../src/print.c:1150
8  in Fprin1 of ../../src/print.c:648
9  in print_error_message of ../../src/print.c:964
10 in Fcommand_error_default_function of ../../src/keyboard.c:1049
11 in funcall_subr of ../../src/eval.c:2901
12 in Ffuncall of ../../src/eval.c:2821
13 in call3 of ../../src/eval.c:2688
14 in cmd_error_internal of ../../src/keyboard.c:1004
15 in read_process_output_error_handler of ../../src/process.c:5803
16 in internal_condition_case_1 of ../../src/eval.c:1352
17 in read_and_dispose_of_process_output of ../../src/process.c:6008
18 in read_process_output of ../../src/process.c:5919
19 in wait_reading_process_output of ../../src/process.c:5616
20 in kbd_buffer_get_event of ../../src/keyboard.c:3822
21 in read_event_from_main_queue of ../../src/keyboard.c:2157
22 in read_decoded_event_from_main_queue of ../../src/keyboard.c:2221
23 in read_char of ../../src/keyboard.c:2808
24 in read_filtered_event of ../../src/lread.c:674
25 in Fread_event of ../../src/lread.c:786
26 in funcall_subr of ../../src/eval.c:2901
27 in Ffuncall of ../../src/eval.c:2821
28 in exec_byte_code of ../../src/bytecode.c:632
29 in funcall_lambda of ../../src/eval.c:3022
30 in Ffuncall of ../../src/eval.c:2823
31 in exec_byte_code of ../../src/bytecode.c:632
32 in funcall_lambda of ../../src/eval.c:3022
33 in Ffuncall of ../../src/eval.c:2823
34 in exec_byte_code of ../../src/bytecode.c:632
35 in funcall_lambda of ../../src/eval.c:3022
36 in Ffuncall of ../../src/eval.c:2823
37 in exec_byte_code of ../../src/bytecode.c:632
38 in funcall_lambda of ../../src/eval.c:3022
39 in Ffuncall of ../../src/eval.c:2823
40 in exec_byte_code of ../../src/bytecode.c:632
41 in funcall_lambda of ../../src/eval.c:3022
42 in Ffuncall of ../../src/eval.c:2823
43 in exec_byte_code of ../../src/bytecode.c:632
44 in funcall_lambda of ../../src/eval.c:3022
45 in Ffuncall of ../../src/eval.c:2823
46 in exec_byte_code of ../../src/bytecode.c:632
47 in funcall_lambda of ../../src/eval.c:3022
48 in Ffuncall of ../../src/eval.c:2823
49 in Fapply of ../../src/eval.c:2441
50 in funcall_subr of ../../src/eval.c:2876
51 in Ffuncall of ../../src/eval.c:2821
52 in exec_byte_code of ../../src/bytecode.c:632
53 in funcall_lambda of ../../src/eval.c:3022
54 in Ffuncall of ../../src/eval.c:2823
55 in call1 of ../../src/eval.c:2672
56 in timer_check_2 of ../../src/keyboard.c:4333
57 in timer_check of ../../src/keyboard.c:4395
58 in readable_events of ../../src/keyboard.c:3352
59 in get_input_pending of ../../src/keyboard.c:6808
60 in swallow_events of ../../src/keyboard.c:4135
61 in sit_for of ../../src/dispnew.c:5783
62 in read_char of ../../src/keyboard.c:2723
63 in read_key_sequence of ../../src/keyboard.c:9150
64 in command_loop_1 of ../../src/keyboard.c:1370
65 in internal_condition_case of ../../src/eval.c:1332
66 in command_loop_2 of ../../src/keyboard.c:1111
67 in internal_catch of ../../src/eval.c:1097
68 in command_loop of ../../src/keyboard.c:1090
69 in recursive_edit_1 of ../../src/keyboard.c:696
70 in Frecursive_edit of ../../src/keyboard.c:767
71 in main of ../../src/emacs.c:1720

0  in terminate_due_to_signal of ../../src/emacs.c:369
1  in emacs_abort of ../../src/sysdep.c:2426
2  in print_vectorlike of ../../src/print.c:1759
3  in print_object of ../../src/print.c:2104
4  in print of ../../src/print.c:1150
5  in Fprin1 of ../../src/print.c:648
6  in print_error_message of ../../src/print.c:964
7  in Fcommand_error_default_function of ../../src/keyboard.c:1049
8  in funcall_subr of ../../src/eval.c:2901
9  in Ffuncall of ../../src/eval.c:2821
10 in call3 of ../../src/eval.c:2688
11 in cmd_error_internal of ../../src/keyboard.c:1004
12 in read_process_output_error_handler of ../../src/process.c:5803
13 in internal_condition_case_1 of ../../src/eval.c:1352
14 in read_and_dispose_of_process_output of ../../src/process.c:6008
15 in read_process_output of ../../src/process.c:5919
16 in wait_reading_process_output of ../../src/process.c:5616
17 in sit_for of ../../src/dispnew.c:5824
18 in read_char of ../../src/keyboard.c:2723
19 in read_key_sequence of ../../src/keyboard.c:9150
20 in command_loop_1 of ../../src/keyboard.c:1370
21 in internal_condition_case of ../../src/eval.c:1332
22 in command_loop_2 of ../../src/keyboard.c:1111
23 in internal_catch of ../../src/eval.c:1097
24 in command_loop of ../../src/keyboard.c:1090
25 in recursive_edit_1 of ../../src/keyboard.c:696
26 in Frecursive_edit of ../../src/keyboard.c:767
27 in main of ../../src/emacs.c:1720


Am I doing something wrong in my code ?
Is it the conversion from rust to emacs that fails somewhere ?
Or do you think it's a bug in emacs ?

Note that git-tube also makes crash my emacs when the buffer contains lots of diffs

@ubolonton

This comment has been minimized.

Copy link
Owner

commented Apr 13, 2018

Interesting. I'll check it out.

@sebastiencs

This comment has been minimized.

Copy link
Author

commented Apr 13, 2018

Thanks !

@ubolonton

This comment has been minimized.

Copy link
Owner

commented Apr 13, 2018

Can you upload an input string that causes the crash somewhere?

@ubolonton

This comment has been minimized.

Copy link
Owner

commented Apr 14, 2018

The associated Lisp code that uses the lib would also help.

@ubolonton

This comment has been minimized.

Copy link
Owner

commented Apr 14, 2018

I tried this on a large JSON array (10k elements, 7MB) and got a similar crash when trying to print out the whole returned list. It looks like Emacs has problem printing it.

@sebastiencs

This comment has been minimized.

Copy link
Author

commented Apr 14, 2018

Sure, I will provide a reproducible case, I should have done that first.
I will do that tomorrow

@sebastiencs

This comment has been minimized.

Copy link
Author

commented Apr 14, 2018

I have attached a json file, and here is a lisp code that make emacs crash:

(let* ((buffer (find-file-noselect "data.json"))
       (json (emacs-json-parse-string
              (with-current-buffer buffer (buffer-string))))
       (params (gethash "params" json))
       (symbols (gethash "symbols" params)))
  (maphash (lambda (key val)
             (equal key "test")
             (equal val "test"))
           symbols))

The stack trace with this code :

Stack traces

Stack trace of thread 4946:

#0  0x00007efe89845050 raise (libpthread.so.0)
#1  0x000000000050679f terminate_due_to_signal (emacs-27.0.50)
#2  0x0000000000520d30 emacs_abort (emacs-27.0.50)
#3  0x00000000005a64be print_object (emacs-27.0.50)
#4  0x00000000005a4f54 print_object (emacs-27.0.50)
#5  0x00000000005a1d2a Fprin1 (emacs-27.0.50)
#6  0x00000000005a37c0 print_error_message (emacs-27.0.50)
#7  0x00000000005a352c Ferror_message_string (emacs-27.0.50)
#8  0x00000000005856a6 signal_or_quit (emacs-27.0.50)
#9  0x0000000000585308 Fsignal (emacs-27.0.50)
#10 0x0000000000582c7a xsignal2 (emacs-27.0.50)
#11 0x000000000056c507 wrong_type_argument (emacs-27.0.50)
#12 0x000000000059485d Fmaphash (emacs-27.0.50)
#13 0x000000000058283d eval_sub (emacs-27.0.50)
#14 0x000000000058351d FletX (emacs-27.0.50)
#15 0x00000000005825b9 eval_sub (emacs-27.0.50)
#16 0x0000000000585ba2 Feval (emacs-27.0.50)
#17 0x000000000058708f funcall_subr (emacs-27.0.50)
#18 0x00000000005865f1 Ffuncall (emacs-27.0.50)
#19 0x00000000005c5698 exec_byte_code (emacs-27.0.50)
#20 0x0000000000586555 Ffuncall (emacs-27.0.50)
#21 0x00000000005c5698 exec_byte_code (emacs-27.0.50)
#22 0x0000000000586555 Ffuncall (emacs-27.0.50)
#23 0x000000000057ff4e Ffuncall_interactively (emacs-27.0.50)
#24 0x00000000005865f1 Ffuncall (emacs-27.0.50)
#25 0x000000000058125a Fcall_interactively (emacs-27.0.50)
#26 0x00000000005870af funcall_subr (emacs-27.0.50)
#27 0x00000000005865f1 Ffuncall (emacs-27.0.50)
#28 0x00000000005c5698 exec_byte_code (emacs-27.0.50)
#29 0x0000000000586555 Ffuncall (emacs-27.0.50)
#30 0x0000000000586ddd call1 (emacs-27.0.50)
#31 0x000000000050a563 command_loop_1 (emacs-27.0.50)
#32 0x0000000000584c36 internal_condition_case (emacs-27.0.50)
#33 0x00000000005183e4 command_loop_2 (emacs-27.0.50)
#34 0x000000000058431a internal_catch (emacs-27.0.50)
#35 0x00000000005094e2 command_loop (emacs-27.0.50)
#36 0x0000000000509424 recursive_edit_1 (emacs-27.0.50)
#37 0x000000000050966f Frecursive_edit (emacs-27.0.50)
#38 0x0000000000508128 main (emacs-27.0.50)
#39 0x00007efe886ecf2a __libc_start_main (libc.so.6)
#40 0x000000000041b2fa _start (emacs-27.0.50)

Stack trace of thread 4976:
#0  0x00007efe8984082d pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0)
#1  0x00007efe7e6bae0c _ZNSt18condition_variable4waitERSt11unique_lockISt5mutexE (libstdc++.so.6)
#2  0x00007efe8cda9317 _ZN7bmalloc9Scavenger13threadRunLoopEv (libjavascriptcoregtk-4.0.so.18)
#3  0x00007efe7e6c114f execute_native_thread_routine (libstdc++.so.6)
#4  0x00007efe8983a50b start_thread (libpthread.so.0)
#5  0x00007efe887c116f __clone (libc.so.6)

Stack trace of thread 4977:
#0  0x00007efe887b6c6b __poll (libc.so.6)
#1  0x00007efe91ef4e99 g_main_context_iterate.isra.23 (libglib-2.0.so.0)
#2  0x00007efe91ef4fac g_main_context_iteration (libglib-2.0.so.0)
#3  0x00007efe91ef4ff1 glib_worker_main (libglib-2.0.so.0)
#4  0x00007efe91f1c486 g_thread_proxy (libglib-2.0.so.0)
#5  0x00007efe8983a50b start_thread (libpthread.so.0)
#6  0x00007efe887c116f __clone (libc.so.6)

Stack trace of thread 5780:
#0  0x00007efe887b6c6b __poll (libc.so.6)
#1  0x00007efe91ef4e99 g_main_context_iterate.isra.23 (libglib-2.0.so.0)
#2  0x00007efe91ef4fac g_main_context_iteration (libglib-2.0.so.0)
#3  0x00007efe712a642d dconf_gdbus_worker_thread (libdconfsettings.so)
#4  0x00007efe91f1c486 g_thread_proxy (libglib-2.0.so.0)
#5  0x00007efe8983a50b start_thread (libpthread.so.0)
#6  0x00007efe887c116f __clone (libc.so.6)

Stack trace of thread 5779:
#0  0x00007efe887b6c6b __poll (libc.so.6)
#1  0x00007efe91ef4e99 g_main_context_iterate.isra.23 (libglib-2.0.so.0)
#2  0x00007efe91ef5232 g_main_loop_run (libglib-2.0.so.0)
#3  0x00007efe924ddb56 gdbus_shared_thread_func (libgio-2.0.so.0)
#4  0x00007efe91f1c486 g_thread_proxy (libglib-2.0.so.0)
#5  0x00007efe8983a50b start_thread (libpthread.so.0)
#6  0x00007efe887c116f __clone (libc.so.6)


There is a wrong-type-argument with a (corrupted ?) data, emacs try to print it and crash

data.json.zip

@ubolonton

This comment has been minimized.

Copy link
Owner

commented Apr 14, 2018

I think I know the problem.

I assumed all raw emacs_value's obtained from emacs_env live (protected from GC) as long as the emacs_env itself. Turns out that's not the case, at least in 25.3.1. I'm working on the fix now.

This is potentially an Emacs's bug (or bad design) though. If you have access to Emacs 26, can you help me check if it also crashes there?

@ubolonton

This comment has been minimized.

Copy link
Owner

commented Apr 14, 2018

FWIW, that assumption was in line with this https://phst.github.io/emacs-modules#value-lifetime. That's not an official doc though.

@ubolonton

This comment has been minimized.

Copy link
Owner

commented Apr 14, 2018

This temp workaround works for me (I got a different way to reproduce the issue locally):

After this line https://github.com/ubolonton/emacs-module-rs/blob/master/src/lib.rs#L168, add

// Leak all values.
raw_call_no_exit!(env, make_global_ref, raw);
@sebastiencs

This comment has been minimized.

Copy link
Author

commented Apr 14, 2018

If you have access to Emacs 26, can you help me check if it also crashes there?

I'm using emacs' master branch so it would crash there too.

FWIW, that assumption was in line with this https://phst.github.io/emacs-modules#value-lifetime. That's not an official doc though.

I think that it's the person who wrote emacs module system, so it's a legit source.
@phst, any thoughts on this ?

After this line https://github.com/ubolonton/emacs-module-rs/blob/master/src/lib.rs#L168, add

Yes that fix it for me too :)
Thanks for working on this so promptly, I wouldn't have found out by myself

@phst

This comment has been minimized.

Copy link

commented Apr 15, 2018

I assumed all raw emacs_value's obtained from emacs_env live (protected from GC) as long as the emacs_env itself. Turns out that's not the case, at least in 25.3.1.
This is potentially an Emacs's bug (or bad design) though.

Yes, that would definitely be a bug. Can you file a bug against Emacs with a minimal example? Thanks.

@ubolonton

This comment has been minimized.

Copy link
Owner

commented Apr 16, 2018

That's great to hear. I'll try to make a minimal example (without json parsing).

ubolonton added a commit that referenced this issue Apr 18, 2018

@ubolonton

This comment has been minimized.

Copy link
Owner

commented Apr 18, 2018

I'm getting closer to a minimal example https://github.com/ubolonton/emacs-module-rs/blob/crash/test-module/src/test_basics.rs#L130

It could be some weird interaction between Emacs, Rust compiler and this lib.

  • This will crash
    (|| {
        v.push("0".into_lisp(env).unwrap());
    })();
    env.message("")?;
    env.call("garbage-collect", &[])?;
    env.call("print", &[v[0]])?;
  • This will not
    v.push("0".into_lisp(env)?);
    env.message("")?;
    env.call("garbage-collect", &[]);
    env.call("print", &[v[0]])?;

I'll try to reproduce this in C.

@ubolonton

This comment has been minimized.

Copy link
Owner

commented Apr 22, 2018

@phst I'm not sure if it's the same thing, but I got minimal examples in https://github.com/ubolonton/emacs-module-crash.

I've filed an Emacs bug report.

ubolonton added a commit that referenced this issue Sep 9, 2018

ubolonton added a commit that referenced this issue Sep 9, 2018

@ubolonton ubolonton closed this in #3 Sep 15, 2018

@monnier

This comment has been minimized.

Copy link

commented Sep 21, 2018

FWIW, that assumption was in line with this https://phst.github.io/emacs-modules#value-lifetime. That's not an official doc though.

Indeed, the life-time described there reflects an early design. It was changed before emacs-modules were integrated into an Emacs release. The system used instead is that emacs_values that live on the C stack are safe (thanks to the conservative stack scanning) but those that live elsewhere need to be manually registered/unregistered.

@phst

This comment has been minimized.

Copy link

commented Jan 17, 2019

Indeed, the life-time described there reflects an early design.

It reflects the current and intended design. The Emacs implementation is simply buggy.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.