Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

*** stack smashing detected ***: terminated Fatal error 6: Aborted #3296

Closed
sahirzm opened this issue Apr 15, 2024 · 44 comments
Closed

*** stack smashing detected ***: terminated Fatal error 6: Aborted #3296

sahirzm opened this issue Apr 15, 2024 · 44 comments
Labels
bug crash Executable is crashing or panicking

Comments

@sahirzm
Copy link

sahirzm commented Apr 15, 2024

Problem

Hi,
I updated tree-sitter on ArchLinux from 0.22.2-1 -> 0.22.4-1. After the upgrade, emacs has started to crash whenever I load any tsx file with the following backtrace:

*** stack smashing detected ***: terminated
Fatal error 6: Aborted
Backtrace:
emacs(+0x18a436)[0x5de6a212b436]
emacs(+0x24df3)[0x5de6a1fc5df3]
emacs(+0x25d01)[0x5de6a1fc6d01]
emacs(+0x2cf70d)[0x5de6a227070d]
/usr/lib/libc.so.6(+0x3c770)[0x727ab68fd770]
/usr/lib/libc.so.6(+0x8d32c)[0x727ab694e32c]
/usr/lib/libc.so.6(gsignal+0x18)[0x727ab68fd6c8]
/usr/lib/libc.so.6(abort+0xd7)[0x727ab68e54b8]
/usr/lib/libc.so.6(+0x25395)[0x727ab68e6395]
/usr/lib/libc.so.6(+0x11473b)[0x727ab69d573b]
/usr/lib/libc.so.6(+0x115a56)[0x727ab69d6a56]
emacs(+0x299883)[0x5de6a223a883]
/usr/lib/emacs/29.3/native-lisp/29.3-561b282f/treesit-37439c61-97df641d.eln(F747265657369742d666f6e742d6c6f636b2d666f6e746966792d726567696f6e_treesit_font_lock_fontify_region_0+0x1f2)[0x727a8eb6e832]
emacs(+0x21052e)[0x5de6a21b152e]
/usr/bin/../lib/emacs/29.3/native-lisp/29.3-561b282f/preloaded/font-lock-895216f6-1f3b244f.eln(F666f6e742d6c6f636b2d666f6e746966792d73796e746163746963616c6c792d726567696f6e_font_lock_fontify_syntactically_region_0+0x62)[0x727ab1e38f42]
emacs(+0x21052e)[0x5de6a21b152e]
/usr/bin/../lib/emacs/29.3/native-lisp/29.3-561b282f/preloaded/font-lock-895216f6-1f3b244f.eln(F666f6e742d6c6f636b2d64656661756c742d666f6e746966792d726567696f6e_font_lock_default_fontify_region_0+0x4af)[0x727ab1e36bef]
emacs(+0x21052e)[0x5de6a21b152e]
/usr/bin/../lib/emacs/29.3/native-lisp/29.3-561b282f/preloaded/font-lock-895216f6-1f3b244f.eln(F666f6e742d6c6f636b2d666f6e746966792d726567696f6e_font_lock_fontify_region_0+0x93)[0x727ab1e35873]
emacs(+0x25789e)[0x5de6a21f889e]
emacs(+0x21052e)[0x5de6a21b152e]
emacs(+0x211041)[0x5de6a21b2041]
emacs(+0x20ccac)[0x5de6a21adcac]
/usr/bin/../lib/emacs/29.3/native-lisp/29.3-561b282f/preloaded/jit-lock-8a988e43-a9956d8b.eln(F6a69742d6c6f636b2d2d72756e2d66756e6374696f6e73_jit_lock__run_functions_0+0xd8)[0x727ab1acac88]
emacs(+0x21052e)[0x5de6a21b152e]
/usr/bin/../lib/emacs/29.3/native-lisp/29.3-561b282f/preloaded/jit-lock-8a988e43-a9956d8b.eln(F6a69742d6c6f636b2d666f6e746966792d6e6f77_jit_lock_fontify_now_0+0x80a)[0x727ab1acb59a]
emacs(+0x21052e)[0x5de6a21b152e]
/usr/bin/../lib/emacs/29.3/native-lisp/29.3-561b282f/preloaded/jit-lock-8a988e43-a9956d8b.eln(F6a69742d6c6f636b2d66756e6374696f6e_jit_lock_function_0+0x26f)[0x727ab1aca98f]
emacs(+0x21052e)[0x5de6a21b152e]
emacs(+0x2d2b21)[0x5de6a2273b21]
emacs(+0x5706b)[0x5de6a1ff806b]
emacs(+0x5af00)[0x5de6a1ffbf00]
emacs(+0x5c086)[0x5de6a1ffd086]
emacs(+0x5e036)[0x5de6a1fff036]
emacs(+0x5c2b6)[0x5de6a1ffd2b6]
emacs(+0x7ae89)[0x5de6a201be89]
emacs(+0x82d92)[0x5de6a2023d92]
emacs(+0x73863)[0x5de6a2014863]
emacs(+0x20b43c)[0x5de6a21ac43c]
emacs(+0x737b8)[0x5de6a20147b8]
emacs(+0x774b2)[0x5de6a20184b2]
...
zsh: IOT instruction (core dumped)  emacs

I downgraded back to 0.22.2-1 and do not face any issues.

Steps to reproduce

  1. pacman -Syyu <- updates tree-sitter to 0.22.4-1
  2. Start emacs
  3. Open any tsx file.

Expected behavior

Emacs should not crash and I can open tsx files.

Tree-sitter version (tree-sitter --version)

tree-sitter 0.20.8 (d4c1bf7)

Operating system/version

ArchLinux

@sahirzm sahirzm added the bug label Apr 15, 2024
@amaanq
Copy link
Member

amaanq commented Apr 15, 2024

there's not much in this backtrace that we can go off of. are you able to build with debug syms and the frame pointer enabled? (-fno-omit-frame-pointer, if using clang you might also need -mno-omit-leaf-frame-pointer)

@thesamesam
Copy link

thesamesam commented Apr 15, 2024

I suspect we've hit the same thing in Gentoo as https://bugs.gentoo.org/930039 - it looks like 0.22.2 vs 0.22.4 breaks ABI?

EDIT: quoting some details from the Gentoo bug...

libabigail's abidiff output is:

$ abidiff /var/tmp/portage/dev-libs/tree-sitter-0.22.{2,4}/image/usr/lib64/libtree-sitter.so.0.22 --fail-no-debug-info --debug-info-dir1 /var/tmp/portage/dev-libs/tree-sitter-0.22.2/image/usr/lib/debug --debug-info-dir2 /var/tmp/portage/dev-libs/tree-sitter-0.22.4/image/usr/lib/debug
Functions changes summary: 0 Removed, 4 Changed (45 filtered out), 0 Added functions
Variables changes summary: 0 Removed, 0 Changed, 0 Added variable

4 functions with some indirect sub-type change:

  [C] 'function const size_t* ts_parser_cancellation_flag(const TSParser*)' at parser.c:1921:1 has some indirect sub-type changes:
    parameter 1 of type 'const TSParser*' has sub-type changes:
      in pointed to type 'const TSParser':
        in unqualified underlying type 'typedef TSParser' at api.h:45:1:
          underlying type 'struct TSParser' at parser.c:90:1 changed:
            type size hasn't changed
            1 data member insertion:
              'bool has_scanner_error', at offset 11616 (in bits) at parser.c:113:1

  [C] 'function void ts_query_cursor_delete(TSQueryCursor*)' at query.c:2986:1 has some indirect sub-type changes:
    parameter 1 of type 'TSQueryCursor*' has sub-type changes:
      in pointed to type 'typedef TSQueryCursor' at api.h:48:1:
        underlying type 'struct TSQueryCursor' at query.c:301:1 changed:
          type size changed from 1152 to 1216 (in bits)
          15 data member changes:
            type of 'TSTreeCursor cursor' changed:
              underlying type 'struct TSTreeCursor' at api.h:105:1 changed:
                type size changed from 192 to 256 (in bits)
                1 data member change:
                  type of 'uint32_t context[2]' changed:
                    type name changed from 'uint32_t[2]' to 'uint32_t[3]'
                    array type size changed from 64 to 96
                    array type subrange 1 changed length from 2 to 3
            'struct {QueryState* contents; uint32_t size; uint32_t capacity;} states' offset changed from 256 to 320 (in bits) (by +64 bits)
            'struct {QueryState* contents; uint32_t size; uint32_t capacity;} finished_states' offset changed from 384 to 448 (in bits) (by +64 bits)
            'CaptureListPool capture_list_pool' offset changed from 512 to 576 (in bits) (by +64 bits)
            'uint32_t depth' offset changed from 832 to 896 (in bits) (by +64 bits)
            'uint32_t max_start_depth' offset changed from 864 to 928 (in bits) (by +64 bits)
            'uint32_t start_byte' offset changed from 896 to 960 (in bits) (by +64 bits)
            'uint32_t end_byte' offset changed from 928 to 992 (in bits) (by +64 bits)
            'TSPoint start_point' offset changed from 960 to 1024 (in bits) (by +64 bits)
            'TSPoint end_point' offset changed from 1024 to 1088 (in bits) (by +64 bits)
            'uint32_t next_state_id' offset changed from 1088 to 1152 (in bits) (by +64 bits)
            'bool on_visible_node' offset changed from 1120 to 1184 (in bits) (by +64 bits)
            'bool ascending' offset changed from 1128 to 1192 (in bits) (by +64 bits)
            'bool halted' offset changed from 1136 to 1200 (in bits) (by +64 bits)
            'bool did_exceed_match_limit' offset changed from 1144 to 1208 (in bits) (by +64 bits)

  [C] 'function TSTreeCursor ts_tree_cursor_copy(const TSTreeCursor*)' at tree_cursor.c:695:1 has some indirect sub-type changes:

  [C] 'function TSTreeCursor ts_tree_cursor_new(TSNode)' at tree_cursor.c:153:1 has some indirect sub-type changes:

I've filed a bug upstream with libabigail after discussion on IRC wrt why it didn't flag it as a breaking change (https://sourceware.org/bugzilla/show_bug.cgi?id=31642).

Truncated backtrace (full at https://bugs.gentoo.org/930039#c2) from our pkgcheck tool which uses tree-sitter-bash:

tests/checks/test_codingstyle.py::TestStaticSrcUri::test_no_report[${P}]
Program received signal SIGSEGV, Segmentation fault.
0x00007ffff49a3726 in ts_tree_cursor_init (self=0x555555650008, node=...) at lib/src/tree_cursor.c:167
167       array_push(&self->stack, ((TreeCursorEntry) {
(gdb)
#0  0x00007ffff49a3726 in ts_tree_cursor_init (self=0x555555650008, node=...) at lib/src/tree_cursor.c:167
#1  ts_tree_cursor_reset (_self=0x555555650008, node=...) at lib/src/tree_cursor.c:160
#2  ts_query_cursor_exec (self=0x555555650000, query=0x555555d8f3f0, node=...) at lib/src/query.c:3045
#3  0x00007ffff529dce0 in query_captures (self=0x7ffff2618210, args=(<tree_sitter.Node at remote 0x7ffff2180b70>,), kwargs=0x0) at tree_sitter/binding.c:2112
#4  0x00007ffff7b0c603 in method_vectorcall_VARARGS_KEYWORDS (func=<method_descriptor at remote 0x7ffff260f740>, args=0x7ffff21f8a98, nargsf=<optimized out>, kwnames=<optimized out>) at Objects/descrobject.c:344
#5  0x00007ffff7af816e in _PyObject_VectorcallTstate (tstate=0x555555576f90, callable=<method_descriptor at remote 0x7ffff260f740>, args=0x7ffff21f8a98, nargsf=<optimized out>, kwnames=0x0) at ./Include/cpython/abstract.h:114
[...]

Valgrind output:

tests/checks/test_codingstyle.py::TestStaticSrcUri::test_no_report[${P}] ==602430== Invalid write of size 8
==602430==    at 0xDC597D4: ts_query_cursor_set_byte_range (query.c:3064)
==602430==    by 0xD609CA2: query_captures (binding.c:2110)
==602430==    by 0x49A2602: method_vectorcall_VARARGS_KEYWORDS (descrobject.c:344)
==602430==    by 0x498E16D: UnknownInlinedFun (abstract.h:114)
==602430==    by 0x498E16D: UnknownInlinedFun (abstract.h:123)
==602430==    by 0x498E16D: UnknownInlinedFun (ceval.c:5893)
==602430==    by 0x498E16D: _PyEval_EvalFrameDefault (ceval.c:4198)
==602430==    by 0x49D4AE9: UnknownInlinedFun (pycore_ceval.h:46)
==602430==    by 0x49D4AE9: UnknownInlinedFun (genobject.c:213)
==602430==    by 0x49D4AE9: gen_iternext (genobject.c:580)
==602430==    by 0x498E39F: _PyEval_EvalFrameDefault (ceval.c:4001)
==602430==    by 0x49D4E02: UnknownInlinedFun (pycore_ceval.h:46)
==602430==    by 0x49D4E02: gen_send_ex2 (genobject.c:213)
==602430==    by 0x4990580: _PyEval_EvalFrameDefault (ceval.c:2586)
==602430==    by 0x49D4AE9: UnknownInlinedFun (pycore_ceval.h:46)
==602430==    by 0x49D4AE9: UnknownInlinedFun (genobject.c:213)
==602430==    by 0x49D4AE9: gen_iternext (genobject.c:580)
==602430==    by 0x49DCFC4: list_extend (listobject.c:960)
==602430==    by 0x49AC14C: method_vectorcall_O (descrobject.c:460)
==602430==    by 0x498E16D: UnknownInlinedFun (abstract.h:114)
==602430==    by 0x498E16D: UnknownInlinedFun (abstract.h:123)
==602430==    by 0x498E16D: UnknownInlinedFun (ceval.c:5893)
==602430==    by 0x498E16D: _PyEval_EvalFrameDefault (ceval.c:4198)
==602430==  Address 0xa430078 is 24 bytes after a block of size 944 in arena "client"
==602430==

valgrind: m_mallocfree.c:304 (get_bszB_as_is): Assertion 'bszB_lo == bszB_hi' failed.
valgrind: Heap block lo/hi size mismatch: lo = 1008, hi = 18446744069414584320.
This is probably caused by your program erroneously writing past the
end of a heap block and corrupting heap metadata.  If you fix any
invalid writes reported by Memcheck, this assertion failure will
probably go away.  Please try that before reporting this as a bug.
```

@jostillmanns
Copy link

I suspect we've hit the same thing in Gentoo as https://bugs.gentoo.org/930039 - it looks like 0.22.2 vs 0.22.4 breaks ABI?

tree-sitter 0.22.4 also breaks things for me (crashes emacs) vs 0.22.2 works fine.

@seanhagen
Copy link

Been running into this on two different machines; opening any file that tries to use tree-sitter to handle syntax highlighting causes Emacs to crash with the same stack trace posted above.

@maxbrunsfeld
Copy link
Contributor

could anyone attach a debugger to emacs and help give some insight into the crash?

@maxbrunsfeld
Copy link
Contributor

The public TSTreeCursor struct has changed, so I do think that will change the ABI.

@maxbrunsfeld
Copy link
Contributor

maxbrunsfeld commented Apr 15, 2024

Just so I understand - on Arch Linux, Emacs uses tree-sitter via a dynamic library? Is the Arch package based on Git tags in the Tree-sitter repository? So if we push a tag to this repository, emacs users on Arch just start using the new version, based on the fact that it is a patch release?

@seanhagen
Copy link

seanhagen commented Apr 15, 2024

I think the only way Emacs can use tree-sitter is via dynamic library, I'm not sure it can be statically linked. At least, I can't find any information from some quick searching on how to have Emacs statically link dependencies.

The Arch package I'm using is extra/tree-sitter, and the PKGBUILD file in the package repository seems to indicate it checks out this repo using a specific tag ( currently 0.22.5 for the latest ). I think the package maintainer has to update the PKGBUILD file to point it to a new version. I'm not sure how the package is updated, but it does seem to be kept up-to-date with the latest version in this repository.

@maxbrunsfeld
Copy link
Contributor

That makes sense. Does the Emacs package specify a Semver range of Tree-sitter library that it is compatible with?

@xxyzz
Copy link

xxyzz commented Apr 16, 2024

Here is the backtrace from gdb on Arch Linux with emacs 29.3-2 and tree-sitter 0.22.5-1:

#0  __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at pthread_kill.c:44
#1  0x00007ffff3aab393 in __pthread_kill_internal (signo=6, threadid=<optimized out>) at pthread_kill.c:78
#2  0x00007ffff3a5a6c8 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
#3  0x00007ffff3a424b8 in __GI_abort () at abort.c:79
#4  0x00007ffff3a43395 in __libc_message_impl (fmt=fmt@entry=0x7ffff3bbb161 "*** %s ***: terminated\n") at ../sysdeps/posix/libc_fatal.c:132
#5  0x00007ffff3b3273b in __GI___fortify_fail (msg=msg@entry=0x7ffff3bbb179 "stack smashing detected") at fortify_fail.c:24
#6  0x00007ffff3b33a56 in __stack_chk_fail () at stack_chk_fail.c:24
#7  0x000055555579e063 in Ftreesit_subtree_stat (node=<optimized out>) at /usr/src/debug/emacs/emacs-29.3-wayland/src/treesit.c:3498
#8  0x00007fffe61d3832 in F747265657369742d666f6e742d6c6f636b2d666f6e746966792d726567696f6e_treesit_font_lock_fontify_region_0 () at /usr/lib/emacs/29.3/native-lisp/29.3-4b59e32f/treesit-37439c61-97df641d.eln
#9  0x000055555570a61e in Ffuncall (nargs=4, args=0x7fffffff8000) at /usr/src/debug/emacs/emacs-29.3-wayland/src/eval.c:2999
#10 0x00007fffef16cf42 in F666f6e742d6c6f636b2d666f6e746966792d73796e746163746963616c6c792d726567696f6e_font_lock_fontify_syntactically_region_0 ()
    at /usr/bin/../lib/emacs/29.3/native-lisp/29.3-4b59e32f/preloaded/font-lock-895216f6-1f3b244f.eln
#11 0x000055555570a61e in Ffuncall (nargs=4, args=0x7fffffff8170) at /usr/src/debug/emacs/emacs-29.3-wayland/src/eval.c:2999
#12 0x00007fffef16abef in F666f6e742d6c6f636b2d64656661756c742d666f6e746966792d726567696f6e_font_lock_default_fontify_region_0 ()
    at /usr/bin/../lib/emacs/29.3/native-lisp/29.3-4b59e32f/preloaded/font-lock-895216f6-1f3b244f.eln
#13 0x000055555570a61e in Ffuncall (nargs=4, args=0x7fffffff8220) at /usr/src/debug/emacs/emacs-29.3-wayland/src/eval.c:2999
#14 0x00007fffef169873 in F666f6e742d6c6f636b2d666f6e746966792d726567696f6e_font_lock_fontify_region_0 () at /usr/bin/../lib/emacs/29.3/native-lisp/29.3-4b59e32f/preloaded/font-lock-895216f6-1f3b244f.eln
#15 0x000055555575b8be in exec_byte_code (fun=0x2cd73, args_template=4294967295, nargs=2, args=0x7fffee400040) at /usr/src/debug/emacs/emacs-29.3-wayland/src/bytecode.c:809
#16 0x000055555570a61e in Ffuncall (nargs=2, args=args@entry=0x7fffffff8480) at /usr/src/debug/emacs/emacs-29.3-wayland/src/eval.c:2999
#17 0x000055555570ec31 in run_hook_wrapped_funcall (nargs=<optimized out>, args=0x7fffffff8480) at /usr/src/debug/emacs/emacs-29.3-wayland/src/eval.c:2777
#18 0x000055555570967c in run_hook_with_args (nargs=2, args=0x7fffffff8480, funcall=0x55555570ec10 <run_hook_wrapped_funcall>) at /usr/src/debug/emacs/emacs-29.3-wayland/src/eval.c:2858
#19 0x00007fffeedcec88 in F6a69742d6c6f636b2d2d72756e2d66756e6374696f6e73_jit_lock__run_functions_0 () at /usr/bin/../lib/emacs/29.3/native-lisp/29.3-4b59e32f/preloaded/jit-lock-8a988e43-a9956d8b.eln
#20 0x000055555570a61e in Ffuncall (nargs=3, args=0x7fffffff85a0) at /usr/src/debug/emacs/emacs-29.3-wayland/src/eval.c:2999
#21 0x00007fffeedcf59a in F6a69742d6c6f636b2d666f6e746966792d6e6f77_jit_lock_fontify_now_0 () at /usr/bin/../lib/emacs/29.3/native-lisp/29.3-4b59e32f/preloaded/jit-lock-8a988e43-a9956d8b.eln
#22 0x000055555570a61e in Ffuncall (nargs=3, args=0x7fffffff86b0) at /usr/src/debug/emacs/emacs-29.3-wayland/src/eval.c:2999
#23 0x00007fffeedce98f in F6a69742d6c6f636b2d66756e6374696f6e_jit_lock_function_0 () at /usr/bin/../lib/emacs/29.3/native-lisp/29.3-4b59e32f/preloaded/jit-lock-8a988e43-a9956d8b.eln
#24 0x000055555570a61e in Ffuncall (nargs=nargs@entry=2, args=args@entry=0x7fffffff8770) at /usr/src/debug/emacs/emacs-29.3-wayland/src/eval.c:2999
#25 0x00005555557f28d1 in internal_condition_case_n.constprop.0
    (bfun=0x55555570a520 <Ffuncall>, nargs=nargs@entry=2, args=args@entry=0x7fffffff8770, hfun=hfun@entry=0x5555555a1790 <safe_eval_handler>, handlers=0x30)
    at /usr/src/debug/emacs/emacs-29.3-wayland/src/eval.c:1558
#26 0x00005555555a190b in safe__call (inhibit_quit=inhibit_quit@entry=false, nargs=2, func=<optimized out>, ap=ap@entry=0x7fffffff87d0) at /usr/src/debug/emacs/emacs-29.3-wayland/src/xdisp.c:3059
#27 0x00005555555a7a10 in safe_call (nargs=<optimized out>, func=<optimized out>) at /usr/src/debug/emacs/emacs-29.3-wayland/src/xdisp.c:3074
#28 0x00005555555abd96 in safe_call1 (arg=0x6, fn=<optimized out>) at /usr/src/debug/emacs/emacs-29.3-wayland/src/xdisp.c:3085
#29 handle_fontified_prop (it=0x7fffffff89c0) at /usr/src/debug/emacs/emacs-29.3-wayland/src/xdisp.c:4601
#30 0x00005555555b1a76 in handle_stop (it=0x7fffffff89c0) at /usr/src/debug/emacs/emacs-29.3-wayland/src/xdisp.c:4134
#31 0x00005555555a8976 in start_display (it=0x7fffffff89c0, w=0x55555653c8e8, pos=...) at /usr/src/debug/emacs/emacs-29.3-wayland/src/xdisp.c:3751
#32 0x00005555555caaf9 in try_window (window=window@entry=0x55555653c8ed, pos=..., flags=flags@entry=1) at /usr/src/debug/emacs/emacs-29.3-wayland/src/xdisp.c:20748
#33 0x00005555555d2a02 in redisplay_window (window=<optimized out>, just_this_one_p=<optimized out>) at /usr/src/debug/emacs/emacs-29.3-wayland/src/xdisp.c:20140
#34 0x00005555555c34c3 in redisplay_window_0 (window=window@entry=0x55555653c8ed) at /usr/src/debug/emacs/emacs-29.3-wayland/src/xdisp.c:17623
#35 0x0000555555707aec in internal_condition_case_1 (bfun=0x5555555c3490 <redisplay_window_0>, arg=0x55555653c8ed, handlers=<optimized out>, hfun=0x5555555c34d0 <redisplay_window_error>)
    at /usr/src/debug/emacs/emacs-29.3-wayland/src/eval.c:1498
#36 0x00005555555c3418 in redisplay_windows (window=0x55555653c8ed) at /usr/src/debug/emacs/emacs-29.3-wayland/src/xdisp.c:17592
#37 0x00005555555c715c in redisplay_internal () at /usr/src/debug/emacs/emacs-29.3-wayland/src/xdisp.c:17042
#38 0x0000555555679d5e in redisplay () at /usr/src/debug/emacs/emacs-29.3-wayland/src/xdisp.c:16225
#39 read_char (commandflag=1, map=0x5555585f1d23, prev_event=0x0, used_mouse_menu=0x7fffffffe0db, end_time=0x0) at /usr/src/debug/emacs/emacs-29.3-wayland/src/keyboard.c:2636
#40 0x0000555555682ee1 in read_key_sequence (keybuf=<optimized out>, prompt=0x0, dont_downcase_last=<optimized out>, can_return_switch_frame=true, fix_current_buffer=true, prevent_redisplay=false) at /usr/src/debug/emacs/emacs-29.3-wayland/src/keyboard.c:10084
#41 0x000055555567282f in command_loop_1 () at /usr/src/debug/emacs/emacs-29.3-wayland/src/keyboard.c:1384
#42 0x0000555555707a57 in internal_condition_case (bfun=0x5555556725d0 <command_loop_1>, handlers=<optimized out>, hfun=0x555555670bd0 <cmd_error>) at /usr/src/debug/emacs/emacs-29.3-wayland/src/eval.c:1474
#43 0x000055555566ddae in command_loop_2 (handlers=handlers@entry=0x90) at /usr/src/debug/emacs/emacs-29.3-wayland/src/keyboard.c:1133
#44 0x000055555570799a in internal_catch (tag=<optimized out>, func=0x55555566dd80 <command_loop_2>, arg=0x90) at /usr/src/debug/emacs/emacs-29.3-wayland/src/eval.c:1197
#45 0x00005555556710d9 in command_loop () at /usr/src/debug/emacs/emacs-29.3-wayland/src/keyboard.c:1111
#46 0x000055555580740b in recursive_edit_1.isra.0 () at /usr/src/debug/emacs/emacs-29.3-wayland/src/keyboard.c:720
#47 0x00005555556714b1 in Frecursive_edit () at /usr/src/debug/emacs/emacs-29.3-wayland/src/keyboard.c:803
#48 0x0000555555586d19 in main (argc=<optimized out>, argv=0x7fffffffe748) at /usr/src/debug/emacs/emacs-29.3-wayland/src/emacs.c:2521

The emacs build file doesn't seem to set the dependency package version.

@maxbrunsfeld
Copy link
Contributor

The Tree-sitter C library's ABI has indeed changed (due to the TSQueryCursor struct field addition). I think that the Arch Emacs package should probably pin a specific version of the Tree-sitter package. I really don't want Emacs users to be broken by changes to Tree-sitter. Does anyone know the maintainers of the Arch packages?

In this repo, we try to not make breaking changes unnecessarily, but the library is pre 1.0, in its Semver versioning, and even once it goes 1.0, it's going to be easier to maintain API stability than full-on ABI stability.

@MatthewGentoo
Copy link
Contributor

Breaking ABI compatibility is not an issue at all, especially pre-1.0.

It would make things easier for consumers if the library's soname was bumped when ABI changed though. Having a different soname allows for the incompatible versions to be installed side by side, and is a signal that consumers need to be rebuilt when it changes.

@maxbrunsfeld
Copy link
Contributor

maxbrunsfeld commented Apr 16, 2024

Good point @MatthewGentoo - Do you know if there is a good tool for automatically detecting ABI changes in a header file? It would be cool to have our version bump script automatically bump the soname when warranted, before creating a new tag.

@thesamesam
Copy link

@maxbrunsfeld The output in #3296 (comment) is from abidiff (from libabigail) which is pretty good. Just in this instance, it didn't seem to catch the problem as an explicit breaking change (it noted diffs. but not that it was breaking). I've reported that upstream - who are great. There is also https://github.com/lvc/abi-compliance-checker.

@ObserverOfTime
Copy link
Member

ObserverOfTime commented Apr 16, 2024

Breaking ABI compatibility is not an issue at all, especially pre-1.0.

It is an issue when there's no such indication.
It might be a good idea to revert 09d2b23 and reserve it for ABI 15.

It would be cool to have our version bump script automatically bump the soname when warranted

The soname matches the version minus the patch. (0.22)

@ObserverOfTime ObserverOfTime added the crash Executable is crashing or panicking label Apr 16, 2024
@ObserverOfTime ObserverOfTime added this to the 0.22 milestone Apr 16, 2024
@ObserverOfTime
Copy link
Member

Downstream emacs issue: https://gitlab.archlinux.org/archlinux/packaging/packages/emacs/-/issues/2

@jesse-c
Copy link

jesse-c commented Apr 17, 2024

For anyone wanting to downgrade on Arch, I've been able to get Emacs non-crashing again with sudo pacman -U file:///var/cache/pacman/pkg/tree-sitter-0.22.2-1-x86_64.pkg.tar.zst [1].

[1] https://wiki.archlinux.org/title/downgrading_packages

@ObserverOfTime
Copy link
Member

ObserverOfTime commented Apr 17, 2024

You can also try rebuilding emacs and see if that fixes it.

@jesse-c
Copy link

jesse-c commented Apr 17, 2024

That didn't work for me, but hopefully does for others.

@1player
Copy link

1player commented Apr 17, 2024

If you don't have the old version of tree-sitter locally, you can download it from the Arch Linux archive at https://archive.archlinux.org/packages/t/tree-sitter/tree-sitter-0.22.2-1-x86_64.pkg.tar.zst

Downgrading to this version fixes it for me, for now.

@ObserverOfTime
Copy link
Member

ObserverOfTime commented Apr 17, 2024

There is also https://github.com/lvc/abi-compliance-checker.

That one catches the breaking change and produces a nice HTML report.
Unfortunately, we can't use the HTML report in CI (except as an artifact).
We might be able to use the XML report with some post-processing though.

abi-dumper /tmp/ts/0.22.2/lib/libtree-sitter.so -vnum 0.22.2 -o 0222.dump \
  -dir -public-headers "$PWD/lib/include" -include-paths "$PWD/lib/src;lib/include"
abi-dumper /tmp/ts/0.22.5/lib/libtree-sitter.so -vnum 0.22.5 -o 0225.dump \
   -dir -public-headers "$PWD/lib/include" -include-paths "$PWD/lib/src;$PWD/lib/include"
abi-compliance-checker -abi -l tree-sitter -old 0222.dump -new 0225.dump -check-private-abi # -report-format xml

However, it does not include line/column numbers in the report like abidiff does.

@maxbrunsfeld
Copy link
Contributor

maxbrunsfeld commented Apr 17, 2024

It sounds like in the future, we should bump the library's minor version when changing any structs in api.h, because with our current strategy, the soname is based on the minor version.

@ObserverOfTime
Copy link
Member

ObserverOfTime commented Apr 17, 2024

To be precise: the soname is currently only the major version but the file extension also includes the minor so that multiple versions can be installed side-by-side. I would not mind changing the soname (or extension) to an arbitrary number but we will need a workflow that checks for breaking ABI changes so we know when to bump it.

@adonig
Copy link

adonig commented Apr 24, 2024

I'm encountering the same issue with Emacs 29.3 after upgrading to Fedora 40. It comes with tree-sitter 0.22.5.

Fortunately, I still have a version of Emacs 30 that I compiled on Fedora 39 previously, and it still works.

@braun-steven
Copy link

braun-steven commented Apr 24, 2024

@adonig you can downgrade to the previous version of libtree-sitter via sudo dnf install libtree-sitter-0.22.2-1.fc40 and add excludepkgs=libtree-sitter to /etc/dnf/dnf.conf until the issue is resolved. Works for me on Fedora 40 with Emacs 29.3.

@seagle0128

This comment was marked as off-topic.

@ObserverOfTime
Copy link
Member

It will not be fixed on this side since reverting the change would introduce a regression.
Either emacs has to adapt to the breaking change or you have to downgrade tree-sitter.
(Rebuilding emacs with the upgraded library may or may not fix it.)

@braun-steven
Copy link

Same issue here. Don't know why this critical issue hasn't been fixed in last week.

I think the update to 0.22.5 went to stable only yesterday: https://bodhi.fedoraproject.org/updates/FEDORA-2024-e22c159068

@amaanq amaanq closed this as not planned Won't fix, can't repro, duplicate, stale Apr 24, 2024
@ObserverOfTime
Copy link
Member

I think the update to 0.22.5 went to stable only yesterday

The issue was introduced in 0.22.3

@braun-steven
Copy link

0.22.3 was never in the fedora repos. The last version in fc40 was 0.22.2 from three weeks ago: https://bodhi.fedoraproject.org/updates/FEDORA-2024-4b1211e6ae

awerdx520 added a commit to awerdx520/emacs.d that referenced this issue Apr 24, 2024
1. GNU Emacs 29.3 (build 1, x86_64-pc-linux-gnu, GTK+ Version
3.24.41, cairo version 1.18.0) 和 treesit 0.22.5-1 存在兼容性
bug 导致 Emacs 崩溃。

tree-sitter/tree-sitter#3296

2. 添加 devdocs 包
@joostkremers
Copy link

It will not be fixed on this side since reverting the change would introduce a regression. Either emacs has to adapt to the breaking change or you have to downgrade tree-sitter. (Rebuilding emacs with the upgraded library may or may not fix it.)

Just so that this is clear to me: The solution to the immediate problem would be for the relevant Linux distros to issue a new Emacs package?

@quentin-ag
Copy link

Rebuilding emacs with the upgraded library may or may not fix it

Arch users may be interested to know that rebuilding emacs 29.3-2 with tree-sitter 0.22.5-1 did not fix the issue for me.

@mavit
Copy link

mavit commented Apr 25, 2024

It sounds like in the future, we should bump the library's minor version when changing any structs in api.h, because with our current strategy, the soname is based on the minor version.

My understanding is that if the ABI of, say, libexample.so.0.22 changes, it’s not the 22 that’s expected to be bumped to 23, but rather the 0.22 that’s expected to be bumped to 1.0.

@dcreager
Copy link
Contributor

My understanding is that if the ABI of, say, libexample.so.0.22 changes, it’s not the 22 that’s expected to be bumped to 23, but rather the 0.22 that’s expected to be bumped to 1.0.

That's correct, I've written about this in the past for those who want the gory details. @maxbrunsfeld for us, I think we have to commit to not using the semver version in our library names, and to follow libtool-compatible rules for selecting a SONAME instead. Even though our semver version is pre-1.0, I think we need to be diligent about bumping the SONAME version whenever we introduce a breaking change to the ABI. SONAMEs are different, and don't have the same "everything goes before 1.0" rule that semver has. And it is often the case that a libary SONAME is not consistent with the package version number, regardless of whether the project uses semver.

@mbauhardt
Copy link

mbauhardt commented Apr 25, 2024

same issue with VoidLinux and tree-sitter-0.22.5_1 and Emacs-29.3

My workaround at the moment is to downgrade tree-sitter

check available packages on disk

$  ls /var/cache/xbps/tree-sitter-0.2*
/var/cache/xbps/tree-sitter-0.20.8_1.x86_64.xbps       /var/cache/xbps/tree-sitter-0.22.5_1.x86_64.xbps
/var/cache/xbps/tree-sitter-0.20.8_1.x86_64.xbps.sig2  /var/cache/xbps/tree-sitter-0.22.5_1.x86_64.xbps.sig2

register old package from cache

sudo xbps-rindex -a /var/cache/xbps/tree-sitter-0.20.8_1.x86_64.xbps

downgrade

sudo xbps-install -R /var/cache/xbps/ -f tree-sitter-0.20.8_1

opening emacs and I can work for example with rust/golang or any other tree-sitter based buffer,

@ObserverOfTime ObserverOfTime removed this from the 0.22 milestone Apr 25, 2024
@ObserverOfTime
Copy link
Member

The solution to the immediate problem would be for the relevant Linux distros to issue a new Emacs package?

Either that, if it works, or Emacs would have to update their code.
Any further comments should be made in distro bug trackers or Emacs itself.

The soname discussion can be moved to #3308 or a new issue.

@maxbrunsfeld
Copy link
Contributor

maxbrunsfeld commented Apr 25, 2024

Just so that this is clear to me: The solution to the immediate problem would be for the relevant Linux distros to issue a new Emacs package?

I really think that in order to provide a reliable end-user experience, and prevent issue like this, Emacs should just statically link a particular version of Tree-sitter.

If some Linux distros mandates that all libraries, no matter how small, must be distributed as dynamic libraries (which is just... staggeringly impractical IMO), then the Emacs package should specify a particular version of the Tree-sitter package to use - not a version range.

I think we have to commit to not using the semver version in our library names, and to follow libtool-compatible rules for selecting a SONAME instead.

I agree with this. Ideally, we should set up tooling to automatically bump the soname on breaking ABI changes. But if any Emacs package maintainer is seeing this issue - we do not yet have this tooling - this library is maintained by a small team, and may not have perfect ABI stability! Consider taking a more conservative approach to dependency versioning, so that end users don't have to deal with situations like this!

@thesamesam
Copy link

This wasn't specific to Emacs, though, it broke other tree-sitter consumers too. Distributions just need to rebuild all consumers of tree-sitter for now on new versions, which is what we're doing going forward in Gentoo.

@seagle0128

This comment was marked as off-topic.

@alanshutko
Copy link

It might be worth considering whether to build dynamic libraries at all. Would it make sense to build static libraries only until it hits 1.0 and has a stable ABI?

@ObserverOfTime
Copy link
Member

Go ask the distros.

@MatthewGentoo
Copy link
Contributor

We still like dynamic libraries :)

Someone distributing their application themselves (e.g. via a GitHub release) can still choose to statically link.

@amaanq
Copy link
Member

amaanq commented Apr 27, 2024

Dynamic libraries are fine, distros just need to ensure that the package that has tree-sitter as a shared lib dep uses the exact same version that the distro provides, but going forward we will try to be more mindful about abi breakage

netbsd-srcmastr pushed a commit to NetBSD/pkgsrc that referenced this issue Apr 30, 2024
@hrehfeld

This comment was marked as off-topic.

@tree-sitter tree-sitter locked and limited conversation to collaborators Apr 30, 2024
natrys referenced this issue in void-linux/void-packages May 7, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug crash Executable is crashing or panicking
Projects
None yet
Development

Successfully merging a pull request may close this issue.