Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Crash when opening scrollback buffer #3049

Closed
travankor opened this issue Oct 22, 2020 · 25 comments
Closed

Crash when opening scrollback buffer #3049

travankor opened this issue Oct 22, 2020 · 25 comments
Labels

Comments

@travankor
Copy link

travankor commented Oct 22, 2020

On kitty 0.19.1 on Linux I can sometimes reproduce a crash when opening the scrollback buffer and there is a secondary shell like gdb opened (not really sure about the last part). I have also noticed that when kitty does not crash, the scrollback buffer usually has garbage characters in its output. I am on python3.9 if this matters, too.

Steps to reproduce the behavior:
1. Open gdb
2. Open scrollback buffer
3. Crashes or corrupted scrollback buffer

Expected behavior
No crashes. Scrollback buffer is not corrupted.

Backtrace:

#0  __memmove_avx_unaligned_erms () at ../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S:262
262     ../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S: No such file or directory.
[Current thread is 1 (Thread 0x7f66564a3740 (LWP 12780))]
(gdb) bt full
#0  __memmove_avx_unaligned_erms () at ../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S:262
No locals.
#1  0x00007f66556ac236 in memcpy (__len=<optimized out>, __src=<optimized out>, __dest=<optimized out>, __dest=<optimized out>, __src=<optimized out>, __len=<optimized out>) at /usr/include/bits/string_fortified.h:34
No locals.
#2  pagerhist_as_bytes (self=<optimized out>, args=<optimized out>) at kitty/history.c:432
        ph = 0x7f6654c5d990
        l = <optimized out>
        sz = 12298
        ans = 0x55f2fdf408d0
        buf = 0x55f2fdf408f0 "\351\367\204\002\362U"
        copied = 12298
#3  0x00007f66556ac32a in pagerhist_as_text (self=<optimized out>, args=<optimized out>) at kitty/history.c:441
        ans = 0x0
        bytes = <optimized out>
#4  0x00007f665688b7f2 in ?? () from /usr/lib/libpython3.9.so.1.0
No symbol table info available.
#5  0x00007f6656847453 in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.9.so.1.0
No symbol table info available.
#6  0x00007f6656942aff in ?? () from /usr/lib/libpython3.9.so.1.0
No symbol table info available.
#7  0x00007f6656881b7a in _PyFunction_Vectorcall () from /usr/lib/libpython3.9.so.1.0
No symbol table info available.
#8  0x00007f6656846dee in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.9.so.1.0
No symbol table info available.
#9  0x00007f6656942aff in ?? () from /usr/lib/libpython3.9.so.1.0
No symbol table info available.
#10 0x00007f6656881b7a in _PyFunction_Vectorcall () from /usr/lib/libpython3.9.so.1.0
No symbol table info available.
#11 0x00007f6656884798 in ?? () from /usr/lib/libpython3.9.so.1.0
No symbol table info available.
#12 0x00007f66568471e7 in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.9.so.1.0
No symbol table info available.
#13 0x00007f6656942aff in ?? () from /usr/lib/libpython3.9.so.1.0
No symbol table info available.
#14 0x00007f6656881b7a in _PyFunction_Vectorcall () from /usr/lib/libpython3.9.so.1.0
No symbol table info available.
#15 0x00007f665688482c in ?? () from /usr/lib/libpython3.9.so.1.0
No symbol table info available.
#16 0x00007f66568818be in PyVectorcall_Call () from /usr/lib/libpython3.9.so.1.0
No symbol table info available.
#17 0x00007f665684589f in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.9.so.1.0
No symbol table info available.
#18 0x00007f66568409bb in ?? () from /usr/lib/libpython3.9.so.1.0
No symbol table info available.
#19 0x00007f6656847453 in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.9.so.1.0
No symbol table info available.
#20 0x00007f66568409bb in ?? () from /usr/lib/libpython3.9.so.1.0
No symbol table info available.
#21 0x00007f6656884704 in ?? () from /usr/lib/libpython3.9.so.1.0
No symbol table info available.
#22 0x00007f6656882a7e in ?? () from /usr/lib/libpython3.9.so.1.0
No symbol table info available.
#23 0x00007f6656883391 in _PyObject_CallMethod_SizeT () from /usr/lib/libpython3.9.so.1.0
No symbol table info available.
#24 0x00007f66556b9410 in on_key_input (ev=<optimized out>) at kitty/keys.c:173
        ret = <optimized out>
        w = 0x55f2fdf1b9b0
        action = 1
        key = 72
        screen = 0x7f6650526010
--Type <RET> for more, q to quit, c to continue without paging--
        has_text = false
        native_key = <optimized out>
        mods = <optimized out>
        text = 0x7f6654b995a0 ""
        ok_to_send = <optimized out>
        w = <optimized out>
        action = <optimized out>
        native_key = <optimized out>
        key = <optimized out>
        mods = <optimized out>
        text = <optimized out>
        screen = <optimized out>
        has_text = <optimized out>
        ok_to_send = <optimized out>
        cret_ = <optimized out>
        ret = <optimized out>
        consumed = <optimized out>
#25 key_callback (w=<optimized out>, ev=<optimized out>) at kitty/glfw.c:268
        key_modifier = <optimized out>
        key_modifier = <optimized out>
#26 key_callback (w=<optimized out>, ev=<optimized out>) at kitty/glfw.c:253
        key_modifier = <optimized out>
#27 0x00007f6654b3f538 in ?? ()
No symbol table info available.
#28 0x00007ffcc6d2c060 in ?? ()
No symbol table info available.
#29 0x0000000000000000 in ?? ()
No symbol table info available.
(gdb) quit

dmesg:

[42370.910368] kitty[12780]: segfault at 0 ip 00007f665677940f sp 00007ffcc6d2b0a8 error 4 in libc-2.30.so[7f665663d000+149000]
[42370.910387] Code: 17 e0 c5 f8 77 c3 48 3b 15 8e 0c 06 00 0f 83 25 01 00 00 48 39 f7 72 0f 74 12 4c 8d 0c 16 4c 39 cf 0f 82 c5 01 00 00 48 89 d1 <f3> a4 c3 80 fa 10 73 17 80 fa 08 73 27 80 fa 04 73 33 80 fa 01 77

@travankor travankor added the bug label Oct 22, 2020
@kovidgoyal
Copy link
Owner

Run kitty as

kitty --dump-bytes dump.bin --config NONE -o scrollback_pager-history_size=whatever

reproduce the crash with as few steps as possible and post dump.bin

@travankor
Copy link
Author

I'm not sure I can easily reproduce this one although I have been sporadically running into this crash.

@kovidgoyal
Copy link
Owner

it's possible there is a bug in the new ring buffer implementation that
was done for supporting hyperlinks in the scrollback buffer, but it's
not obvious to me where it is.

@travankor
Copy link
Author

travankor commented Oct 28, 2020

Can ASAN help pinpoint this crash?

@kovidgoyal
Copy link
Owner

It might, but that also requires a way to reproduce. You can build and
run kitty with ASAN by doing

make asan

@kovidgoyal
Copy link
Owner

I did find one memory related potential bug in this area: 792a3e7

@ssoriche
Copy link

I have the Apple Crash report if that would help? Has happened to me once per day over the past three days when trying to access the scrollback.

@kovidgoyal
Copy link
Owner

the apple crash report is the same as a backtrace. You need to either build with asan and post the error report from that, or find a way to reproduce and post the dump as described above.

https://sw.kovidgoyal.net/kitty/build.html

@hatzel
Copy link

hatzel commented Dec 4, 2020

Yeah, I ran into the same bug. I have not been able to reproduce it. It does happen probably around once a week though.

What I tried was dumping huge amounts of data into kitty cat /dev/urandom | base64 and than open the scrollback buffer. I was not able to reproduce it that way.

I also run kitty with -1 usually so maybe it could be related to that as well.

Will probably mess around with this some more, hopefully we can find out what's causing this.

@kovidgoyal
Copy link
Owner

I suggest building and running with asan as I described above, that might give us a more illuminating traceback

@kovidgoyal
Copy link
Owner

Note that since I cannot reproduce I have no way of confirming the fix, so please test.

@refractalize
Copy link

This happens pretty frequently for me, although I can't reproduce it in a controlled manner. It happens almost without fail when I've been working for a few hours already and I want to see the scrollback. So it's perhaps something to do with either the size of the scrollback, or some kind of encoding or control chars used in the contents of the scrollback. I can't reproduce it at will tho.

I'm using Macos 11.2. I've seen it seg fault on both 19.3 and from recent HEAD builds. My config:

scrollback_pager less --ignore-case --chop-long-lines --RAW-CONTROL-CHARS +INPUT_LINE_NUMBER
scrollback_pager_history_size 4096
map cmd+f show_scrollback

I also see seg faults when using launch with @screen_scrollback:

map cmd+shift+f launch --cwd=current --stdin-source=@screen_scrollback --type=overlay /usr/local/bin/nvim -R -

I'd like to help get this fixed because it's really preventing me from using scrollback seriously (tho when it does work it's sublime - I'm a recovering tmux user). I've tried to build using make asan but I'm seeing this:

{master} (11:54:43 +0.128) kitty λ make asan             
python3 setup.py build  --debug --sanitize
CC: clang (12, 0)
[12/59] Compiling kitty/shaders.c ... done
Compiling kitty/unicode-data.c ...
clang -MMD -DDEBUG -DPRIMARY_VERSION=4000 -DSECONDARY_VERSION=19 -DGL_SILENCE_DEPRECATION -Wextra -Wfloat-conversion -Wno-missing-field-initializers -Wall -Wstrict-prototypes -std=c11 -pedantic-errors -Werror -g3 -Og -fsanitize=address -fsanitize=undefined -fno-omit-frame-pointer -fwrapv -fstack-protector-strong -pipe -fvisibility=hidden -D_FORTIFY_SOURCE=2 -DKITTY_DEBUG_BUILD -pthread -I/usr/local/Cellar/libpng/1.6.37/include/libpng16 -I/usr/local/Cellar/little-cms2/2.11/include -I/usr/local/Cellar/harfbuzz/2.7.4/include/harfbuzz -I/usr/local/opt/freetype/include/freetype2 -I/usr/local/Cellar/graphite2/1.3.14/include -I/usr/local/Cellar/glib/2.66.6/include/glib-2.0 -I/usr/local/Cellar/glib/2.66.6/lib/glib-2.0/include -I/usr/local/opt/gettext/include -I/usr/local/Cellar/pcre/8.44/include -I/usr/local/Cellar/python@3.9/3.9.1_8/Frameworks/Python.framework/Versions/3.9/include/python3.9 -c kitty/unicode-data.c -o build/fast_data_types-unicode-data.c.o
In file included from <built-in>:369:
<command line>:5:9: error: '_FORTIFY_SOURCE' macro redefined [-Werror,-Wmacro-redefined]
#define _FORTIFY_SOURCE 2
        ^
<built-in>:355:9: note: previous definition is here
#define _FORTIFY_SOURCE 0
        ^
1 error generated.
make: *** [asan] Error 1

Any ideas?

@kovidgoyal
Copy link
Owner

i just made a commit that should fix building with asan on macOS

@refractalize
Copy link

Ok, built with make asan, but when I run python3 . I see this:

==84835==ERROR: Interceptors are not working. This may be because AddressSanitizer is loaded too late (e.g. via dlopen). Please launch the executable with:
DYLD_INSERT_LIBRARIES=/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/12.0.0/lib/darwin/libclang_rt.asan_osx_dynamic.dylib
"interceptors not installed" && 0[1]    84835 abort      python3 .

I've run it with

export DYLD_INSERT_LIBRARIES=/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/12.0.0/lib/darwin/libclang_rt.asan_osx_dynamic.dylib

But doesn't seem to make any difference...

@kovidgoyal
Copy link
Owner

u run it with kitty/launcher/kitty not python3

@refractalize
Copy link

refractalize commented Feb 24, 2021

I'm not making much progress here. I run kitty/launcher/kitty -c kitty.conf with kitty.conf being the sample kitty.conf at the bottom of the config page. I press ctrl-shift+enter to get a new window and it crashes immediately with the following. Hope this helps!

This is on bd67814

{master} (08:44:57 +21.914) kitty λ kitty/launcher/kitty -c kitty.conf
=================================================================
==32139==ERROR: AddressSanitizer: heap-use-after-free on address 0x61100023eeb8 at pc 0x000108c24f37 bp 0x7ffeeb667e70 sp 0x7ffeeb667e68
WRITE of size 4 at 0x61100023eeb8 thread T0
    #0 0x108c24f36 in on_key_input keys.c:147
    #1 0x108bb8491 in key_callback glfw.c:270
    #2 0x10a1be099 in _glfwInputKeyboard input.c:341
    #3 0x10a1f590e in -[GLFWContentView keyDown:] cocoa_window.m:1145
    #4 0x7fff22f45b07 in -[NSWindow(NSEventRouting) _reallySendEvent:isDelayedEvent:]+0x1951 (AppKit:x86_64+0x1c9b07)
    #5 0x7fff22f43f99 in -[NSWindow(NSEventRouting) sendEvent:]+0x15a (AppKit:x86_64+0x1c7f99)
    #6 0x7fff22f42d1c in -[NSApplication(NSEvent) sendEvent:]+0xaf8 (AppKit:x86_64+0x1c6d1c)
    #7 0x7fff2321af1d in -[NSApplication _handleEvent:]+0x40 (AppKit:x86_64+0x49ef1d)
    #8 0x7fff22dab6ae in -[NSApplication run]+0x26e (AppKit:x86_64+0x2f6ae)
    #9 0x10a1dfb3c in _glfwPlatformRunMainLoop cocoa_init.m:651
    #10 0x10a1bda00 in glfwRunMainLoop init.c:349
    #11 0x108bae145 in run_main_loop glfw.c:1332
    #12 0x108accbda in main_loop child-monitor.c:1012
    #13 0x104634ba8 in method_vectorcall_NOARGS+0x62 (Python:x86_64+0x73ba8)
    #14 0x104720f6e in call_function+0xa7 (Python:x86_64+0x15ff6e)
    #15 0x10471e7a1 in _PyEval_EvalFrameDefault+0xb067 (Python:x86_64+0x15d7a1)
    #16 0x1047125f1 in _PyEval_EvalCode+0x192 (Python:x86_64+0x1515f1)
    #17 0x104629403 in _PyFunction_Vectorcall+0x177 (Python:x86_64+0x68403)
    #18 0x104720f6e in call_function+0xa7 (Python:x86_64+0x15ff6e)
    #19 0x10471e87a in _PyEval_EvalFrameDefault+0xb140 (Python:x86_64+0x15d87a)
    #20 0x1047125f1 in _PyEval_EvalCode+0x192 (Python:x86_64+0x1515f1)
    #21 0x104629403 in _PyFunction_Vectorcall+0x177 (Python:x86_64+0x68403)
    #22 0x104628a12 in _PyObject_FastCallDictTstate+0x56 (Python:x86_64+0x67a12)
    #23 0x10469582f in slot_tp_call+0xba (Python:x86_64+0xd482f)
    #24 0x1046287e5 in _PyObject_MakeTpCall+0x80 (Python:x86_64+0x677e5)
    #25 0x104720fdc in call_function+0x115 (Python:x86_64+0x15ffdc)
    #26 0x10471e87a in _PyEval_EvalFrameDefault+0xb140 (Python:x86_64+0x15d87a)
    #27 0x10462934b in _PyFunction_Vectorcall+0xbf (Python:x86_64+0x6834b)
    #28 0x104720f6e in call_function+0xa7 (Python:x86_64+0x15ff6e)
    #29 0x10471e87a in _PyEval_EvalFrameDefault+0xb140 (Python:x86_64+0x15d87a)
    #30 0x10462934b in _PyFunction_Vectorcall+0xbf (Python:x86_64+0x6834b)
    #31 0x104720f6e in call_function+0xa7 (Python:x86_64+0x15ff6e)
    #32 0x10471e87a in _PyEval_EvalFrameDefault+0xb140 (Python:x86_64+0x15d87a)
    #33 0x10462934b in _PyFunction_Vectorcall+0xbf (Python:x86_64+0x6834b)
    #34 0x104720f6e in call_function+0xa7 (Python:x86_64+0x15ff6e)
    #35 0x10471e87a in _PyEval_EvalFrameDefault+0xb140 (Python:x86_64+0x15d87a)
    #36 0x1047125f1 in _PyEval_EvalCode+0x192 (Python:x86_64+0x1515f1)
    #37 0x10470d2ec in builtin_exec+0x184 (Python:x86_64+0x14c2ec)
    #38 0x1046738b6 in cfunction_vectorcall_FASTCALL+0x5e (Python:x86_64+0xb28b6)
    #39 0x104720f6e in call_function+0xa7 (Python:x86_64+0x15ff6e)
    #40 0x10471e87a in _PyEval_EvalFrameDefault+0xb140 (Python:x86_64+0x15d87a)
    #41 0x1047125f1 in _PyEval_EvalCode+0x192 (Python:x86_64+0x1515f1)
    #42 0x104629403 in _PyFunction_Vectorcall+0x177 (Python:x86_64+0x68403)
    #43 0x104720f6e in call_function+0xa7 (Python:x86_64+0x15ff6e)
    #44 0x10471e87a in _PyEval_EvalFrameDefault+0xb140 (Python:x86_64+0x15d87a)
    #45 0x1047125f1 in _PyEval_EvalCode+0x192 (Python:x86_64+0x1515f1)
    #46 0x104629403 in _PyFunction_Vectorcall+0x177 (Python:x86_64+0x68403)
    #47 0x104791707 in pymain_run_module+0xd3 (Python:x86_64+0x1d0707)
    #48 0x1047911ff in pymain_run_python+0x1de (Python:x86_64+0x1d01ff)
    #49 0x104790fe4 in Py_RunMain+0x16 (Python:x86_64+0x1cffe4)
    #50 0x104792303 in pymain_main+0x22 (Python:x86_64+0x1d1303)
    #51 0x1047922da in Py_Main+0x29 (Python:x86_64+0x1d12da)
    #52 0x104596dd3 in run_embedded+0xf93 (kitty:x86_64+0x100002dd3)
    #53 0x104595947 in main+0x817 (kitty:x86_64+0x100001947)
    #54 0x7fff204bf620 in start+0x0 (libdyld.dylib:x86_64+0x15620)

0x61100023eeb8 is located 248 bytes inside of 256-byte region [0x61100023edc0,0x61100023eec0)
freed by thread T0 here:
    #0 0x10499a412 in wrap_realloc+0xa2 (libclang_rt.asan_osx_dynamic.dylib:x86_64h+0x49412)
    #1 0x108d3293e in add_window state.c:194
    #2 0x108d2582b in pyadd_window state.c:1142
    #3 0x104672f35 in cfunction_call+0x59 (Python:x86_64+0xb1f35)
    #4 0x1046287e5 in _PyObject_MakeTpCall+0x80 (Python:x86_64+0x677e5)
    #5 0x104720fdc in call_function+0x115 (Python:x86_64+0x15ffdc)
    #6 0x10471e87a in _PyEval_EvalFrameDefault+0xb140 (Python:x86_64+0x15d87a)
    #7 0x1047125f1 in _PyEval_EvalCode+0x192 (Python:x86_64+0x1515f1)
    #8 0x104629403 in _PyFunction_Vectorcall+0x177 (Python:x86_64+0x68403)
    #9 0x104628a92 in _PyObject_FastCallDictTstate+0xd6 (Python:x86_64+0x67a92)
    #10 0x104697b7a in slot_tp_init+0xbf (Python:x86_64+0xd6b7a)
    #11 0x1046a134d in type_call+0x10f (Python:x86_64+0xe034d)
    #12 0x1046287e5 in _PyObject_MakeTpCall+0x80 (Python:x86_64+0x677e5)
    #13 0x104720fdc in call_function+0x115 (Python:x86_64+0x15ffdc)
    #14 0x10471e92a in _PyEval_EvalFrameDefault+0xb1f0 (Python:x86_64+0x15d92a)
    #15 0x1047125f1 in _PyEval_EvalCode+0x192 (Python:x86_64+0x1515f1)
    #16 0x104629403 in _PyFunction_Vectorcall+0x177 (Python:x86_64+0x68403)
    #17 0x10462bf85 in method_vectorcall+0x9f (Python:x86_64+0x6af85)
    #18 0x104720f6e in call_function+0xa7 (Python:x86_64+0x15ff6e)
    #19 0x10471e92a in _PyEval_EvalFrameDefault+0xb1f0 (Python:x86_64+0x15d92a)
    #20 0x1047125f1 in _PyEval_EvalCode+0x192 (Python:x86_64+0x1515f1)
    #21 0x104629403 in _PyFunction_Vectorcall+0x177 (Python:x86_64+0x68403)
    #22 0x104720f6e in call_function+0xa7 (Python:x86_64+0x15ff6e)
    #23 0x10471e7a1 in _PyEval_EvalFrameDefault+0xb067 (Python:x86_64+0x15d7a1)
    #24 0x1047125f1 in _PyEval_EvalCode+0x192 (Python:x86_64+0x1515f1)
    #25 0x104629403 in _PyFunction_Vectorcall+0x177 (Python:x86_64+0x68403)
    #26 0x10462c0c6 in method_vectorcall+0x1e0 (Python:x86_64+0x6b0c6)
    #27 0x10471ea76 in _PyEval_EvalFrameDefault+0xb33c (Python:x86_64+0x15da76)
    #28 0x10462934b in _PyFunction_Vectorcall+0xbf (Python:x86_64+0x6834b)
    #29 0x104720f6e in call_function+0xa7 (Python:x86_64+0x15ff6e)

previously allocated by thread T0 here:
    #0 0x10499a412 in wrap_realloc+0xa2 (libclang_rt.asan_osx_dynamic.dylib:x86_64h+0x49412)
    #1 0x108d3293e in add_window state.c:194
    #2 0x108d2582b in pyadd_window state.c:1142
    #3 0x104672f35 in cfunction_call+0x59 (Python:x86_64+0xb1f35)
    #4 0x1046287e5 in _PyObject_MakeTpCall+0x80 (Python:x86_64+0x677e5)
    #5 0x104720fdc in call_function+0x115 (Python:x86_64+0x15ffdc)
    #6 0x10471e87a in _PyEval_EvalFrameDefault+0xb140 (Python:x86_64+0x15d87a)
    #7 0x1047125f1 in _PyEval_EvalCode+0x192 (Python:x86_64+0x1515f1)
    #8 0x104629403 in _PyFunction_Vectorcall+0x177 (Python:x86_64+0x68403)
    #9 0x104628a92 in _PyObject_FastCallDictTstate+0xd6 (Python:x86_64+0x67a92)
    #10 0x104697b7a in slot_tp_init+0xbf (Python:x86_64+0xd6b7a)
    #11 0x1046a134d in type_call+0x10f (Python:x86_64+0xe034d)
    #12 0x1046287e5 in _PyObject_MakeTpCall+0x80 (Python:x86_64+0x677e5)
    #13 0x104720fdc in call_function+0x115 (Python:x86_64+0x15ffdc)
    #14 0x10471e92a in _PyEval_EvalFrameDefault+0xb1f0 (Python:x86_64+0x15d92a)
    #15 0x1047125f1 in _PyEval_EvalCode+0x192 (Python:x86_64+0x1515f1)
    #16 0x104629403 in _PyFunction_Vectorcall+0x177 (Python:x86_64+0x68403)
    #17 0x10462bf85 in method_vectorcall+0x9f (Python:x86_64+0x6af85)
    #18 0x104720f6e in call_function+0xa7 (Python:x86_64+0x15ff6e)
    #19 0x10471e92a in _PyEval_EvalFrameDefault+0xb1f0 (Python:x86_64+0x15d92a)
    #20 0x1047125f1 in _PyEval_EvalCode+0x192 (Python:x86_64+0x1515f1)
    #21 0x104629403 in _PyFunction_Vectorcall+0x177 (Python:x86_64+0x68403)
    #22 0x104720f6e in call_function+0xa7 (Python:x86_64+0x15ff6e)
    #23 0x10471e7a1 in _PyEval_EvalFrameDefault+0xb067 (Python:x86_64+0x15d7a1)
    #24 0x10462934b in _PyFunction_Vectorcall+0xbf (Python:x86_64+0x6834b)
    #25 0x104720f6e in call_function+0xa7 (Python:x86_64+0x15ff6e)
    #26 0x10471e7a1 in _PyEval_EvalFrameDefault+0xb067 (Python:x86_64+0x15d7a1)
    #27 0x1047125f1 in _PyEval_EvalCode+0x192 (Python:x86_64+0x1515f1)
    #28 0x104629403 in _PyFunction_Vectorcall+0x177 (Python:x86_64+0x68403)
    #29 0x104628a92 in _PyObject_FastCallDictTstate+0xd6 (Python:x86_64+0x67a92)

SUMMARY: AddressSanitizer: heap-use-after-free keys.c:147 in on_key_input
Shadow bytes around the buggy address:
  0x1c2200047d80: fd fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x1c2200047d90: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x1c2200047da0: fd fd fd fd fd fd fd fd fd fa fa fa fa fa fa fa
  0x1c2200047db0: fa fa fa fa fa fa fa fa fd fd fd fd fd fd fd fd
  0x1c2200047dc0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
=>0x1c2200047dd0: fd fd fd fd fd fd fd[fd]fa fa fa fa fa fa fa fa
  0x1c2200047de0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x1c2200047df0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x1c2200047e00: fa fa fa fa fa fa fa fa 00 00 00 00 00 00 00 00
  0x1c2200047e10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x1c2200047e20: 00 fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
  Shadow gap:              cc
==32139==ABORTING
[1]    32139 abort      kitty/launcher/kitty -c kitty.conf

@kovidgoyal
Copy link
Owner

That's an unrelated issue introduced in the recent key handling
refactoring, now fixed.

@refractalize
Copy link

Thanks @kovidgoyal. I'll work in this ASAN-enabled build and let you know when I see it break.

@refractalize
Copy link

I've been using the ASAN build for a couple of days now and thankfully no segfaults. Hopefully this is good news and this issue has somehow been fixed in recent commits, or it was indeed the key handling issue that caused this. That, or I've just been unlucky and I need to keep going to reproduce it (I'd normally have seen it by now though). I'll switch to the non-ASAN build next week to see if there's anything different.

@refractalize
Copy link

finally, it segfaulted (when looking at scrollback). Here's the dump:

{master} (16:36:42 +25.724) kitty λ kitty/launcher/kitty              
[055 16:38:16.514782] [PARSE ERROR] CSI code 0x74 has 3 > 2 parameters
[055 16:39:38.940766] [PARSE ERROR] CSI code 0x74 has 3 > 2 parameters
[055 16:40:46.785332] [PARSE ERROR] CSI code 0x74 has 3 > 2 parameters
[055 16:50:25.168150] [PARSE ERROR] CSI code 0x74 has 3 > 2 parameters
[055 16:50:26.486216] [PARSE ERROR] CSI code 0x74 has 3 > 2 parameters
[055 16:54:13.237386] [PARSE ERROR] CSI code 0x74 has 3 > 2 parameters
[055 16:54:14.718729] [PARSE ERROR] CSI code 0x74 has 3 > 2 parameters
[056 15:18:27.242543] [PARSE ERROR] CSI code 0x74 has 3 > 2 parameters
[056 15:18:30.605431] [PARSE ERROR] CSI code 0x74 has 3 > 2 parameters
[056 15:23:15.433209] [PARSE ERROR] CSI code 0x74 has 3 > 2 parameters
[056 15:23:39.471724] [PARSE ERROR] CSI code 0x74 has 3 > 2 parameters
[056 15:24:42.279789] [PARSE ERROR] CSI code 0x74 has 3 > 2 parameters
[056 15:24:44.112031] [PARSE ERROR] CSI code 0x74 has 3 > 2 parameters
[056 16:42:13.947589] [PARSE ERROR] CSI code 0x74 has 3 > 2 parameters
[056 16:42:15.991175] [PARSE ERROR] CSI code 0x74 has 3 > 2 parameters
[056 17:30:15.755896] [PARSE ERROR] CSI code 0x74 has 3 > 2 parameters
kitty/ringbuf.c:71:16: runtime error: member access within null pointer of type 'const struct ringbuf_t'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior kitty/ringbuf.c:71:16 in 
AddressSanitizer:DEADLYSIGNAL
=================================================================
==51414==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000018 (pc 0x000108ec2775 bp 0x7ffeeb3a8b80 sp 0x7ffeeb3a8b70 T0)
==51414==The signal is caused by a READ memory access.
==51414==Hint: address points to the zero page.
    #0 0x108ec2775 in ringbuf_buffer_size ringbuf.c:71
    #1 0x108ec2998 in ringbuf_capacity ringbuf.c:92
    #2 0x108ec2c9e in ringbuf_bytes_used ringbuf.c:118
    #3 0x108e3e006 in pagerhist_as_bytes history.c:406
    #4 0x108e3db7f in pagerhist_as_text history.c:419
    #5 0x1048f3ba8 in method_vectorcall_NOARGS+0x62 (Python:x86_64+0x73ba8)
    #6 0x1049dff6e in call_function+0xa7 (Python:x86_64+0x15ff6e)
    #7 0x1049dd7a1 in _PyEval_EvalFrameDefault+0xb067 (Python:x86_64+0x15d7a1)
    #8 0x1049d15f1 in _PyEval_EvalCode+0x192 (Python:x86_64+0x1515f1)
    #9 0x1048e8403 in _PyFunction_Vectorcall+0x177 (Python:x86_64+0x68403)
    #10 0x1049dff6e in call_function+0xa7 (Python:x86_64+0x15ff6e)
    #11 0x1049dd87a in _PyEval_EvalFrameDefault+0xb140 (Python:x86_64+0x15d87a)
    #12 0x1049d15f1 in _PyEval_EvalCode+0x192 (Python:x86_64+0x1515f1)
    #13 0x1048e8403 in _PyFunction_Vectorcall+0x177 (Python:x86_64+0x68403)
    #14 0x1048eaf85 in method_vectorcall+0x9f (Python:x86_64+0x6af85)
    #15 0x1049dff6e in call_function+0xa7 (Python:x86_64+0x15ff6e)
    #16 0x1049dd92a in _PyEval_EvalFrameDefault+0xb1f0 (Python:x86_64+0x15d92a)
    #17 0x1049d15f1 in _PyEval_EvalCode+0x192 (Python:x86_64+0x1515f1)
    #18 0x1048e8403 in _PyFunction_Vectorcall+0x177 (Python:x86_64+0x68403)
    #19 0x1048eb0c6 in method_vectorcall+0x1e0 (Python:x86_64+0x6b0c6)
    #20 0x1049dda76 in _PyEval_EvalFrameDefault+0xb33c (Python:x86_64+0x15da76)
    #21 0x1048e834b in _PyFunction_Vectorcall+0xbf (Python:x86_64+0x6834b)
    #22 0x1049dff6e in call_function+0xa7 (Python:x86_64+0x15ff6e)
    #23 0x1049dd7a1 in _PyEval_EvalFrameDefault+0xb067 (Python:x86_64+0x15d7a1)
    #24 0x1048e834b in _PyFunction_Vectorcall+0xbf (Python:x86_64+0x6834b)
    #25 0x1048eb052 in method_vectorcall+0x16c (Python:x86_64+0x6b052)
    #26 0x1048e88ff in _PyObject_CallFunctionVa+0xa2 (Python:x86_64+0x688ff)
    #27 0x1048e9209 in _PyObject_CallMethod_SizeT+0xc3 (Python:x86_64+0x69209)
    #28 0x108e6402a in on_key_input keys.c:159
    #29 0x108df7de1 in key_callback glfw.c:270
    #30 0x10a131099 in _glfwInputKeyboard input.c:341
    #31 0x10a16890e in -[GLFWContentView keyDown:] cocoa_window.m:1145
    #32 0x7fff22f45b07 in -[NSWindow(NSEventRouting) _reallySendEvent:isDelayedEvent:]+0x1951 (AppKit:x86_64+0x1c9b07)
    #33 0x7fff22f43f99 in -[NSWindow(NSEventRouting) sendEvent:]+0x15a (AppKit:x86_64+0x1c7f99)
    #34 0x7fff22f42d1c in -[NSApplication(NSEvent) sendEvent:]+0xaf8 (AppKit:x86_64+0x1c6d1c)
    #35 0x7fff2321af1d in -[NSApplication _handleEvent:]+0x40 (AppKit:x86_64+0x49ef1d)
    #36 0x7fff22dab6ae in -[NSApplication run]+0x26e (AppKit:x86_64+0x2f6ae)
    #37 0x10a152b3c in _glfwPlatformRunMainLoop cocoa_init.m:651
    #38 0x10a130a00 in glfwRunMainLoop init.c:349
    #39 0x108deda95 in run_main_loop glfw.c:1332
    #40 0x108d0c52a in main_loop child-monitor.c:1012
    #41 0x1048f3ba8 in method_vectorcall_NOARGS+0x62 (Python:x86_64+0x73ba8)
    #42 0x1049dff6e in call_function+0xa7 (Python:x86_64+0x15ff6e)
    #43 0x1049dd7a1 in _PyEval_EvalFrameDefault+0xb067 (Python:x86_64+0x15d7a1)
    #44 0x1049d15f1 in _PyEval_EvalCode+0x192 (Python:x86_64+0x1515f1)
    #45 0x1048e8403 in _PyFunction_Vectorcall+0x177 (Python:x86_64+0x68403)
    #46 0x1049dff6e in call_function+0xa7 (Python:x86_64+0x15ff6e)
    #47 0x1049dd87a in _PyEval_EvalFrameDefault+0xb140 (Python:x86_64+0x15d87a)
    #48 0x1049d15f1 in _PyEval_EvalCode+0x192 (Python:x86_64+0x1515f1)
    #49 0x1048e8403 in _PyFunction_Vectorcall+0x177 (Python:x86_64+0x68403)
    #50 0x1048e7a12 in _PyObject_FastCallDictTstate+0x56 (Python:x86_64+0x67a12)
    #51 0x10495482f in slot_tp_call+0xba (Python:x86_64+0xd482f)
    #52 0x1048e77e5 in _PyObject_MakeTpCall+0x80 (Python:x86_64+0x677e5)
    #53 0x1049dffdc in call_function+0x115 (Python:x86_64+0x15ffdc)
    #54 0x1049dd87a in _PyEval_EvalFrameDefault+0xb140 (Python:x86_64+0x15d87a)
    #55 0x1048e834b in _PyFunction_Vectorcall+0xbf (Python:x86_64+0x6834b)
    #56 0x1049dff6e in call_function+0xa7 (Python:x86_64+0x15ff6e)
    #57 0x1049dd87a in _PyEval_EvalFrameDefault+0xb140 (Python:x86_64+0x15d87a)
    #58 0x1048e834b in _PyFunction_Vectorcall+0xbf (Python:x86_64+0x6834b)
    #59 0x1049dff6e in call_function+0xa7 (Python:x86_64+0x15ff6e)
    #60 0x1049dd87a in _PyEval_EvalFrameDefault+0xb140 (Python:x86_64+0x15d87a)
    #61 0x1048e834b in _PyFunction_Vectorcall+0xbf (Python:x86_64+0x6834b)
    #62 0x1049dff6e in call_function+0xa7 (Python:x86_64+0x15ff6e)
    #63 0x1049dd87a in _PyEval_EvalFrameDefault+0xb140 (Python:x86_64+0x15d87a)
    #64 0x1049d15f1 in _PyEval_EvalCode+0x192 (Python:x86_64+0x1515f1)
    #65 0x1049cc2ec in builtin_exec+0x184 (Python:x86_64+0x14c2ec)
    #66 0x1049328b6 in cfunction_vectorcall_FASTCALL+0x5e (Python:x86_64+0xb28b6)
    #67 0x1049dff6e in call_function+0xa7 (Python:x86_64+0x15ff6e)
    #68 0x1049dd87a in _PyEval_EvalFrameDefault+0xb140 (Python:x86_64+0x15d87a)
    #69 0x1049d15f1 in _PyEval_EvalCode+0x192 (Python:x86_64+0x1515f1)
    #70 0x1048e8403 in _PyFunction_Vectorcall+0x177 (Python:x86_64+0x68403)
    #71 0x1049dff6e in call_function+0xa7 (Python:x86_64+0x15ff6e)
    #72 0x1049dd87a in _PyEval_EvalFrameDefault+0xb140 (Python:x86_64+0x15d87a)
    #73 0x1049d15f1 in _PyEval_EvalCode+0x192 (Python:x86_64+0x1515f1)
    #74 0x1048e8403 in _PyFunction_Vectorcall+0x177 (Python:x86_64+0x68403)
    #75 0x104a50707 in pymain_run_module+0xd3 (Python:x86_64+0x1d0707)
    #76 0x104a501ff in pymain_run_python+0x1de (Python:x86_64+0x1d01ff)
    #77 0x104a4ffe4 in Py_RunMain+0x16 (Python:x86_64+0x1cffe4)
    #78 0x104a51303 in pymain_main+0x22 (Python:x86_64+0x1d1303)
    #79 0x104a512da in Py_Main+0x29 (Python:x86_64+0x1d12da)
    #80 0x104854dd3 in run_embedded+0xf93 (kitty:x86_64+0x100002dd3)
    #81 0x104853947 in main+0x817 (kitty:x86_64+0x100001947)
    #82 0x7fff204bf620 in start+0x0 (libdyld.dylib:x86_64+0x15620)

==51414==Register values:
rax = 0x0000000000000003  rbx = 0x0000000000000018  rcx = 0x0000100000000003  rdx = 0x000000000000003f  
rdi = 0x0000000000000000  rsi = 0x000000010591bda0  rbp = 0x00007ffeeb3a8b80  rsp = 0x00007ffeeb3a8b70  
 r8 = 0x0000000104e29640   r9 = 0x00007ffeeb3a7e40  r10 = 0x0000000000000000  r11 = 0x0000000000000206  
r12 = 0x00007ffeeb3a8bc0  r13 = 0x00001fffdd675178  r14 = 0x0000000000000000  r15 = 0x0000000109bd7b01  
AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV ringbuf.c:71 in ringbuf_buffer_size
==51414==ABORTING
[1]    51414 abort      kitty/launcher/kitty
{master} (14:53:13 +46:16:25.883) kitty [134] λ 

@kovidgoyal
Copy link
Owner

OK I'll take a look when I have a moment, it seems to be a bug in the
ringbuf library kitty uses, at first glance.

@kovidgoyal
Copy link
Owner

See if this fixes it: 3ee7e5f

@refractalize
Copy link

Thanks @kovidgoyal, will test with this and get back to you.

@refractalize
Copy link

I've been using this for about whole 3 days now, well beyond the point at which that I'd normally see a segfault when opening the scrollback. I think we can say that it's been fixed, but I'll let you know obviously if I see something. I'm still using the ASAN build (I don't notice any perf issue).

Anyway, just wanted to let you know that (so far) things are looking positive.

@kovidgoyal
Copy link
Owner

glad to hear it :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

5 participants