heap-use-after-free if callback deletes a variable while listing variables

[zeertzjq]

Neovim version (nvim -v)

v0.9.0-dev-156+g29fe3348b

Vim (not Nvim) behaves the same?

N/A

Operating system/version

Arch Linux

Terminal name/version

kitty 0.26.4

$TERM environment variable

xterm-kitty

Installation

build from repo

How to reproduce the issue

1. Run nvim -u NONE (nvim --clean also reproduces the issue but I think nvim -u NONE is cleaner here)
2. Source the following Lua file:

vim.cmd([[
  let g:var1 = 1
  let g:var2 = 2
]])

local ns = vim.api.nvim_create_namespace('test')

vim.ui_attach(ns, { ext_messages = true }, function(event, _, content)
  if event == 'msg_show' and content[1][2] == 'var1                  #1' then
    vim.cmd('unlet var2')
  end
end)

vim.cmd('let g:')

vim.ui_detach(ns)

Expected behavior

No crash

Actual behavior

heap-use-after-free:

=================================================================
==293331==ERROR: AddressSanitizer: heap-use-after-free on address 0x6030000841a1 at pc 0x5643a4583788 bp 0x7ffd4f50f030 sp 0x7ffd4f50e7f0
READ of size 4 at 0x6030000841a1 thread T0
    #0 0x5643a4583787 in __interceptor_memchr.part.0 asan_interceptors.cpp.o
    #1 0x5643a53baa6d in msg_puts_attr_len **/src/nvim/message.c:2024:3
    #2 0x5643a4d9b297 in list_one_var_a **/src/nvim/eval/vars.c:1179:5
    #3 0x5643a4d89c42 in list_one_var **/src/nvim/eval/vars.c:1164:3
    #4 0x5643a4d898e8 in list_hashtable_vars **/src/nvim/eval/vars.c:422:9
    #5 0x5643a4d9991b in list_glob_vars **/src/nvim/eval/vars.c:431:3
    #6 0x5643a4d993a4 in list_arg_vars **/src/nvim/eval/vars.c:499:17
    #7 0x5643a4d830cb in ex_let_const **/src/nvim/eval/vars.c:199:21
    #8 0x5643a4d82826 in ex_let **/src/nvim/eval/vars.c:161:3
    #9 0x5643a4e54a7f in execute_cmd0 **/src/nvim/ex_docmd.c:1619:7
    #10 0x5643a4e31a94 in do_one_cmd **/src/nvim/ex_docmd.c:2274:7
    #11 0x5643a4e2178e in do_cmdline **/src/nvim/ex_docmd.c:583:20
    #12 0x5643a583e3df in source_using_linegetter **/src/nvim/runtime.c:1802:16
    #13 0x5643a583dc51 in do_source_str **/src/nvim/runtime.c:1853:10
    #14 0x5643a487b386 in nvim_exec **/src/nvim/api/vimscript.c:67:3
    #15 0x5643a466ec2f in nlua_api_nvim_exec **/build/src/nvim/auto/lua_api_c_bindings.generated.c:5480:22
    #16 0x5643a6000f95 in lj_BC_FUNCC (**/build/bin/nvim+0x26fef95) (BuildId: 27fa072e05f9e23fbf8d36b2c3b9541486acb568)
    #17 0x5643a5fed3d4 in lua_pcall **/.deps/build/src/luajit/src/lj_api.c:1145:12
    #18 0x5643a5164e5d in nlua_pcall **/src/nvim/lua/executor.c:140:16
    #19 0x5643a51692b2 in nlua_exec_file **/src/nvim/lua/executor.c:1712:7
    #20 0x5643a5841277 in do_source **/src/nvim/runtime.c:2019:5
    #21 0x5643a583ce1d in cmd_source **/src/nvim/runtime.c:1618:14
    #22 0x5643a583c99d in ex_source **/src/nvim/runtime.c:1626:3
    #23 0x5643a4e54a7f in execute_cmd0 **/src/nvim/ex_docmd.c:1619:7
    #24 0x5643a4e31a94 in do_one_cmd **/src/nvim/ex_docmd.c:2274:7
    #25 0x5643a4e2178e in do_cmdline **/src/nvim/ex_docmd.c:583:20
    #26 0x5643a54b586b in nv_colon **/src/nvim/normal.c:4059:20
    #27 0x5643a54a1bf5 in normal_execute **/src/nvim/normal.c:1173:3
    #28 0x5643a5ab72e7 in state_enter **/src/nvim/state.c:88:26
    #29 0x5643a5471e00 in normal_enter **/src/nvim/normal.c:471:3
    #30 0x5643a5193e30 in main **/src/nvim/main.c:580:3
    #31 0x7f26bbd5e28f  (/usr/lib/libc.so.6+0x2328f) (BuildId: 1e94beb079e278ac4f2c8bce1f53091548ea1584)
    #32 0x7f26bbd5e349 in __libc_start_main (/usr/lib/libc.so.6+0x23349) (BuildId: 1e94beb079e278ac4f2c8bce1f53091548ea1584)
    #33 0x5643a4539d94 in _start /build/glibc/src/glibc/csu/../sysdeps/x86_64/start.S:115

0x6030000841a1 is located 17 bytes inside of 28-byte region [0x603000084190,0x6030000841ac)
freed by thread T0 here:
    #0 0x5643a45edc52 in __interceptor_free.part.0 asan_malloc_linux.cpp.o
    #1 0x5643a537b664 in xfree **/src/nvim/memory.c:128:3
    #2 0x5643a4d9018e in delete_var **/src/nvim/eval/vars.c:1157:3
    #3 0x5643a4d8face in do_unlet **/src/nvim/eval/vars.c:978:7
    #4 0x5643a4d8b587 in do_unlet_var **/src/nvim/eval/vars.c:854:16
    #5 0x5643a4d8aabc in ex_unletlock **/src/nvim/eval/vars.c:816:35
    #6 0x5643a4d89d5d in ex_unlet **/src/nvim/eval/vars.c:743:3
    #7 0x5643a4e54a7f in execute_cmd0 **/src/nvim/ex_docmd.c:1619:7
    #8 0x5643a4e31a94 in do_one_cmd **/src/nvim/ex_docmd.c:2274:7
    #9 0x5643a4e2178e in do_cmdline **/src/nvim/ex_docmd.c:583:20
    #10 0x5643a583e3df in source_using_linegetter **/src/nvim/runtime.c:1802:16
    #11 0x5643a583dc51 in do_source_str **/src/nvim/runtime.c:1853:10
    #12 0x5643a487b386 in nvim_exec **/src/nvim/api/vimscript.c:67:3
    #13 0x5643a466ec2f in nlua_api_nvim_exec **/build/src/nvim/auto/lua_api_c_bindings.generated.c:5480:22
    #14 0x5643a6000f95 in lj_BC_FUNCC (**/build/bin/nvim+0x26fef95) (BuildId: 27fa072e05f9e23fbf8d36b2c3b9541486acb568)

previously allocated by thread T0 here:
    #0 0x5643a45eec69 in malloc (**/build/bin/nvim+0xcecc69) (BuildId: 27fa072e05f9e23fbf8d36b2c3b9541486acb568)
    #1 0x5643a537b372 in try_malloc **/src/nvim/memory.c:80:15
    #2 0x5643a537b53c in xmalloc **/src/nvim/memory.c:114:15
    #3 0x5643a4d927bb in set_var_const **/src/nvim/eval/vars.c:1330:9
    #4 0x5643a4b33030 in set_var_lval **/src/nvim/eval.c:1685:7
    #5 0x5643a4d883d3 in ex_let_one **/src/nvim/eval/vars.c:728:9
    #6 0x5643a4d84e0e in ex_let_vars **/src/nvim/eval/vars.c:276:9
    #7 0x5643a4d84ae8 in ex_let_const **/src/nvim/eval/vars.c:249:13
    #8 0x5643a4d82826 in ex_let **/src/nvim/eval/vars.c:161:3
    #9 0x5643a4e54a7f in execute_cmd0 **/src/nvim/ex_docmd.c:1619:7
    #10 0x5643a4e31a94 in do_one_cmd **/src/nvim/ex_docmd.c:2274:7
    #11 0x5643a4e2178e in do_cmdline **/src/nvim/ex_docmd.c:583:20
    #12 0x5643a583e3df in source_using_linegetter **/src/nvim/runtime.c:1802:16
    #13 0x5643a583dc51 in do_source_str **/src/nvim/runtime.c:1853:10
    #14 0x5643a487b386 in nvim_exec **/src/nvim/api/vimscript.c:67:3
    #15 0x5643a466ec2f in nlua_api_nvim_exec **/build/src/nvim/auto/lua_api_c_bindings.generated.c:5480:22
    #16 0x5643a6000f95 in lj_BC_FUNCC (**/build/bin/nvim+0x26fef95) (BuildId: 27fa072e05f9e23fbf8d36b2c3b9541486acb568)

SUMMARY: AddressSanitizer: heap-use-after-free asan_interceptors.cpp.o in __interceptor_memchr.part.0
Shadow bytes around the buggy address:
  0x0c06800087e0: fd fd fa fa fd fd fd fd fa fa fd fd fd fd fa fa
  0x0c06800087f0: fd fd fd fd fa fa fd fd fd fd fa fa fd fd fd fd
  0x0c0680008800: fa fa fd fd fd fd fa fa fd fd fd fd fa fa fd fd
  0x0c0680008810: fd fd fa fa fd fd fd fd fa fa fd fd fd fd fa fa
  0x0c0680008820: 00 00 00 00 fa fa fd fd fd fd fa fa 00 00 00 04
=>0x0c0680008830: fa fa fd fd[fd]fd fa fa 00 00 00 00 fa fa fd fd
  0x0c0680008840: fd fd fa fa fd fd fd fd fa fa fd fd fd fd fa fa
  0x0c0680008850: fd fd fd fd fa fa fd fd fd fd fa fa fa fa fa fa
  0x0c0680008860: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c0680008870: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c0680008880: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==293331==ABORTING

[zeertzjq]

Related: #20790

[zeertzjq]

When the deleted variable is not the current variable the stacktrace is a bit different.

vim.cmd([[
let g:var1 = 1
let g:var2 = 2
let g:var3 = 3
]])

local ns = vim.api.nvim_create_namespace('test')

vim.ui_attach(ns, { ext_messages = true }, function(event, _, content)
  if event == 'msg_show' and content[1][2] == 'var1                  #1' then
    vim.cmd('unlet var3')
  end
end)

vim.cmd('let g:')

vim.ui_detach(ns)

With nvim -u NONE:

**/src/nvim/eval/vars.c:415:33: runtime error: member access within misaligned address 0x560c8e700a7f for type 'dictitem_T', which requires 8 byte alignment
0x560c8e700a7f: note: pointer points here
 00 00 00 00 00  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  90 0a 70 8e 0c 56 00 00  90 0a 70
             ^ 
    #0 0x560c8c09e346 in list_hashtable_vars **/src/nvim/eval/vars.c:415:33
    #1 0x560c8c0ae5db in list_glob_vars **/src/nvim/eval/vars.c:431:3
    #2 0x560c8c0ae064 in list_arg_vars **/src/nvim/eval/vars.c:499:17
    #3 0x560c8c097d8b in ex_let_const **/src/nvim/eval/vars.c:199:21
    #4 0x560c8c0974e6 in ex_let **/src/nvim/eval/vars.c:161:3
    #5 0x560c8c16973f in execute_cmd0 **/src/nvim/ex_docmd.c:1619:7
    #6 0x560c8c146754 in do_one_cmd **/src/nvim/ex_docmd.c:2274:7
    #7 0x560c8c13644e in do_cmdline **/src/nvim/ex_docmd.c:583:20
    #8 0x560c8cb5309f in source_using_linegetter **/src/nvim/runtime.c:1802:16
    #9 0x560c8cb52911 in do_source_str **/src/nvim/runtime.c:1853:10
    #10 0x560c8bb90386 in nvim_exec **/src/nvim/api/vimscript.c:67:3
    #11 0x560c8b983c2f in nlua_api_nvim_exec **/build/src/nvim/auto/lua_api_c_bindings.generated.c:5480:22
    #12 0x560c8d315c55 in lj_BC_FUNCC (**/build/bin/nvim+0x26fec55) (BuildId: 20388b3d2397123d4e392155c73b34ecf61e21fd)
    #13 0x560c8d302094 in lua_pcall **/.deps/build/src/luajit/src/lj_api.c:1145:12
    #14 0x560c8c479b1d in nlua_pcall **/src/nvim/lua/executor.c:140:16
    #15 0x560c8c47df72 in nlua_exec_file **/src/nvim/lua/executor.c:1712:7
    #16 0x560c8cb55f37 in do_source **/src/nvim/runtime.c:2019:5
    #17 0x560c8cb51add in cmd_source **/src/nvim/runtime.c:1618:14
    #18 0x560c8cb5165d in ex_source **/src/nvim/runtime.c:1626:3
    #19 0x560c8c16973f in execute_cmd0 **/src/nvim/ex_docmd.c:1619:7
    #20 0x560c8c146754 in do_one_cmd **/src/nvim/ex_docmd.c:2274:7
    #21 0x560c8c13644e in do_cmdline **/src/nvim/ex_docmd.c:583:20
    #22 0x560c8c7ca52b in nv_colon **/src/nvim/normal.c:4059:20
    #23 0x560c8c7b68b5 in normal_execute **/src/nvim/normal.c:1173:3
    #24 0x560c8cdcbfa7 in state_enter **/src/nvim/state.c:88:26
    #25 0x560c8c786ac0 in normal_enter **/src/nvim/normal.c:471:3
    #26 0x560c8c4a8af0 in main **/src/nvim/main.c:580:3
    #27 0x7f2b14edd28f  (/usr/lib/libc.so.6+0x2328f) (BuildId: 1e94beb079e278ac4f2c8bce1f53091548ea1584)
    #28 0x7f2b14edd349 in __libc_start_main (/usr/lib/libc.so.6+0x23349) (BuildId: 1e94beb079e278ac4f2c8bce1f53091548ea1584)
    #29 0x560c8b84ed94 in _start /build/glibc/src/glibc/csu/../sysdeps/x86_64/start.S:115

SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior **/src/nvim/eval/vars.c:415:33 in 

With nvim --clean:

=================================================================
==295739==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x619000003188 at pc 0x564be6cba070 bp 0x7ffc71002fb0 sp 0x7ffc71002fa8
READ of size 8 at 0x619000003188 thread T0
    #0 0x564be6cba06f in list_hashtable_vars **/src/nvim/eval/vars.c:408:10
    #1 0x564be6cca5db in list_glob_vars **/src/nvim/eval/vars.c:431:3
    #2 0x564be6cca064 in list_arg_vars **/src/nvim/eval/vars.c:499:17
    #3 0x564be6cb3d8b in ex_let_const **/src/nvim/eval/vars.c:199:21
    #4 0x564be6cb34e6 in ex_let **/src/nvim/eval/vars.c:161:3
    #5 0x564be6d8573f in execute_cmd0 **/src/nvim/ex_docmd.c:1619:7
    #6 0x564be6d62754 in do_one_cmd **/src/nvim/ex_docmd.c:2274:7
    #7 0x564be6d5244e in do_cmdline **/src/nvim/ex_docmd.c:583:20
    #8 0x564be776f09f in source_using_linegetter **/src/nvim/runtime.c:1802:16
    #9 0x564be776e911 in do_source_str **/src/nvim/runtime.c:1853:10
    #10 0x564be67ac386 in nvim_exec **/src/nvim/api/vimscript.c:67:3
    #11 0x564be659fc2f in nlua_api_nvim_exec **/build/src/nvim/auto/lua_api_c_bindings.generated.c:5480:22
    #12 0x564be7f31c55 in lj_BC_FUNCC (**/build/bin/nvim+0x26fec55) (BuildId: 20388b3d2397123d4e392155c73b34ecf61e21fd)
    #13 0x564be7f1e094 in lua_pcall **/.deps/build/src/luajit/src/lj_api.c:1145:12
    #14 0x564be7095b1d in nlua_pcall **/src/nvim/lua/executor.c:140:16
    #15 0x564be7099f72 in nlua_exec_file **/src/nvim/lua/executor.c:1712:7
    #16 0x564be7771f37 in do_source **/src/nvim/runtime.c:2019:5
    #17 0x564be776dadd in cmd_source **/src/nvim/runtime.c:1618:14
    #18 0x564be776d65d in ex_source **/src/nvim/runtime.c:1626:3
    #19 0x564be6d8573f in execute_cmd0 **/src/nvim/ex_docmd.c:1619:7
    #20 0x564be6d62754 in do_one_cmd **/src/nvim/ex_docmd.c:2274:7
    #21 0x564be6d5244e in do_cmdline **/src/nvim/ex_docmd.c:583:20
    #22 0x564be73e652b in nv_colon **/src/nvim/normal.c:4059:20
    #23 0x564be73d28b5 in normal_execute **/src/nvim/normal.c:1173:3
    #24 0x564be79e7fa7 in state_enter **/src/nvim/state.c:88:26
    #25 0x564be73a2ac0 in normal_enter **/src/nvim/normal.c:471:3
    #26 0x564be70c4af0 in main **/src/nvim/main.c:580:3
    #27 0x7f572b2cc28f  (/usr/lib/libc.so.6+0x2328f) (BuildId: 1e94beb079e278ac4f2c8bce1f53091548ea1584)
    #28 0x7f572b2cc349 in __libc_start_main (/usr/lib/libc.so.6+0x23349) (BuildId: 1e94beb079e278ac4f2c8bce1f53091548ea1584)
    #29 0x564be646ad94 in _start /build/glibc/src/glibc/csu/../sysdeps/x86_64/start.S:115

0x619000003188 is located 8 bytes to the right of 1024-byte region [0x619000002d80,0x619000003180)
allocated by thread T0 here:
    #0 0x564be651ffc1 in __interceptor_calloc (**/build/bin/nvim+0xcecfc1) (BuildId: 20388b3d2397123d4e392155c73b34ecf61e21fd)
    #1 0x564be72ac3c6 in xcalloc **/src/nvim/memory.c:142:15
    #2 0x564be6f6baab in hash_may_resize **/src/nvim/hashtab.c:351:7
    #3 0x564be6f6aaa1 in hash_add_item **/src/nvim/hashtab.c:233:3
    #4 0x564be6f6a7c5 in hash_add **/src/nvim/hashtab.c:212:3
    #5 0x564be6c45324 in tv_dict_add **/src/nvim/eval/typval.c:2202:10
    #6 0x564be6cc35c7 in set_var_const **/src/nvim/eval/vars.c:1332:9
    #7 0x564be6a64030 in set_var_lval **/src/nvim/eval.c:1685:7
    #8 0x564be6cb9093 in ex_let_one **/src/nvim/eval/vars.c:728:9
    #9 0x564be6cb5ace in ex_let_vars **/src/nvim/eval/vars.c:276:9
    #10 0x564be6cb57a8 in ex_let_const **/src/nvim/eval/vars.c:249:13
    #11 0x564be6cb34e6 in ex_let **/src/nvim/eval/vars.c:161:3
    #12 0x564be6d8573f in execute_cmd0 **/src/nvim/ex_docmd.c:1619:7
    #13 0x564be6d62754 in do_one_cmd **/src/nvim/ex_docmd.c:2274:7
    #14 0x564be6d5244e in do_cmdline **/src/nvim/ex_docmd.c:583:20
    #15 0x564be7772406 in do_source **/src/nvim/runtime.c:2036:5
    #16 0x564be7766dbc in source_callback **/src/nvim/runtime.c:218:9
    #17 0x564be775c5cb in do_in_path **/src/nvim/runtime.c:291:15
    #18 0x564be776381e in do_in_path_and_pp **/src/nvim/runtime.c:542:13
    #19 0x564be7766dfe in source_in_path **/src/nvim/runtime.c:794:10
    #20 0x564be7767744 in load_plugins **/src/nvim/runtime.c:1112:5
    #21 0x564be70c3be0 in main **/src/nvim/main.c:408:3
    #22 0x7f572b2cc28f  (/usr/lib/libc.so.6+0x2328f) (BuildId: 1e94beb079e278ac4f2c8bce1f53091548ea1584)

SUMMARY: AddressSanitizer: heap-buffer-overflow **/src/nvim/eval/vars.c:408:10 in list_hashtable_vars
Shadow bytes around the buggy address:
  0x0c327fff85e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c327fff85f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c327fff8600: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c327fff8610: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c327fff8620: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x0c327fff8630: fa[fa]fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c327fff8640: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c327fff8650: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c327fff8660: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c327fff8670: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c327fff8680: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==295739==ABORTING

[zeertzjq]

Similar crash in Vim: vim/vim#11435