Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

memory leak in yajl_tree_parse function. #250

Open
NotmebutWind opened this issue May 8, 2023 · 13 comments
Open

memory leak in yajl_tree_parse function. #250

NotmebutWind opened this issue May 8, 2023 · 13 comments

Comments

@NotmebutWind
Copy link

there is memory leak in yajl_tree_parse function
reproduce step:
1. compile the yajl with asan cmake . -DCMAKE_C_COMPILER=gcc -fsanitize=address -g
2. example/parse_config.c
3. crash input: "{"@\\\n\\\\""

==1696156==ERROR: LeakSanitizer: detected memory leaks
Direct leak of 24 byte(s) in 1 object(s) allocated from:
    #0 0x7ffff767d867 in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:145
    #1 0x555555561953 in context_push ./yajl/src/yajl_tree.c:117
    #2 0x555555563380 in handle_start_map ./yajl/src/yajl_tree.c:339
    #3 0x55555556d903 in yajl_do_parse ./yajl/src/yajl_parser.c:269
    #4 0x555555564f57 in yajl_parse ./yajl/src/yajl.c:130
    #5 0x5555555639c5 in yajl_tree_parse ./yajl/src/yajl_tree.c:435
    #6 0x555555560eeb in main parse_config.c:26
Indirect leak of 40 byte(s) in 1 object(s) allocated from:
    #0 0x7ffff767d867 in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:145
    #1 0x555555561424 in value_alloc ./yajl/src/yajl_tree.c:63
    #2 0x55555556322f in handle_start_map ./yajl/src/yajl_tree.c:331
    #3 0x55555556d903 in yajl_do_parse ./yajl/src/yajl_parser.c:269
    #4 0x555555564f57 in yajl_parse ./yajl/src/yajl.c:130
    #5 0x5555555639c5 in yajl_tree_parse ./yajl/src/yajl_tree.c:435
    #6 0x555555560eeb in main parse_config.c:26
Indirect leak of 9 byte(s) in 1 object(s) allocated from:
    #0 0x7ffff767d867 in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:145
    #1 0x555555562907 in handle_string ./yajl/src/yajl_tree.c:280
    #2 0x55555556f9a3 in yajl_do_parse ./yajl/src/yajl_parser.c:399
    #3 0x555555564f57 in yajl_parse ./yajl/src/yajl.c:130
    #4 0x5555555639c5 in yajl_tree_parse ./yajl/src/yajl_tree.c:435
    #5 0x555555560eeb in main parse_config.c:26
SUMMARY: AddressSanitizer: 73 byte(s) leaked in 3 allocation(s).

this may casue crash and dos for which use our lib.

@fuanan-3
Copy link

fuanan-3 commented Jun 7, 2023

please refer to:
openEuler-BaseService@23a122e

@coldtobi
Copy link

coldtobi commented Jul 1, 2023

This issue has been assigned CVE-2023-33460

@berrange
Copy link

  1. crash input: "{"@\\n\\""

It is slightly ambiguous whether the leading and trailing quotes here were intended to be part of the example JSON document. AFAICT though, to get the same set of three LeakSanitizer reports, requires omitting the leading/trailing quotes. i.e. parsing this document:

{"@\\\n\\\\"

And these three leaks were actually fixed by a much older commit: openEuler-BaseService@3d65cb0c

If you include the trailing and leading quotes, asking it to parse this document:

"{"@\\\n\\\\""

Then you only get two different leaks:

==642532==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 40 byte(s) in 1 object(s) allocated from:
    #0 0x7fa2aa2d92ff in malloc (/lib64/libasan.so.8+0xd92ff) (BuildId: dc689b05ca2577037af24700212bb5cce1f91c8a)
    #1 0x403b5c in value_alloc /home/berrange/src/external/yajl/src/yajl_tree.c:63
    #2 0x403b5c in handle_string /home/berrange/src/external/yajl/src/yajl_tree.c:276

Indirect leak of 2 byte(s) in 1 object(s) allocated from:
    #0 0x7fa2aa2d92ff in malloc (/lib64/libasan.so.8+0xd92ff) (BuildId: dc689b05ca2577037af24700212bb5cce1f91c8a)
    #1 0x403ba1 in handle_string /home/berrange/src/external/yajl/src/yajl_tree.c:280

SUMMARY: AddressSanitizer: 42 byte(s) leaked in 2 allocation(s).

and they are indeed fixed by this newer commit:

please refer to: openEuler-BaseService@23a122e

IOW, anyone patching a vanilla lloyd yajl 2.1.0 needs to pull both commits

openEuler-BaseService@23a122e
openEuler-BaseService@3d65cb0c

berrange pushed a commit to berrange/yajl that referenced this issue Jul 10, 2023
reason: fix memory leak problem
(cherry picked from commit 3d65cb0
 in https://github.com/openEuler-BaseService)

Fixes: lloyd#250  (CVE-2023-33460)
berrange pushed a commit to berrange/yajl that referenced this issue Jul 10, 2023
berrange pushed a commit to berrange/yajl that referenced this issue Jul 10, 2023
reason: fix memory leak problem
(cherry picked from commit 3d65cb0
 in https://github.com/openEuler-BaseService)

Fixes: lloyd#250  (CVE-2023-33460)
berrange pushed a commit to berrange/yajl that referenced this issue Jul 10, 2023
robohack added a commit to robohack/yajl that referenced this issue Jul 17, 2023
- use wrapped malloc() et al wrappers consistently
- update example/parse_config.c to do memory leak detection
- add a regression test using example/parse_config

Several issues in lloyd/yajl complained about this leak, and comments in
lloyd/yajl#102 showed a mostly correct fix though none of these issues
mentioned or actually fixed the directly related error reporting
problem.

Fixes lloyd/yajl#102, fixes lloyd/yajl#113, fixes lloyd/yajl#168, fixes
lloyd/yajl#191, fixes lloyd/yajl#223, fixes lloyd/yajl#250.  Also fixes
lloy/yajl#185.
kaniini added a commit to wolfi-dev/os that referenced this issue Jul 19, 2023
lloyd/yajl#250
Signed-off-by: Ariadne Conill <ariadne@dereferenced.org>
likema added a commit to likema/yajl that referenced this issue Dec 2, 2023
Memory leak in yajl 2.1.0 with use of yajl_tree_parse function
See lloyd#250 (comment)

Origin: openEuler-BaseService@23a122e
Bug-Debian: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1039984
Bug: lloyd#250
@pedrohc
Copy link

pedrohc commented Jan 5, 2024

Anyone knows what next release number will contain this fix in yajl?

@berrange
Copy link

berrange commented Jan 5, 2024

Anyone knows what next release number will contain this fix in yajl?

This project hasn't had any development activity in ~9 years, so I doubt there'll be any next release.

@chuanchang
Copy link

chuanchang commented Apr 1, 2024

@NotmebutWind @berrange I'm not cmake expert, and I tried to add the following lines into CMakeLists.txt then compile it, but it doesn't work for me, I also tried to use gcc or clang directly[1][2], but it always raised undefined reference, the yajl_tree.h indeed exists under the build/yajl-2.1.1/include/yajl/ and /usr/include/yajl/, so I think my testing must be wrong, please correct me, it will be better if your guys can share your configuration of the CMakeLists.txt, thank you so much!

SET(CMAKE_BUILD_TYPE "Debug")
SET(CMAKE_CXX_FLAGS_DEBUG "$ENV{CXXFLAGS} -O0 -Wall -g -Wno-attributes")
set(CMAKE_CXX_STANDARD 11)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fsanitize=address -fno-omit-frame-pointer")

[1]

[root@sweetpig-21 yajl]# ls build/yajl-2.1.1/include/yajl/
yajl_common.h  yajl_gen.h  yajl_parse.h  yajl_tree.h  yajl_version.h
[root@sweetpig-21 yajl]# ls /usr/include/yajl/
yajl_common.h  yajl_gen.h  yajl_parse.h  yajl_tree.h  yajl_version.h
[root@sweetpig-21 yajl]# rpm -qf /usr/include/yajl/
yajl-devel-2.1.0-22.el9.x86_64
[root@sweetpig-21 yajl]# pwd
/root/yajl
[root@sweetpig-21 yajl]# git log -1
commit 5e3a7856e643b4d6410ddc3f84bc2f38174f2872 (HEAD -> master, origin/master, origin/HEAD)
Merge: 12ee82a a30c6f7
Author: Lloyd Hilaiel <lloyd@hilaiel.com>
Date:   Thu Sep 24 13:16:46 2015 -0600

    Merge pull request #157 from setempler/bugfix-configure
    
    Fixed configure script to accept "-h" and "--prefix".
[root@sweetpig-21 yajl]# rpm -qf /usr/include/yajl/
yajl-devel-2.1.0-22.el9.x86_64
[root@sweetpig-21 yajl]# cmake --version
cmake version 3.26.5

CMake suite maintained and supported by Kitware (kitware.com/cmake).
[root@sweetpig-21 yajl]# rpm -q libasan liblsan
libasan-11.4.1-3.el9.x86_64
liblsan-11.4.1-3.el9.x86_64
[root@sweetpig-21 example]# gcc -v -Lyajl -O0 -Wall -g -Wno-attributes -fsanitize=address -fno-omit-frame-pointer parse_config.c -o leak_test
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-redhat-linux/11/lto-wrapper
OFFLOAD_TARGET_NAMES=nvptx-none
OFFLOAD_TARGET_DEFAULT=1
Target: x86_64-redhat-linux
Configured with: ../configure --enable-bootstrap --enable-host-pie --enable-host-bind-now --enable-languages=c,c++,fortran,lto --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-shared --enable-threads=posix --enable-checking=release --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-linker-build-id --with-gcc-major-version-only --enable-plugin --enable-initfini-array --without-isl --enable-multilib --with-linker-hash-style=gnu --enable-offload-targets=nvptx-none --without-cuda-driver --enable-gnu-indirect-function --enable-cet --with-tune=generic --with-arch_64=x86-64-v2 --with-arch_32=x86-64 --build=x86_64-redhat-linux --with-build-config=bootstrap-lto --enable-link-serialization=1
Thread model: posix
Supported LTO compression algorithms: zlib zstd
gcc version 11.4.1 20231218 (Red Hat 11.4.1-3) (GCC) 
COLLECT_GCC_OPTIONS='-v' '-Lyajl' '-O0' '-Wall' '-g' '-Wno-attributes' '-fsanitize=address' '-fno-omit-frame-pointer' '-o' 'leak_test' '-mtune=generic' '-march=x86-64-v2' '-dumpdir' 'leak_test-'
 /usr/libexec/gcc/x86_64-redhat-linux/11/cc1 -quiet -v parse_config.c -quiet -dumpdir leak_test- -dumpbase parse_config.c -dumpbase-ext .c -mtune=generic -march=x86-64-v2 -g -O0 -Wall -Wno-attributes -version -fsanitize=address -fno-omit-frame-pointer -o /tmp/cclffuKw.s
GNU C17 (GCC) version 11.4.1 20231218 (Red Hat 11.4.1-3) (x86_64-redhat-linux)
	compiled by GNU C version 11.4.1 20231218 (Red Hat 11.4.1-3), GMP version 6.2.0, MPFR version 4.1.0-p9, MPC version 1.2.1, isl version none
GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
ignoring nonexistent directory "/usr/lib/gcc/x86_64-redhat-linux/11/include-fixed"
ignoring nonexistent directory "/usr/lib/gcc/x86_64-redhat-linux/11/../../../../x86_64-redhat-linux/include"
#include "..." search starts here:
#include <...> search starts here:
 /usr/lib/gcc/x86_64-redhat-linux/11/include
 /usr/local/include
 /usr/include
End of search list.
GNU C17 (GCC) version 11.4.1 20231218 (Red Hat 11.4.1-3) (x86_64-redhat-linux)
	compiled by GNU C version 11.4.1 20231218 (Red Hat 11.4.1-3), GMP version 6.2.0, MPFR version 4.1.0-p9, MPC version 1.2.1, isl version none
GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
Compiler executable checksum: 29483c90d60225520f7ec2f3fbd4b681
COLLECT_GCC_OPTIONS='-v' '-Lyajl' '-O0' '-Wall' '-g' '-Wno-attributes' '-fsanitize=address' '-fno-omit-frame-pointer' '-o' 'leak_test' '-mtune=generic' '-march=x86-64-v2' '-dumpdir' 'leak_test-'
 as -v --gdwarf-5 --64 -o /tmp/ccowXVbC.o /tmp/cclffuKw.s
GNU assembler version 2.35.2 (x86_64-redhat-linux) using BFD version version 2.35.2-43.el9
COMPILER_PATH=/usr/libexec/gcc/x86_64-redhat-linux/11/:/usr/libexec/gcc/x86_64-redhat-linux/11/:/usr/libexec/gcc/x86_64-redhat-linux/:/usr/lib/gcc/x86_64-redhat-linux/11/:/usr/lib/gcc/x86_64-redhat-linux/
LIBRARY_PATH=/usr/lib/gcc/x86_64-redhat-linux/11/:/usr/lib/gcc/x86_64-redhat-linux/11/../../../../lib64/:/lib/../lib64/:/usr/lib/../lib64/:/usr/lib/gcc/x86_64-redhat-linux/11/../../../:/lib/:/usr/lib/
COLLECT_GCC_OPTIONS='-v' '-Lyajl' '-O0' '-Wall' '-g' '-Wno-attributes' '-fsanitize=address' '-fno-omit-frame-pointer' '-o' 'leak_test' '-mtune=generic' '-march=x86-64-v2' '-dumpdir' 'leak_test.'
 /usr/libexec/gcc/x86_64-redhat-linux/11/collect2 -plugin /usr/libexec/gcc/x86_64-redhat-linux/11/liblto_plugin.so -plugin-opt=/usr/libexec/gcc/x86_64-redhat-linux/11/lto-wrapper -plugin-opt=-fresolution=/tmp/cc60jISU.res -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_s -plugin-opt=-pass-through=-lc -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_s --build-id --no-add-needed --eh-frame-hdr --hash-style=gnu -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 -o leak_test /usr/lib/gcc/x86_64-redhat-linux/11/../../../../lib64/crt1.o /usr/lib/gcc/x86_64-redhat-linux/11/../../../../lib64/crti.o /usr/lib/gcc/x86_64-redhat-linux/11/crtbegin.o -Lyajl -L/usr/lib/gcc/x86_64-redhat-linux/11 -L/usr/lib/gcc/x86_64-redhat-linux/11/../../../../lib64 -L/lib/../lib64 -L/usr/lib/../lib64 -L/usr/lib/gcc/x86_64-redhat-linux/11/../../.. /usr/lib/gcc/x86_64-redhat-linux/11/libasan_preinit.o -lasan /tmp/ccowXVbC.o -lgcc --push-state --as-needed -lgcc_s --pop-state -lc -lgcc --push-state --as-needed -lgcc_s --pop-state /usr/lib/gcc/x86_64-redhat-linux/11/crtend.o /usr/lib/gcc/x86_64-redhat-linux/11/../../../../lib64/crtn.o
/usr/bin/ld: /tmp/ccowXVbC.o: in function `main':
/root/yajl/example/parse_config.c:47: undefined reference to `yajl_tree_parse'
/usr/bin/ld: /root/yajl/example/parse_config.c:61: undefined reference to `yajl_tree_get'
/usr/bin/ld: /root/yajl/example/parse_config.c:66: undefined reference to `yajl_tree_free'
collect2: error: ld returned 1 exit status

[2]

[root@sweetpig-21 yajl]# ls /root/yajl/build/yajl-2.1.1/include/
yajl
[root@sweetpig-21 yajl]# ls /root/yajl/build/yajl-2.1.1/include/yajl
yajl_common.h  yajl_gen.h  yajl_parse.h  yajl_tree.h  yajl_version.h
[root@sweetpig-21 yajl]# clang -v -I/root/yajl/build/yajl-2.1.1/include/ -L/root/yajl/build/yajl-2.1.1/lib/ -fsanitize=address -O1 -fno-omit-frame-pointer -g example/parse_config.c
clang version 17.0.6 (Red Hat, Inc. 17.0.6-5.el9)
Target: x86_64-redhat-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
Found candidate GCC installation: /opt/rh/gcc-toolset-13/root/usr/lib/gcc/x86_64-redhat-linux/13
Selected GCC installation: /opt/rh/gcc-toolset-13/root/usr/lib/gcc/x86_64-redhat-linux/13
Candidate multilib: .;@m64
Candidate multilib: 32;@m32
Selected multilib: .;@m64
 "/usr/bin/clang-17" -cc1 -triple x86_64-redhat-linux-gnu -emit-obj -dumpdir a- -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name parse_config.c -mrelocation-model static -mframe-pointer=all -fmath-errno -ffp-contract=on -fno-rounding-math -mconstructor-aliases -funwind-tables=2 -target-cpu x86-64 -tune-cpu generic -debug-info-kind=constructor -dwarf-version=4 -debugger-tuning=gdb -v -fcoverage-compilation-dir=/root/yajl -resource-dir /usr/bin/../lib/clang/17 -I /root/yajl/build/yajl-2.1.1/include/ -internal-isystem /usr/bin/../lib/clang/17/include -internal-isystem /usr/local/include -internal-isystem /opt/rh/gcc-toolset-13/root/usr/lib/gcc/x86_64-redhat-linux/13/../../../../x86_64-redhat-linux/include -internal-externc-isystem /include -internal-externc-isystem /usr/include -O1 -fdebug-compilation-dir=/root/yajl -ferror-limit 19 -fsanitize=address -fsanitize-system-ignorelist=/usr/bin/../lib/clang/17/share/asan_ignorelist.txt -fno-sanitize-memory-param-retval -fsanitize-address-use-after-scope -fsanitize-address-globals-dead-stripping -fno-assume-sane-operator-new -fgnuc-version=4.2.1 -fcolor-diagnostics -faddrsig -D__GCC_HAVE_DWARF2_CFI_ASM=1 -o /tmp/parse_config-476506.o -x c example/parse_config.c
clang -cc1 version 17.0.6 based upon LLVM 17.0.6 default target x86_64-redhat-linux-gnu
ignoring nonexistent directory "/opt/rh/gcc-toolset-13/root/usr/lib/gcc/x86_64-redhat-linux/13/../../../../x86_64-redhat-linux/include"
ignoring nonexistent directory "/include"
#include "..." search starts here:
#include <...> search starts here:
 /root/yajl/build/yajl-2.1.1/include
 /usr/bin/../lib/clang/17/include
 /usr/local/include
 /usr/include
End of search list.
 "/opt/rh/gcc-toolset-13/root/usr/lib/gcc/x86_64-redhat-linux/13/../../../../bin/ld" --hash-style=gnu --build-id --eh-frame-hdr -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 -o a.out /lib/../lib64/crt1.o /lib/../lib64/crti.o /opt/rh/gcc-toolset-13/root/usr/lib/gcc/x86_64-redhat-linux/13/crtbegin.o -L/root/yajl/build/yajl-2.1.1/lib/ -L/usr/bin/../lib/clang/17/lib/x86_64-redhat-linux-gnu -L/opt/rh/gcc-toolset-13/root/usr/lib/gcc/x86_64-redhat-linux/13 -L/opt/rh/gcc-toolset-13/root/usr/lib/gcc/x86_64-redhat-linux/13/../../../../lib64 -L/lib/../lib64 -L/usr/lib/../lib64 -L/lib -L/usr/lib --whole-archive /usr/bin/../lib/clang/17/lib/x86_64-redhat-linux-gnu/libclang_rt.asan_static.a --no-whole-archive --whole-archive /usr/bin/../lib/clang/17/lib/x86_64-redhat-linux-gnu/libclang_rt.asan.a --no-whole-archive --dynamic-list=/usr/bin/../lib/clang/17/lib/x86_64-redhat-linux-gnu/libclang_rt.asan.a.syms /tmp/parse_config-476506.o --no-as-needed -lpthread -lrt -lm -ldl -lresolv -lgcc --as-needed -lgcc_s --no-as-needed -lc -lgcc --as-needed -lgcc_s --no-as-needed /opt/rh/gcc-toolset-13/root/usr/lib/gcc/x86_64-redhat-linux/13/crtend.o /lib/../lib64/crtn.o
/opt/rh/gcc-toolset-13/root/usr/lib/gcc/x86_64-redhat-linux/13/../../../../bin/ld: /tmp/parse_config-476506.o: in function `main':
/root/yajl/example/parse_config.c:47: undefined reference to `yajl_tree_parse'
/opt/rh/gcc-toolset-13/root/usr/lib/gcc/x86_64-redhat-linux/13/../../../../bin/ld: /root/yajl/example/parse_config.c:61: undefined reference to `yajl_tree_get'
/opt/rh/gcc-toolset-13/root/usr/lib/gcc/x86_64-redhat-linux/13/../../../../bin/ld: /root/yajl/example/parse_config.c:66: undefined reference to `yajl_tree_free'
clang: error: linker command failed with exit code 1 (use -v to see invocation)

I also tried to modify example/CMakeLists.txt with the following lines, although I can compile parse_config.c successfully, I can't see expected leak, in addition, the ./parse_config "{"@\\n\\"" will hang forever.

cmake_minimum_required(VERSION 3.10)
project (demo)

SET(CMAKE_BUILD_TYPE "Debug")
SET(CMAKE_CXX_FLAGS_DEBUG "$ENV{CXXFLAGS} -O0 -Wall -g -Wno-attributes")
set(CMAKE_CXX_STANDARD 11)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fsanitize=address -fno-omit-frame-pointer")

SET (SRCS parse_config.c)

\# use the library we build, duh.
INCLUDE_DIRECTORIES(${CMAKE_CURRENT_BINARY_DIR}/../${YAJL_DIST_NAME}/include)
LINK_DIRECTORIES(${CMAKE_CURRENT_BINARY_DIR}/../${YAJL_DIST_NAME}/lib)

ADD_EXECUTABLE(parse_config ${SRCS})

#TARGET_LINK_LIBRARIES(parse_config yajl_s)
TARGET_LINK_LIBRARIES(parse_config yajl)

@robohack
Copy link

robohack commented Apr 1, 2024

CMake is the worst abomination, perhaps ever, in the software world, and certainly in the software configuration and construction sphere.

Try my "fork" using BSD Make to build and test:

https://github.com/robohack/yajl

I've already fixed this problem, and I'll be making a new release in the next week or so.

@chuanchang
Copy link

@robohack thanks for your help! I gave a try for your yajl project with bmake on the linux system, the following is my steps, please correct me if I'm wrong, thank you so much!

[root@sweetpig-21 yajl_robohack]# export USE_ASAN=1
[root@sweetpig-21 yajl_robohack]# MAKEOBJDIRPREFIX=$(pwd -P)/build bmake MKDOC=no
===> src (obj)
===> src/yajl (obj)
[Creating objdir /root/yajl_robohack/build/root/yajl_robohack/src/yajl...]
===> doc (obj)
bmake[1]: "/root/yajl_robohack/Makefile.inc" line 372: warning: duplicate script for target "yajl-showenv" ignored
bmake[1]: "/root/yajl_robohack/Makefile.inc" line 372: warning: using previous script for "yajl-showenv" defined here
[Creating objdir /root/yajl_robohack/build/root/yajl_robohack/doc...]
===> reformatter (obj)
[Creating objdir /root/yajl_robohack/build/root/yajl_robohack/reformatter...]
===> verify (obj)
[Creating objdir /root/yajl_robohack/build/root/yajl_robohack/verify...]
===> example (obj)
[Creating objdir /root/yajl_robohack/build/root/yajl_robohack/example...]
===> perf (obj)
[Creating objdir /root/yajl_robohack/build/root/yajl_robohack/perf...]
===> test (obj)
bmake[1]: "/root/yajl_robohack/Makefile.inc" line 372: warning: duplicate script for target "yajl-showenv" ignored
bmake[1]: "/root/yajl_robohack/Makefile.inc" line 372: warning: using previous script for "yajl-showenv" defined here
===> test/api (obj)
[Creating objdir /root/yajl_robohack/build/root/yajl_robohack/test/api...]
===> test/parsing (obj)
[Creating objdir /root/yajl_robohack/build/root/yajl_robohack/test/parsing...]
...ignore...
===> test/api (all)
cc -pipe -g -I/root/yajl_robohack/test/api/../../src   -O2 -std=c99 -pipe -fno-strict-aliasing -fstack-protector-all                   -MD -MF gen-extra-close.d -MT gen-extra-close.o             -c /root/yajl_robohack/test/api/gen-extra-close.c
cc -pipe  -fsanitize=address -Wl,--warn-common          -o gen-extra-close  gen-extra-close.o  -L/root/yajl_robohack/build/root/yajl_robohack/test/api/../../src -lyajl_s
===> test/parsing (all)
cc -pipe -g -I/root/yajl_robohack/test/parsing/../../src   -O2 -std=c99 -pipe -fno-strict-aliasing -fstack-protector-all                   -MD -MF yajl_test.d -MT yajl_test.o             -c /root/yajl_robohack/test/parsing/yajl_test.c
cc -pipe  -fsanitize=address -Wl,--warn-common          -o yajl_test  yajl_test.o  -L/root/yajl_robohack/build/root/yajl_robohack/test/parsing/../../src -lyajl_s
[root@sweetpig-21 yajl_robohack]# ls build/root/yajl_robohack/example/ -lah
total 148K
drwxr-xr-x. 2 root root   85 Apr  2 04:45 .
drwxr-xr-x. 9 root root  100 Apr  2 04:45 ..
-rw-r--r--. 1 root root 1.3K Apr  2 04:45 .depend
-rwxr-xr-x. 1 root root 115K Apr  2 04:45 parse_config
-rw-r--r--. 1 root root 1.3K Apr  2 04:45 parse_config.d
-rw-r--r--. 1 root root  22K Apr  2 04:45 parse_config.o
[root@sweetpig-21 yajl_robohack]# ./build/root/yajl_robohack/example/parse_config {"@\\\n\\\\"

NOTE: the last command will hang forever.

@robohack
Copy link

robohack commented Apr 2, 2024

That's not the right way to run parse_config. It only reads input from standard input and ignores all its command-line parameters. In the way you've run it the program is waiting for you to type the input to it. You can echo the input to it on a pipeline, or redirect it's input from a file.

The regression tests run by bmake regress will do that test (in the example subdirectory). Not with the exact same input, but exercising what was the same bug.

@chuanchang
Copy link

@robohack thank you so much! I can see memory leak now, although it's different from original description, I think it may be we have different configuration for cmake/bmake with sanitizer address ...

ajia@fedora:~/Workspaces/yajl_robohack$ export USE_ASAN=1
ajia@fedora:~/Workspaces/yajl_robohack$ MAKEOBJDIRPREFIX=$(pwd -P)/build bmake regress MKDOC=no
if [  -x /usr/sbin/paxctl ]; then /usr/sbin/paxctl +a /home/ajia/Workspaces/yajl_robohack/build/home/ajia/Workspaces/yajl_robohack/example/parse_config; fi
ulimit -v unlimited && /home/ajia/Workspaces/yajl_robohack/build/home/ajia/Workspaces/yajl_robohack/example/parse_config < /home/ajia/Workspaces/yajl_robohack/example/sample.config
Logging/timeFormat: utc
AddressSanitizer:DEADLYSIGNAL
=================================================================
==25360==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000018 (pc 0x000000401e7b bp 0x000000000000 sp 0x7ffc5ce2dbc0 T0)
==25360==The signal is caused by a READ memory access.
==25360==Hint: address points to the zero page.
    #0 0x401e7b in yajl_object_free /home/ajia/Workspaces/yajl_robohack/src/yajl_tree.c:98
    #1 0x401e7b in yajl_tree_free /home/ajia/Workspaces/yajl_robohack/src/yajl_tree.c:570
    #2 0x4012ab in main /home/ajia/Workspaces/yajl_robohack/example/parse_config.c:66
    #3 0x7f7720c46149 in __libc_start_call_main (/lib64/libc.so.6+0x28149) (BuildId: 788cdd41a15985bf8e0a48d213a46e07d58822df)
    #4 0x7f7720c4620a in __libc_start_main_impl (/lib64/libc.so.6+0x2820a) (BuildId: 788cdd41a15985bf8e0a48d213a46e07d58822df)
    #5 0x401384 in _start (/home/ajia/Workspaces/yajl_robohack/build/home/ajia/Workspaces/yajl_robohack/example/parse_config+0x401384) (BuildId: 82fc4e13a59075766eb18325e9e48a01f185f2e3)

AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV /home/ajia/Workspaces/yajl_robohack/src/yajl_tree.c:98 in yajl_object_free
==25360==ABORTING
*** Error code 1

@robohack
Copy link

robohack commented Apr 3, 2024

If you could provide more details about the context you're testing in, such as commit-id, compiler version, etc., that would be appreciated.

Perhaps open a new issue on my project for it?

@chuanchang
Copy link

Using your robohack/yajl project with https://raw.githubusercontent.com/lloyd/yajl/master/example/parse_config.c, it will get memory leak like above, for more, please see [1].

BTW, it's no any memory leak if we used all of codes from robohack/yajl [2].

[1]

ajia@fedora:~$ cat /etc/redhat-release 
Fedora release 39 (Thirty Nine)
ajia@fedora:~$ rpm -q bmake gcc make kernel
bmake-20230711-2.fc39.x86_64
gcc-13.2.1-6.fc39.x86_64
make-4.4.1-2.fc39.x86_64
kernel-6.6.8-200.fc39.x86_64
ajia@fedora:~$ cd Workspaces/yajl_robohack
ajia@fedora:~/Workspaces/yajl_robohack$ git log -1
commit 082b5986345f1b2962c73263b3e225fd68da6622 (HEAD -> bsdmake, origin/bsdmake, origin/HEAD)
Author: Greg A. Woods <woods@robohack.ca>
Date:   Sun Mar 31 15:42:55 2024 -0700

    example, reformatter, test/parsing, verify: memory leaks are errors!
ajia@fedora:~/Workspaces/yajl_robohack$ cp example/parse_config.c example/parse_config.c.orig
ajia@fedora:~/Workspaces/yajl_robohack$ cp ../yajl.lloyd/example/parse_config.c example/
ajia@fedora:~/Workspaces/yajl_robohack$ export USE_ASAN=1
ajia@fedora:~/Workspaces/yajl_robohack$ MAKEOBJDIRPREFIX=$(pwd -P)/build bmake regress MKDOC=no
===> src
[Creating objdir /home/ajia/Workspaces/yajl_robohack/build/home/ajia/Workspaces/yajl_robohack/src...]
===> src/yajl
[Creating objdir /home/ajia/Workspaces/yajl_robohack/build/home/ajia/Workspaces/yajl_robohack/src/yajl...]
===> doc
bmake[1]: "/home/ajia/Workspaces/yajl_robohack/Makefile.inc" line 372: warning: duplicate script for target "yajl-showenv" ignored
bmake[1]: "/home/ajia/Workspaces/yajl_robohack/Makefile.inc" line 372: warning: using previous script for "yajl-showenv" defined here
[Creating objdir /home/ajia/Workspaces/yajl_robohack/build/home/ajia/Workspaces/yajl_robohack/doc...]
===> reformatter
[Creating objdir /home/ajia/Workspaces/yajl_robohack/build/home/ajia/Workspaces/yajl_robohack/reformatter...]
===> verify
[Creating objdir /home/ajia/Workspaces/yajl_robohack/build/home/ajia/Workspaces/yajl_robohack/verify...]
...ignore...
AddressSanitizer:DEADLYSIGNAL
=================================================================
==19453==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000018 (pc 0x000000401e7b bp 0x000000000000 sp 0x7ffe62dc8c50 T0)
==19453==The signal is caused by a READ memory access.
==19453==Hint: address points to the zero page.
    #0 0x401e7b in yajl_object_free /home/ajia/Workspaces/yajl_robohack/src/yajl_tree.c:98
    #1 0x401e7b in yajl_tree_free /home/ajia/Workspaces/yajl_robohack/src/yajl_tree.c:570
    #2 0x4012ab in main /home/ajia/Workspaces/yajl_robohack/example/parse_config.c:66
    #3 0x7fcaca046149 in __libc_start_call_main (/lib64/libc.so.6+0x28149) (BuildId: 788cdd41a15985bf8e0a48d213a46e07d58822df)
    #4 0x7fcaca04620a in __libc_start_main_impl (/lib64/libc.so.6+0x2820a) (BuildId: 788cdd41a15985bf8e0a48d213a46e07d58822df)
    #5 0x401384 in _start (/home/ajia/Workspaces/yajl_robohack/build/home/ajia/Workspaces/yajl_robohack/example/parse_config+0x401384) (BuildId: 82fc4e13a59075766eb18325e9e48a01f185f2e3)

AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV /home/ajia/Workspaces/yajl_robohack/src/yajl_tree.c:98 in yajl_object_free
==19453==ABORTING
*** Error code 1

Stop.
bmake[1]: stopped in /home/ajia/Workspaces/yajl_robohack/example
*** Error code 1

Stop.
bmake: stopped in /home/ajia/Workspaces/yajl_robohack

[2]

ajia@fedora:~/Workspaces/yajl_robohack$ export USE_ASAN=1
ajia@fedora:~/Workspaces/yajl_robohack$ MAKEOBJDIRPREFIX=$(pwd -P)/build bmake regress MKDOC=no
===> src
[Creating objdir /home/ajia/Workspaces/yajl_robohack/build/home/ajia/Workspaces/yajl_robohack/src...]
===> src/yajl
[Creating objdir /home/ajia/Workspaces/yajl_robohack/build/home/ajia/Workspaces/yajl_robohack/src/yajl...]
===> doc
bmake[1]: "/home/ajia/Workspaces/yajl_robohack/Makefile.inc" line 372: warning: duplicate script for target "yajl-showenv" ignored
bmake[1]: "/home/ajia/Workspaces/yajl_robohack/Makefile.inc" line 372: warning: using previous script for "yajl-showenv" defined here
[Creating objdir /home/ajia/Workspaces/yajl_robohack/build/home/ajia/Workspaces/yajl_robohack/doc...]
===> reformatter
[Creating objdir /home/ajia/Workspaces/yajl_robohack/build/home/ajia/Workspaces/yajl_robohack/reformatter...]
===> verify
...ignore...
===> reformatter
===> verify
===> example
if [  -x /usr/sbin/paxctl ]; then /usr/sbin/paxctl +a /home/ajia/Workspaces/yajl_robohack/build/home/ajia/Workspaces/yajl_robohack/example/parse_config; fi
ulimit -v unlimited && /home/ajia/Workspaces/yajl_robohack/build/home/ajia/Workspaces/yajl_robohack/example/parse_config < /home/ajia/Workspaces/yajl_robohack/example/sample.config
Logging/timeFormat: utc
memory leaks:	0
ulimit -v unlimited && [ $(expr "$(/home/ajia/Workspaces/yajl_robohack/build/home/ajia/Workspaces/yajl_robohack/example/parse_config < /home/ajia/Workspaces/yajl_robohack/example/sample.config 2>&1)" : "Logging/timeFormat: utc.memory leaks:.0") -eq 39 ]
ulimit -v unlimited && echo '{broken:' | /home/ajia/Workspaces/yajl_robohack/build/home/ajia/Workspaces/yajl_robohack/example/parse_config || true
tree_parse_error: lexical error: invalid char in json text.
                                      {broken: 
                    (right here) ------^

memory leaks:	0
ulimit -v unlimited && [ $(expr "$(echo '{broken:' | /home/ajia/Workspaces/yajl_robohack/build/home/ajia/Workspaces/yajl_robohack/example/parse_config 2>&1)" : "tree_parse_error: lexical error: invalid char in json text.*memory leaks:.0") -eq 165 ]
...ignore...
===> test/parsing
if [  -x /usr/sbin/paxctl ]; then /usr/sbin/paxctl +a /home/ajia/Workspaces/yajl_robohack/build/home/ajia/Workspaces/yajl_robohack/test/parsing/yajl_test; fi
ulimit -v unlimited && cd /home/ajia/Workspaces/yajl_robohack/test/parsing && testBin=/home/ajia/Workspaces/yajl_robohack/build/home/ajia/Workspaces/yajl_robohack/test/parsing/yajl_test OBJDIR=/home/ajia/Workspaces/yajl_robohack/build/home/ajia/Workspaces/yajl_robohack/test/parsing sh ./run_tests.sh
using test binary: /home/ajia/Workspaces/yajl_robohack/build/home/ajia/Workspaces/yajl_robohack/test/parsing/yajl_test
 test (ac_difficult_json_c_test_case_with_comments): SUCCESS
 test (ac_simple_with_comments): SUCCESS
 test (ag_false_then_garbage): SUCCESS
 test (ag_null_then_garbage): SUCCESS
 test (ag_true_then_garbage): SUCCESS
 test (am_eof): SUCCESS
...ignore...
 test (string_invalid_escape): SUCCESS
 test (string_invalid_hex_char): SUCCESS
 test (string_with_escapes): SUCCESS
 test (string_with_invalid_newline): SUCCESS
 test (three_byte_utf8): SUCCESS
 test (true): SUCCESS
 test (unescaped_bulgarian): SUCCESS
 test (zerobyte): SUCCESS
59/59 tests successful

@robohack
Copy link

robohack commented Apr 4, 2024

Excellent! Thank you very much!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

7 participants