segfault caused by out_stdout

[nokute78]

I tested 15047b7 and Segfault occurs.
(By the way, Default log level is debug in that commit. Is it intentional?)

How to reproduce

• $ fluent-bit -i cpu -o stdout
• wait a minute

On the other hand, segfault is not happened when output is out_null.

gdb log

It is caused by src/flb_task.c.

Program received signal SIGSEGV, Segmentation fault.
0x0000000000417e53 in flb_task_retry_clean (task=0x6d3f90, data=0x3a3298f040)
    at /home/taka/git/oss/pull_req/fluentbit_env/fluent-bit/src/flb_task.c:124
124     o_ins = out_th->o_ins;

Full log is

$ gdb bin/fluent-bit 
GNU gdb (GDB) Red Hat Enterprise Linux (7.2-90.el6)
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /home/taka/git/oss/pull_req/fluentbit_env/fluent-bit/build/bin/fluent-bit...done.
(gdb) run -i cpu -o stdout -f 1
Starting program: /home/taka/git/oss/pull_req/fluentbit_env/fluent-bit/build/bin/fluent-bit -i cpu -o stdout -f 1
[Thread debugging using libthread_db enabled]
[New Thread 0x2aaaaacc1700 (LWP 27777)]
Fluent-Bit v0.9.0
Copyright (C) Treasure Data

[2016/10/03 22:07:00] [ info] [engine] started
[2016/10/03 22:07:00] [debug] [router] default match rule cpu.0:stdout.0
[2016/10/03 22:07:02] [debug] [task] created task=0x6d3f90 OK
[0] cpu.0: [1475500021, {"cpu_p"=>4.000000, "user_p"=>1.000000, "system_p"=>3.000000, "cpu0.p_cpu"=>4.000000, "cpu0.p_user"=>1.000000, "cpu0.p_system"=>3.000000}]
[2016/10/03 22:07:03] [debug] [task] created task=0x6d41f0 OK
[0] cpu.0: [1475500022, {"cpu_p"=>0.000000, "user_p"=>0.000000, "system_p"=>0.000000, "cpu0.p_cpu"=>0.000000, "cpu0.p_user"=>0.000000, "cpu0.p_system"=>0.000000}]
[2016/10/03 22:07:04] [debug] [task] created task=0x6d4320 OK
[0] cpu.0: [1475500023, {"cpu_p"=>2.000000, "user_p"=>1.000000, "system_p"=>1.000000, "cpu0.p_cpu"=>2.000000, "cpu0.p_user"=>1.000000, "cpu0.p_system"=>1.000000}]
[2016/10/03 22:07:05] [debug] [task] created task=0x6d4470 OK
[0] cpu.0: [1475500024, {"cpu_p"=>1.000000, "user_p"=>1.000000, "system_p"=>0.000000, "cpu0.p_cpu"=>1.000000, "cpu0.p_user"=>1.000000, "cpu0.p_system"=>0.000000}]
[2016/10/03 22:07:06] [debug] [task] created task=0x6d45e0 OK
[0] cpu.0: [1475500025, {"cpu_p"=>4.000000, "user_p"=>2.000000, "system_p"=>2.000000, "cpu0.p_cpu"=>4.000000, "cpu0.p_user"=>2.000000, "cpu0.p_system"=>2.000000}]

Program received signal SIGSEGV, Segmentation fault.
0x0000000000417e53 in flb_task_retry_clean (task=0x6d3f90, data=0x3a3298f040)
    at /home/taka/git/oss/pull_req/fluentbit_env/fluent-bit/src/flb_task.c:124
124     o_ins = out_th->o_ins;
Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.192.el6.x86_64 zlib-1.2.3-29.el6.x86_64
(gdb) 

[nokute78]

On the other hand, segfault is not happened when output is out_null.

It is not right. Segfault also occurs when output plugin is out_null.

[edsiper]

Thanks for the report. I was able to reproduce it, will do some troubleshooting late today.

[edsiper]

root cause: when the thread is destroyed, it was not decrementing the task->users count, so the task always remain and generated conflicts.

[edsiper]

@nokute78 I will keep this ticket open until final confirmation.

[nokute78]

@edsiper Thank you for fixing!
The issue is fixed with 08c71aa in my environment.

[edsiper]

Awesome :)

[edsiper]

for the record.. about:

(By the way, Default log level is debug in that commit. Is it intentional?)

no, it was the pending implementation for 'logging worker support' into the main thread. Now it's fixed in master by 14f5eb0

[nokute78]

Thanks!
Default log level is fixed.