Segment fault bug

[yaoweibin]

I encountered segfault with the ab tests ( ab -n 10000 -c 10 http://127.0.0.1:1980/test/www/test.html). This is the backtrace:
#0 0x00000000004fad45 in net_instaweb::RewriteDriver::DeleteRewriteContext(net_instaweb::RewriteContext*) ()

(gdb) bt
#0 0x00000000004fad45 in net_instaweb::RewriteDriver::DeleteRewriteContext(net_instaweb::RewriteContext*) ()
#1 0x0000000000582ea2 in net_instaweb::Function::CallRun() ()
#2 0x0000000000592e8c in net_instaweb::QueuedWorkerPool::Run(net_instaweb::QueuedWorkerPool::Sequence_, net_instaweb::QueuedWorker_) ()
#3 0x0000000000582ea2 in net_instaweb::Function::CallRun() ()
#4 0x00000000005a6d86 in net_instaweb::Worker::WorkThread::Run() ()
#5 0x00000000005b2ec8 in net_instaweb::PthreadThreadImpl::InvokeRun(void*) ()
#6 0x00000036f6e0677d in start_thread () from /lib64/libpthread.so.0
#7 0x00000036f66d49ad in clone () from /lib64/libc.so.6

[yaoweibin]

Other backtrace:

#0 0x00000036f66705ed in malloc_consolidate () from /lib64/libc.so.6
(gdb) bt
#0 0x00000036f66705ed in malloc_consolidate () from /lib64/libc.so.6
#1 0x00000036f6672bdc in _int_malloc () from /lib64/libc.so.6
#2 0x00000036f6674e4e in malloc () from /lib64/libc.so.6
#3 0x00000036f667528b in posix_memalign () from /lib64/libc.so.6
#4 0x0000000000436ccc in ngx_memalign (alignment=16, size=46912585782320, log=0x2aaab003f6b0) at src/os/unix/ngx_alloc.c:74
#5 0x000000000041d87e in ngx_palloc_block (pool=0x2aaab001e310, size=450) at src/core/ngx_palloc.c:185
#6 0x000000000041daa4 in ngx_palloc (pool=0x2aaab0000020, size=4096) at src/core/ngx_palloc.c:139
#7 0x00000000004bb42a in net_instaweb::NgxBaseFetch::CopyBufferToNginx (this=0x2aaab0023fe0, link_ptr=0x7fffe1720430)
at /home/wenjing.ywb/test/pagespeed/ngx_pagespeed/src/ngx_base_fetch.cc:137
#8 0x00000000004bb792 in net_instaweb::NgxBaseFetch::CollectAccumulatedWrites (this=0x2aaab0023fe0, link_ptr=0x7fffe1720430)
at /home/wenjing.ywb/test/pagespeed/ngx_pagespeed/src/ngx_base_fetch.cc:189
#9 0x00000000004b3213 in ngx_http_pagespeed_update (ev=0x19ab9240) at /home/wenjing.ywb/test/pagespeed/ngx_pagespeed/src/ngx_pagespeed.cc:361
#10 ngx_http_pagespeed_connection_read_handler (ev=0x19ab9240) at /home/wenjing.ywb/test/pagespeed/ngx_pagespeed/src/ngx_pagespeed.cc:454
#11 0x000000000043d335 in ngx_epoll_process_events (cycle=0x19a84410, timer=, flags=)
at src/event/modules/ngx_epoll_module.c:683
#12 0x00000000004338e8 in ngx_process_events_and_timers (cycle=0x19a84410) at src/event/ngx_event.c:249
#13 0x000000000043ab51 in ngx_single_process_cycle (cycle=0x19a84410) at src/os/unix/ngx_process_cycle.c:348
#14 0x000000000041c94d in main (argc=3, argv=0x19a82ae0) at src/core/nginx.c:428

Program terminated with signal 11, Segmentation fault.
#0 0x0000000000815f3d in net_instaweb::RewriteContext::OutputCacheDone(net_instaweb::RewriteContext::CacheLookupResult_) ()
(gdb) bt
#0 0x0000000000815f3d in net_instaweb::RewriteContext::OutputCacheDone(net_instaweb::RewriteContext::CacheLookupResult_) ()
#1 0x0000000000582ea2 in net_instaweb::Function::CallRun() ()
#2 0x0000000000592e8c in net_instaweb::QueuedWorkerPool::Run(net_instaweb::QueuedWorkerPool::Sequence_, net_instaweb::QueuedWorker_) ()
#3 0x0000000000582ea2 in net_instaweb::Function::CallRun() ()
#4 0x00000000005a6d86 in net_instaweb::Worker::WorkThread::Run() ()
#5 0x00000000005b2ec8 in net_instaweb::PthreadThreadImpl::InvokeRun(void*) ()
#6 0x00000036f6e0677d in start_thread () from /lib64/libpthread.so.0
#7 0x00000036f66d49ad in clone () from /lib64/libc.so.6

[bnoordhuis]

@yaoweibin Is this still an issue with the current HEAD? It seems pretty stable for me.

[yaoweibin]

I got a segment fault after I ran the 'nginx_system_test.sh script' several times.

Program terminated with signal 11, Segmentation fault.
#0 0x00000036f667c3bb in memcpy () from /lib64/libc.so.6
(gdb) bt
#0 0x00000036f667c3bb in memcpy () from /lib64/libc.so.6
#1 0x00000000004c9060 in base::StringPiece::copy(char*, unsigned long, unsigned long) const ()
#2 0x00000000004b48de in ngx_http_pagespeed_string_piece_to_buffer_chain (pool=0x18c31410, sp=..., link_ptr=0x7ffffc5f39c0, send_last_buf=false)
at /home/wenjing.ywb/test/pagespeed/ngx_pagespeed/src/ngx_pagespeed.cc:136
#3 0x00000000004c0d07 in net_instaweb::NgxBaseFetch::CopyBufferToNginx (this=0x18c26c80, link_ptr=0x1e7)
at /home/wenjing.ywb/test/pagespeed/ngx_pagespeed/src/ngx_base_fetch.cc:101
#4 0x00000000004c0ede in net_instaweb::NgxBaseFetch::CollectAccumulatedWrites (this=0x18c26c80, link_ptr=0x7ffffc5f39c0)
at /home/wenjing.ywb/test/pagespeed/ngx_pagespeed/src/ngx_base_fetch.cc:121
#5 0x00000000004b5123 in ngx_http_pagespeed_update (ev=0x18bc73a0) at /home/wenjing.ywb/test/pagespeed/ngx_pagespeed/src/ngx_pagespeed.cc:549
#6 (anonymous namespace)::ngx_http_pagespeed_connection_read_handler (ev=0x18bc73a0)
at /home/wenjing.ywb/test/pagespeed/ngx_pagespeed/src/ngx_pagespeed.cc:642
#7 0x000000000043e955 in ngx_epoll_process_events (cycle=0x18b8f4b0, timer=, flags=)
at src/event/modules/ngx_epoll_module.c:683
#8 0x0000000000434e98 in ngx_process_events_and_timers (cycle=0x18b8f4b0) at src/event/ngx_event.c:249
#9 0x000000000043c171 in ngx_single_process_cycle (cycle=0x18b8f4b0) at src/os/unix/ngx_process_cycle.c:348
#10 0x000000000041ddcd in main (argc=3, argv=0x18b8dae0) at src/core/nginx.c:428

[jeffkaufman]

@yaoweibin do you still see segfaults when running with the current code?

[jeffkaufman]

related: #82

[yaoweibin]

No, in my simple stressless tests. It's OK. I'll close this issue.

[songshaodong]

I also met the issue, is program code cause ?

[oschaaf]

@songshaodong Could you post a backtrace?

[songshaodong]

@oschaaf Thanks, My problem has been resolved, memory overflow cause.

backtrace:

#0 0x00000032a960cd2b in pthread_once () from /lib64/libpthread.so.0
#1 0x00000032a8efe954 in backtrace () from /lib64/libc.so.6
#2 0x00000032a8e7054b in __libc_message () from /lib64/libc.so.6
#3 0x00000032a8e75e66 in malloc_printerr () from /lib64/libc.so.6
#4 0x00000032a8e79904 in _int_malloc () from /lib64/libc.so.6
#5 0x00000032a8e7a346 in calloc () from /lib64/libc.so.6
#6 0x00000032a8a0addf in _dl_new_object () from /lib64/ld-linux-x86-64.so.2
#7 0x00000032a8a071de in _dl_map_object_from_fd () from /lib64/ld-linux-x86-64.so.2
#8 0x00000032a8a0839a in _dl_map_object () from /lib64/ld-linux-x86-64.so.2
#9 0x00000032a8a12b05 in dl_open_worker () from /lib64/ld-linux-x86-64.so.2
#10 0x00000032a8a0e266 in _dl_catch_error () from /lib64/ld-linux-x86-64.so.2
#11 0x00000032a8a125aa in _dl_open () from /lib64/ld-linux-x86-64.so.2
#12 0x00000032a8f26b20 in do_dlopen () from /lib64/libc.so.6
#13 0x00000032a8a0e266 in _dl_catch_error () from /lib64/ld-linux-x86-64.so.2
#14 0x00000032a8f26c77 in __libc_dlopen_mode () from /lib64/libc.so.6
#15 0x00000032a8efe825 in init () from /lib64/libc.so.6
#16 0x00000032a960cd33 in pthread_once () from /lib64/libpthread.so.0
#17 0x00000032a8efe954 in backtrace () from /lib64/libc.so.6
#18 0x00000032a8e7054b in __libc_message () from /lib64/libc.so.6
#19 0x00000032a8e75e66 in malloc_printerr () from /lib64/libc.so.6
#20 0x00000032a8e79904 in _int_malloc () from /lib64/libc.so.6
#21 0x00000032a8e7a6b1 in malloc () from /lib64/libc.so.6
#22 0x0000000000434ddc in ngx_alloc ()
#23 0x000000000040ac49 in ngx_palloc_large ()
#24 0x000000000040aa3f in ngx_palloc ()
#25 0x000000000040ae88 in ngx_pcalloc ()
#26 0x00000000004e68f3 in ngx_rtmp_init_session ()
#27 0x00000000004e6640 in ngx_rtmp_init_connection ()
#28 0x0000000000430ee3 in ngx_event_accept ()
#29 0x000000000043d86d in ngx_epoll_process_events ()
#30 0x000000000042d2f8 in ngx_process_events_and_timers ()
#31 0x000000000043b3ca in ngx_worker_process_cycle ()
#32 0x0000000000437c66 in ngx_spawn_process ()
#33 0x000000000043a232 in ngx_start_worker_processes ()
#34 0x0000000000439dd5 in ngx_master_process_cycle ()
#35 0x0000000000407b96 in main ()