go web server craps out on second request

[eyberg]

Page fault
interrupt: 000000000000000e
frame: 0000000100200800
error code: 0000000000000000
address: 0000000000000030
rax: 0000000000000000
rbx: 000000000f02a053                                        
rcx: 00000000007f90f8
rdx: 6f6c203a74736f48
rsi: 000000000f02a053
rdi: 000000010100060c                                  
rbp: 000000050023fd30
rsp: 000000050023fc40
r8: 0000000000000001
r9: 0000000000000000
r10: 0000000000000000
r11: 0000000000000006
r12: 0000000100a018e0
r13: 000000010100060c
r14: 0000000000000000
r15: 0000000000000001
rip: 000000000f01c455   (thread_log_internal + 0000000000000071)
flags: 0000000000010006
stack           
00000000007f90f8
0000000000000000
0000000100a018e0
000000010100060c
0000000000000000
000000000f02ce40        (bootstrap_region + 0000000000000a40)
00000000007f90f8
0000000100601080
0000000000000001
000000000f00bf10        (table_find + 0000000000000085)
0000007000000000
00000000007f97c0
000000c420000f00
0000000000000000
0000007000004260
000000c42000e280
0000000000000040
0000000000000040
00000a000023fce0
0000000000000000
0000000000000001
0000000000000000
0000000000000001
00000000007f90f8
6f6c203a74736f48
000000050023fd10
0000000000800e28
0000000000422868
000000050023fca0
000000000f01c465        (thread_log_internal + 0000000000000081)
starting gdb

(gdb) bt
#0  0x00007fd6b7cb4ca6 in ppoll () at /usr/lib/libc.so.6
#1  0x0000557145ff6781 in ppoll (__ss=0x0, __timeout=0x7ffc203afa90, __nfds=<optimized out>, __fds=<optimized out>) at /usr/include/bits/poll2.h:77           
#2  0x0000557145ff6781 in qemu_poll_ns (fds=<optimized out>, nfds=<optimized out>, timeout=timeout@entry=499000000) at qemu-timer.c:324                       
#3  0x0000557145ff60d3 in os_host_main_loop_wait (timeout=<optimized out>) at main-loop.c:250                                                                 
#4  0x0000557145ff60d3 in main_loop_wait (nonblocking=<optimized out>) at main-loop.c:504                                                                     
#5  0x0000557145d7f74c in main_loop () at vl.c:1920
#6  0x0000557145d7f74c in main (argc=<optimized out>, argv=<optimized out>, envp=<optimized out>) at vl.c:4681      

[convolvatron]

i have a reproduction, not sure if the same one...looks like memory exhaustion(?)

#0 0x000000000f011237 in runtime_memcpy (len=835582, b=0x102200000, a=0xffffffffffffffff) at ../runtime/runtime.h:18
#1 buffer_extend (len=1, b=0x10080f300) at ../runtime/buffer.h:65
#2 push_u8 (x=32 ' ', b=0x10080f300) at ../runtime/buffer.h:279
#3 format_spaces (dest=0x10080f300, fmt=, a=) at ../runtime/runtime_init.c:45
#4 0x000000000f00c460 in vbprintf (d=0x10080f300, fmt=fmt@entry=0x10001fb90, ap=ap@entry=0x10001fbd8) at ../runtime/format.c:18
#5 0x000000000f01c576 in bprintf (b=b@entry=0x10080f300, fmt=fmt@entry=0xf02ac32 "%n %d ") at ../runtime/format.h:34
#6 0x000000000f01c703 in thread_log_internal (t=0x100200400, desc=desc@entry=0xf02a7f4 "sleep") at ../unix/thread.c:201
#7 0x000000000f01c866 in thread_sleep (t=) at ../unix/thread.c:225
#8 0x000000000f01a8fb in default_fault_handler (t=0x100200400, frame=) at ../unix/unix.c:40
#9 0x000000000f01c585 in _apply_default_fault_handler (z=, r0=) at ../unix/unix_internal.h:130
#10 0x000000000f015d96 in common_handler () at ../x86_64/interrupt.c:223
#11 0x000000000f016276 in getrip ()
#12 0x000000000f02d720 in bootstrap_region ()
#13 0x000000000f01ad59 in init_unix (h=h@entry=0xf02d720 <bootstrap_region+2624>, pages=pages@entry=0xf02cce0 <bootstrap_region>,
physical=physical@entry=0xf02ce18 <bootstrap_region+312>, virtual=virtual@entry=0xf02d0f0 <bootstrap_region+1040>,
virtual_pagesized=virtual_pagesized@entry=0x100600a00, backed=backed@entry=0x10080eec0, root=0x20000000000, fs=0x100600800)
at ../unix/unix.c:117
#14 0x000000000f00b7f2 in startup (pages=0xf02cce0 <bootstrap_region>, general=0xf02d720 <bootstrap_region+2624>,
physical=0xf02ce18 <bootstrap_region+312>, virtual=0xf02d0f0 <bootstrap_region+1040>, root=0x20000000000, fs=0x100600800)
at stage3.c:34
#15 0x000000000f016579 in _apply_startup (z=) at ../x86_64/service.c:88
#16 0x000000000f0166dd in runloop () at ../x86_64/service.c:65
#17 0x000000000f01687e in init_service_new_stack (virtual_pagesized=virtual_pagesized@entry=0xf02d3c8 <bootstrap_region+1768>)
at ../x86_64/service.c:174
#18 0x000000000f016c07 in init_service () at ../x86_64/service.c:245
#19 0x0000000000000000 in ?? ()

[eyberg]

hrm.. i reliably get this on the 2nd request sent

[convolvatron]

what are you sending your requests with?

[eyberg]

apache bench, telnet, curl - they all seem to reliably crash on the 2nd request

[convolvatron]

closing this particular one since a fix got committed...this was the queue reuse bug. there are still lots of issues, but we should track them by syndrome