Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

upon upgrade to v5, FTL keeps crashing every few hours (0x74617300 with code: SEGV_MAPERR) #804

Closed
3 tasks done
yik77 opened this issue Jun 6, 2020 · 22 comments · Fixed by #827
Closed
3 tasks done

Comments

@yik77
Copy link

yik77 commented Jun 6, 2020

In raising this issue, I confirm the following (please check boxes, eg [X]) Failure to fill the template will close your issue:

  • I have read and understood the contributors guide.
  • The issue I am reporting can be replicated
  • The issue I am reporting isn't a duplicate

How familiar are you with the codebase?:

{1}


[BUG | ISSUE] Expected Behaviour:
pihole running uninterrupted for hours, days, weeks...
[BUG | ISSUE] Actual Behaviour:
pihole keeps crashing every few hours, checking of pihole-FTL.log shows Segmentation fault at address: 0x74617300 with code: SEGV_MAPERR (Address not mapped to object) and asked me to report it.

[BUG | ISSUE] Steps to reproduce:

Installed on fully updated raspbian buster on r pi zero w. Upgraded to v5. started. crashed.
seg fault at 0x74617300 with code: SEGV_MAPERR.
command pihole restartdns makes it work again, for few hours.

Log file output [if available]

[2020-06-05 19:17:57.787 15838] Compiled 0 whitelist and 0 blacklist regex filters in 286.6 msec
[2020-06-05 19:22:33.936 15838] Resizing "/FTL-dns-cache" from 4096 to 8192
[2020-06-05 19:22:49.402 15838] Resizing "/FTL-queries" from 2523136 to 2752512
[2020-06-05 19:22:56.688 15838] Resizing "/FTL-strings" from 69632 to 73728
[2020-06-05 19:43:02.638 15838] Resizing "/FTL-dns-cache" from 8192 to 12288
[2020-06-05 19:55:38.040 15838] Resizing "/FTL-dns-cache" from 12288 to 16384
[2020-06-05 19:55:40.155 15838] Resizing "/FTL-strings" from 73728 to 77824
[2020-06-05 19:58:20.665 15838] Resizing "/FTL-dns-cache" from 16384 to 20480
[2020-06-05 20:00:36.376 15838] !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
[2020-06-05 20:00:36.376 15838] ----------------------------> FTL crashed! <----------------------------
[2020-06-05 20:00:36.377 15838] !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
[2020-06-05 20:00:36.377 15838] Please report a bug at https://github.com/pi-hole/FTL/issues
[2020-06-05 20:00:36.377 15838] and include in your report already the following details:
[2020-06-05 20:00:36.377 15838] FTL has been running for 2566 seconds
[2020-06-05 20:00:36.378 15838] FTL branch: master
[2020-06-05 20:00:36.378 15838] FTL version: v5.0
[2020-06-05 20:00:36.378 15838] FTL commit: 3d7c095
[2020-06-05 20:00:36.378 15838] FTL date: 2020-05-10 18:58:38 +0100
[2020-06-05 20:00:36.379 15838] FTL user: started as pihole, ended as pihole
[2020-06-05 20:00:36.379 15838] Compiled for arm (compiled on CI) using arm-linux-gnueabihf-gcc (crosstool-NG crosstool-ng-1.22.0-88-g8460611) 4.9.3
[2020-06-05 20:00:36.379 15838] Received signal: Segmentation fault
[2020-06-05 20:00:36.380 15838] at address: 0x74617300
[2020-06-05 20:00:36.380 15838] with code: SEGV_MAPERR (Address not mapped to object)
[2020-06-05 20:00:36.380 15838] Backtrace:
[2020-06-05 20:00:36.382 15838] B[0000]: 0x46d704, /usr/bin/pihole-FTL(+0x2c704) [0x46d704]
[2020-06-05 20:00:36.382 15838] B[0001]: 0xb6e22130, /lib/arm-linux-gnueabihf/libc.so.6(__default_rt_sa_restorer+0) [0xb6e22130]
[2020-06-05 20:00:36.382 15838] B[0002]: 0x4a81b4, /usr/bin/pihole-FTL(iface_check+0x50) [0x4a81b4]
[2020-06-05 20:00:36.383 15838] B[0003]: 0x48d4e0, /usr/bin/pihole-FTL(receive_query+0x268) [0x48d4e0]
[2020-06-05 20:00:36.383 15838] B[0004]: 0x4a3d68, /usr/bin/pihole-FTL(+0x62d68) [0x4a3d68]
[2020-06-05 20:00:36.383 15838] B[0005]: 0x4a5cfc, /usr/bin/pihole-FTL(main_dnsmasq+0x1238) [0x4a5cfc]
[2020-06-05 20:00:36.383 15838] B[0006]: 0x45fd04, /usr/bin/pihole-FTL(main+0xfc) [0x45fd04]
[2020-06-05 20:00:36.384 15838] B[0007]: 0xb6e0c718, /lib/arm-linux-gnueabihf/libc.so.6(__libc_start_main+0x10c) [0xb6e0c718]
[2020-06-05 20:00:36.384 15838] ------ Listing content of directory /dev/shm ------
[2020-06-05 20:00:36.384 15838] File Mode User:Group Filesize Filename
[2020-06-05 20:00:36.385 15838] rwxrwxrwx root:root 260 .
[2020-06-05 20:00:36.385 15838] rwxr-xr-x root:root 4K ..
[2020-06-05 20:00:36.386 15838] rw------- pihole:pihole 4K FTL-per-client-regex
[2020-06-05 20:00:36.387 15838] rw------- pihole:pihole 20K FTL-dns-cache
[2020-06-05 20:00:36.388 15838] rw------- pihole:pihole 29K FTL-overTime
[2020-06-05 20:00:36.389 15838] rw------- pihole:pihole 3M FTL-queries
[2020-06-05 20:00:36.389 15838] rw------- pihole:pihole 20K FTL-upstreams
[2020-06-05 20:00:36.390 15838] rw------- pihole:pihole 643K FTL-clients
[2020-06-05 20:00:36.391 15838] rw------- pihole:pihole 66K FTL-domains
[2020-06-05 20:00:36.391 15838] rw------- pihole:pihole 78K FTL-strings
[2020-06-05 20:00:36.392 15838] rw------- pihole:pihole 12 FTL-settings
[2020-06-05 20:00:36.393 15838] rw------- pihole:pihole 124 FTL-counters
[2020-06-05 20:00:36.393 15838] rw------- pihole:pihole 28 FTL-lock
[2020-06-05 20:00:36.394 15838] ---------------------------------------------------
[2020-06-05 20:00:36.394 15838] Thank you for helping us to improve our FTL engine!
[2020-06-05 20:00:36.394 15838] FTL terminated!

Device specifics

Hardware Type: rPi zero w
OS: raspbian buster fully updated

This template was created based on the work of udemy-dl.

@DL6ER
Copy link
Member

DL6ER commented Jun 6, 2020

Thank you for your submission and we're sorry you're seeing this. With Pi-hole v5.0, we decided to embed the top recent release of dnsmasq v2.81 (release only a few days before Pi-hole v5.0) and this turns out to have been a ... mildly wise decision if you want to put it like that. Since then, we're seeing reports about failures in dnsmasq in a few places.

Can you attach the debugger to your Pi-hole and post the results? Everything is well described in our docs: https://docs.pi-hole.net/ftldns/debugging/

Only with this further information we'll be able to fix this bug.

If you need any assistance or have any more questions, feel free to ask!

@DL6ER
Copy link
Member

DL6ER commented Jun 8, 2020

@yik77 In case you cannot attach the debugger, please run

pihole checkout ftl fix/master-in_zone-and-iface_check_debugging

Even when this does not fix the issue, it should give us more details in the log file just before a crash happens.

The DEBUG: lines will be interesting, also those during startup like

[2020-06-08 09:16:02.153 8515] DEBUG option::interface arg 1 = 0x55555594970a 'enp2s0'
[2020-06-08 09:16:02.153 8515] DEBUG option::interface comma = (nil) '(null)'       
[2020-06-08 09:16:02.153 8515] DEBUG option::interface arg 2 = 0x55555594970a 'enp2s0'
[2020-06-08 09:16:02.153 8515] DEBUG option::interface new->name = 0x555555960c40 'enp2s0'
[2020-06-08 09:16:02.154 8515] DEBUG iface_allowed: daemon->if_names->name = 0x55555594f030 'lo'

@fradev
Copy link

fradev commented Jun 15, 2020

2020-06-15 15:21:30.416 15873] !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
[2020-06-15 15:21:30.416 15873] ----------------------------> FTL crashed! <----------------------------
[2020-06-15 15:21:30.416 15873] !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
[2020-06-15 15:21:30.416 15873] Please report a bug at https://github.com/pi-hole/FTL/issues
[2020-06-15 15:21:30.416 15873] and include in your report already the following details:
[2020-06-15 15:21:30.416 15873] FTL has been running for 1835 seconds
[2020-06-15 15:21:30.416 15873] FTL branch: fix/master-in_zone-and-iface_check_debugging
[2020-06-15 15:21:30.417 15873] FTL version: vDev-f2035de
[2020-06-15 15:21:30.417 15873] FTL commit: f2035de
[2020-06-15 15:21:30.417 15873] FTL date: 2020-06-10 09:43:13 +0200
[2020-06-15 15:21:30.418 15873] FTL user: started as pihole, ended as pihole
[2020-06-15 15:21:30.418 15873] Compiled for aarch64 (compiled on CI) using aarch64-linux-gnu-gcc (Debian 6.3.0-18) 6.3.0 20170516
[2020-06-15 15:21:30.418 15873] Received signal: Segmentation fault
[2020-06-15 15:21:30.418 15873] at address: 0x72610000aaaa36
[2020-06-15 15:21:30.418 15873] with code: SEGV_MAPERR (Address not mapped to object)
[2020-06-15 15:21:30.419 15873] Backtrace:
[2020-06-15 15:21:30.419 15873] B[0000]: 0xaaaaca0aa2b4, /usr/bin/pihole-FTL(+0x322b4) [0xaaaaca0aa2b4]
[2020-06-15 15:21:30.421 15873] B[0001]: 0xffff859aa688, linux-vdso.so.1(__kernel_rt_sigreturn+0) [0xffff859aa688]
[2020-06-15 15:21:30.421 15873] B[0002]: 0xaaaaca0d1c90, /usr/bin/pihole-FTL(answer_request+0xd8) [0xaaaaca0d1c90]
[2020-06-15 15:21:30.423 15873] B[0003]: 0xaaaaca0c4e90, /usr/bin/pihole-FTL(tcp_request+0x540) [0xaaaaca0c4e90]
[2020-06-15 15:21:30.424 15873] B[0004]: 0xaaaaca0db074, /usr/bin/pihole-FTL(+0x63074) [0xaaaaca0db074]
[2020-06-15 15:21:30.426 15873] B[0005]: 0xaaaaca0dcb08, /usr/bin/pihole-FTL(main_dnsmasq+0xe00) [0xaaaaca0dcb08]
[2020-06-15 15:21:30.428 15873] B[0006]: 0xaaaaca09d738, /usr/bin/pihole-FTL(main+0xf0) [0xaaaaca09d738]
[2020-06-15 15:21:30.430 15873] B[0007]: 0xffff857e3d24, /lib/aarch64-linux-gnu/libc.so.6(__libc_start_main+0xe4) [0xffff857e3d24]
[2020-06-15 15:21:30.430 15873] B[0008]: 0xaaaaca09d8cc, /usr/bin/pihole-FTL(+0x258cc) [0xaaaaca09d8cc]
[2020-06-15 15:21:30.432 15873] ------ Listing content of directory /dev/shm ------
[2020-06-15 15:21:30.432 15873] File Mode User:Group Filesize Filename
[2020-06-15 15:21:30.432 15873] rwxrwxrwx root:root 260 .
[2020-06-15 15:21:30.432 15873] rwxr-xr-x root:root 3K ..
[2020-06-15 15:21:30.432 15873] rw------- pihole:pihole 4K FTL-per-client-regex
[2020-06-15 15:21:30.433 15873] rw------- pihole:pihole 20K FTL-dns-cache
[2020-06-15 15:21:30.433 15873] rw------- pihole:pihole 12K FTL-overTime
[2020-06-15 15:21:30.433 15873] rw------- pihole:pihole 5M FTL-queries
[2020-06-15 15:21:30.433 15873] rw------- pihole:pihole 4K FTL-upstreams
[2020-06-15 15:21:30.434 15873] rw------- pihole:pihole 20K FTL-clients
[2020-06-15 15:21:30.434 15873] rw------- pihole:pihole 197K FTL-domains
[2020-06-15 15:21:30.434 15873] rw------- pihole:pihole 274K FTL-strings
[2020-06-15 15:21:30.434 15873] rw------- pihole:pihole 12 FTL-settings
[2020-06-15 15:21:30.435 15873] rw------- pihole:pihole 124 FTL-counters
[2020-06-15 15:21:30.435 15873] rw------- pihole:pihole 56 FTL-lock
[2020-06-15 15:21:30.435 15873] ---------------------------------------------------
[2020-06-15 15:21:30.435 15873] Thank you for helping us to improve our FTL engine!
[2020-06-15 15:21:30.435 15873] FTL terminated!

DO not know if this helps . Let me know if you need the full file

@DL6ER
Copy link
Member

DL6ER commented Jun 15, 2020

@fradev A few lines before the crash will be interesting. Otherwise, please try

pihole checkout ftl development

as we fixed a few small bugs in FTL and it would be great if our fixes already covers the crash you saw.

@fradev
Copy link

fradev commented Jun 15, 2020

here we go

[2020-06-15 15:21:29.432 18198] DEBUG really_insert: cache_get_name(new) = 0xffff8509e628 'api-global.eu-west-1.prodaa.netflix.com'
[2020-06-15 15:21:29.432 18198] DEBUG really_insert: name = 0xaaaad0dbea30 'central.crashplan.com'
[2020-06-15 15:21:29.432 18198] DEBUG really_insert: new = 0xffff8509faf0
[2020-06-15 15:21:29.432 18198] DEBUG really_insert: new = 0x626f6c672d697061 '(null)'
[2020-06-15 15:21:29.432 18198] DEBUG really_insert: cache_get_name(new) = 0xffff8509fb28 'api-global.eu-west-1.prodaa.netflix.com'
[2020-06-15 15:21:29.433 18198] DEBUG really_insert: name = 0xaaaad0dbea30 'central.crashplan.com'
[2020-06-15 15:21:29.433 18198] DEBUG really_insert: new = 0xffff85092960
[2020-06-15 15:21:29.433 18198] DEBUG really_insert: new = 0x696c622e636e7973 '(null)'
[2020-06-15 15:21:29.433 18198] DEBUG really_insert: cache_get_name(new) = 0xffff85092998 'sync.blisspointmedia.com'
[2020-06-15 15:21:29.433 18198] DEBUG really_insert: name = 0xaaaad0dbea30 'central.crashplan.com'
[2020-06-15 15:21:29.433 18198] DEBUG really_insert: new = 0xffff8509e820
[2020-06-15 15:21:29.433 18198] DEBUG really_insert: new = 0x2e6c2e736c6f6f74 '(null)'
[2020-06-15 15:21:29.433 18198] DEBUG really_insert: cache_get_name(new) = 0xffff8509e858 'tools.l.google.com'
[2020-06-15 15:21:29.433 18198] DEBUG really_insert: name = 0xaaaad0dbea30 'central.crashplan.com'
[2020-06-15 15:21:29.433 18198] DEBUG really_insert: new = 0xffff85094560
[2020-06-15 15:21:29.433 18198] DEBUG really_insert: new = 0x612e68746c616568 '(null)'
[2020-06-15 15:21:29.433 18198] DEBUG really_insert: cache_get_name(new) = 0xffff85094598 'health.apple.com'
[2020-06-15 15:21:29.443 15873] DEBUG check_dns_listeners: listener = 0xaaaad0dc4fa0
[2020-06-15 15:21:29.443 15873] DEBUG check_dns_listeners: &listener->family = 0xaaaad0dc4fac
[2020-06-15 15:21:29.443 15873] DEBUG check_dns_listeners: listener->family = 2
[2020-06-15 15:21:29.443 15873] DEBUG check_dns_listeners: &listener->fd = 0xaaaad0dc4fa0
[2020-06-15 15:21:29.443 15873] DEBUG check_dns_listeners: listener->fd = 6
[2020-06-15 15:21:29.443 15873] DEBUG check_dns_listeners: listener->iface = (nil)
[2020-06-15 15:21:29.443 15873] DEBUG check_dns_listeners: listener = 0xaaaad0dc4fd0
[2020-06-15 15:21:29.443 15873] DEBUG check_dns_listeners: &listener->family = 0xaaaad0dc4fdc
[2020-06-15 15:21:29.443 15873] DEBUG check_dns_listeners: listener->family = 10
[2020-06-15 15:21:29.443 15873] DEBUG check_dns_listeners: &listener->fd = 0xaaaad0dc4fd0
[2020-06-15 15:21:29.443 15873] DEBUG check_dns_listeners: listener->fd = 8
[2020-06-15 15:21:29.443 15873] DEBUG check_dns_listeners: listener->iface = (nil)
[2020-06-15 15:21:29.443 15873] DEBUG check_dns_listeners: listener = 0xaaaad0dc4fa0
[2020-06-15 15:21:29.443 15873] DEBUG check_dns_listeners: &listener->family = 0xaaaad0dc4fac
[2020-06-15 15:21:29.443 15873] DEBUG check_dns_listeners: listener->family = 2
[2020-06-15 15:21:29.443 15873] DEBUG check_dns_listeners: &listener->fd = 0xaaaad0dc4fa0
[2020-06-15 15:21:29.443 15873] DEBUG check_dns_listeners: listener->fd = 6
[2020-06-15 15:21:29.443 15873] DEBUG check_dns_listeners: listener->iface = (nil)
[2020-06-15 15:21:29.443 15873] DEBUG check_dns_listeners: listener = 0xaaaad0dc4fd0
[2020-06-15 15:21:29.443 15873] DEBUG check_dns_listeners: &listener->family = 0xaaaad0dc4fdc
[2020-06-15 15:21:29.443 15873] DEBUG check_dns_listeners: listener->family = 10
[2020-06-15 15:21:29.443 15873] DEBUG check_dns_listeners: &listener->fd = 0xaaaad0dc4fd0
[2020-06-15 15:21:29.444 15873] DEBUG check_dns_listeners: listener->fd = 8
[2020-06-15 15:21:29.444 15873] DEBUG check_dns_listeners: listener->iface = (nil)
[2020-06-15 15:21:29.444 15873] DEBUG check_dns_listeners: listener = 0xaaaad0dc4fa0
[2020-06-15 15:21:29.444 15873] DEBUG check_dns_listeners: &listener->family = 0xaaaad0dc4fac
[2020-06-15 15:21:29.444 15873] DEBUG check_dns_listeners: listener->family = 2
[2020-06-15 15:21:29.444 15873] DEBUG check_dns_listeners: &listener->fd = 0xaaaad0dc4fa0
[2020-06-15 15:21:29.444 15873] DEBUG check_dns_listeners: listener->fd = 6
[2020-06-15 15:21:29.444 15873] DEBUG check_dns_listeners: listener->iface = (nil)
[2020-06-15 15:21:29.444 15873] DEBUG check_dns_listeners: listener = 0xaaaad0dc4fd0
[2020-06-15 15:21:29.444 15873] DEBUG check_dns_listeners: &listener->family = 0xaaaad0dc4fdc
[2020-06-15 15:21:29.444 15873] DEBUG check_dns_listeners: listener->family = 10
[2020-06-15 15:21:29.444 15873] DEBUG check_dns_listeners: &listener->fd = 0xaaaad0dc4fd0
[2020-06-15 15:21:29.444 15873] DEBUG check_dns_listeners: listener->fd = 8
[2020-06-15 15:21:29.444 15873] DEBUG check_dns_listeners: listener->iface = (nil)
[2020-06-15 15:21:29.475 15873] DEBUG check_dns_listeners: listener = 0xaaaad0dc4fa0
[2020-06-15 15:21:29.475 15873] DEBUG check_dns_listeners: &listener->family = 0xaaaad0dc4fac
[2020-06-15 15:21:29.475 15873] DEBUG check_dns_listeners: listener->family = 2
[2020-06-15 15:21:29.475 15873] DEBUG check_dns_listeners: &listener->fd = 0xaaaad0dc4fa0
[2020-06-15 15:21:29.475 15873] DEBUG check_dns_listeners: listener->fd = 6
[2020-06-15 15:21:29.475 15873] DEBUG check_dns_listeners: listener->iface = (nil)
[2020-06-15 15:21:29.475 15873] DEBUG check_dns_listeners: listener = 0xaaaad0dc4fd0
[2020-06-15 15:21:29.475 15873] DEBUG check_dns_listeners: &listener->family = 0xaaaad0dc4fdc
[2020-06-15 15:21:29.475 15873] DEBUG check_dns_listeners: listener->family = 10
[2020-06-15 15:21:29.475 15873] DEBUG check_dns_listeners: &listener->fd = 0xaaaad0dc4fd0
[2020-06-15 15:21:29.475 15873] DEBUG check_dns_listeners: listener->fd = 8
[2020-06-15 15:21:29.475 15873] DEBUG check_dns_listeners: listener->iface = (nil)
[2020-06-15 15:21:29.632 15873] DEBUG check_dns_listeners: listener = 0xaaaad0dc4fa0
[2020-06-15 15:21:29.632 15873] DEBUG check_dns_listeners: &listener->family = 0xaaaad0dc4fac
[2020-06-15 15:21:29.632 15873] DEBUG check_dns_listeners: listener->family = 2
[2020-06-15 15:21:29.632 15873] DEBUG check_dns_listeners: &listener->fd = 0xaaaad0dc4fa0
[2020-06-15 15:21:29.632 15873] DEBUG check_dns_listeners: listener->fd = 6
[2020-06-15 15:21:29.632 15873] DEBUG check_dns_listeners: listener->iface = (nil)
[2020-06-15 15:21:29.633 15873] DEBUG check_dns_listeners: listener = 0xaaaad0dc4fd0
[2020-06-15 15:21:29.633 15873] DEBUG check_dns_listeners: &listener->family = 0xaaaad0dc4fdc
[2020-06-15 15:21:29.633 15873] DEBUG check_dns_listeners: listener->family = 10
[2020-06-15 15:21:29.633 15873] DEBUG check_dns_listeners: &listener->fd = 0xaaaad0dc4fd0
[2020-06-15 15:21:29.633 15873] DEBUG check_dns_listeners: listener->fd = 8
[2020-06-15 15:21:29.633 15873] DEBUG check_dns_listeners: listener->iface = (nil)
[2020-06-15 15:21:29.666 15873] DEBUG check_dns_listeners: listener = 0xaaaad0dc4fa0
[2020-06-15 15:21:29.666 15873] DEBUG check_dns_listeners: &listener->family = 0xaaaad0dc4fac
[2020-06-15 15:21:29.666 15873] DEBUG check_dns_listeners: listener->family = 2
[2020-06-15 15:21:29.666 15873] DEBUG check_dns_listeners: &listener->fd = 0xaaaad0dc4fa0
[2020-06-15 15:21:29.666 15873] DEBUG check_dns_listeners: listener->fd = 6
[2020-06-15 15:21:29.666 15873] DEBUG check_dns_listeners: listener->iface = (nil)
[2020-06-15 15:21:29.666 15873] DEBUG check_dns_listeners: listener = 0xaaaad0dc4fd0
[2020-06-15 15:21:29.666 15873] DEBUG check_dns_listeners: &listener->family = 0xaaaad0dc4fdc
[2020-06-15 15:21:29.666 15873] DEBUG check_dns_listeners: listener->family = 10
[2020-06-15 15:21:29.666 15873] DEBUG check_dns_listeners: &listener->fd = 0xaaaad0dc4fd0
[2020-06-15 15:21:29.666 15873] DEBUG check_dns_listeners: listener->fd = 8
[2020-06-15 15:21:29.666 15873] DEBUG check_dns_listeners: listener->iface = (nil)
[2020-06-15 15:21:29.667 15873] DEBUG check_dns_listeners: listener = 0xaaaad0dc4fa0
[2020-06-15 15:21:29.667 15873] DEBUG check_dns_listeners: &listener->family = 0xaaaad0dc4fac
[2020-06-15 15:21:29.667 15873] DEBUG check_dns_listeners: listener->family = 2
[2020-06-15 15:21:29.667 15873] DEBUG check_dns_listeners: &listener->fd = 0xaaaad0dc4fa0
[2020-06-15 15:21:29.667 15873] DEBUG check_dns_listeners: listener->fd = 6
[2020-06-15 15:21:29.667 15873] DEBUG check_dns_listeners: listener->iface = (nil)
[2020-06-15 15:21:29.667 15873] DEBUG check_dns_listeners: listener = 0xaaaad0dc4fd0
[2020-06-15 15:21:29.667 15873] DEBUG check_dns_listeners: &listener->family = 0xaaaad0dc4fdc
[2020-06-15 15:21:29.667 15873] DEBUG check_dns_listeners: listener->family = 10
[2020-06-15 15:21:29.667 15873] DEBUG check_dns_listeners: &listener->fd = 0xaaaad0dc4fd0
[2020-06-15 15:21:29.667 15873] DEBUG check_dns_listeners: listener->fd = 8
[2020-06-15 15:21:29.667 15873] DEBUG check_dns_listeners: listener->iface = (nil)
[2020-06-15 15:21:29.981 15873] DEBUG check_dns_listeners: listener = 0xaaaad0dc4fa0
[2020-06-15 15:21:29.982 15873] DEBUG check_dns_listeners: &listener->family = 0xaaaad0dc4fac
[2020-06-15 15:21:29.982 15873] DEBUG check_dns_listeners: listener->family = 2
[2020-06-15 15:21:29.982 15873] DEBUG check_dns_listeners: &listener->fd = 0xaaaad0dc4fa0
[2020-06-15 15:21:29.982 15873] DEBUG check_dns_listeners: listener->fd = 6
[2020-06-15 15:21:29.982 15873] DEBUG check_dns_listeners: listener->iface = (nil)
[2020-06-15 15:21:29.982 15873] DEBUG check_dns_listeners: listener = 0xaaaad0dc4fd0
[2020-06-15 15:21:29.982 15873] DEBUG check_dns_listeners: &listener->family = 0xaaaad0dc4fdc
[2020-06-15 15:21:29.982 15873] DEBUG check_dns_listeners: listener->family = 10
[2020-06-15 15:21:29.982 15873] DEBUG check_dns_listeners: &listener->fd = 0xaaaad0dc4fd0
[2020-06-15 15:21:29.982 15873] DEBUG check_dns_listeners: listener->fd = 8
[2020-06-15 15:21:29.982 15873] DEBUG check_dns_listeners: listener->iface = (nil)
[2020-06-15 15:21:30.416 15873] DEBUG check_dns_listeners: listener = 0xaaaad0dc4fa0
[2020-06-15 15:21:30.416 15873] DEBUG check_dns_listeners: &listener->family = 0xaaaad0dc4fac
[2020-06-15 15:21:30.416 15873] DEBUG check_dns_listeners: listener->family = 2
[2020-06-15 15:21:30.416 15873] DEBUG check_dns_listeners: &listener->fd = 0xaaaad0dc4fa0
[2020-06-15 15:21:30.416 15873] DEBUG check_dns_listeners: listener->fd = 6
[2020-06-15 15:21:30.416 15873] DEBUG check_dns_listeners: listener->iface = (nil)
[2020-06-15 15:21:30.416 15873] DEBUG answer_request: rec = 0x6372610000aaaa16
[2020-06-15 15:21:30.416 15873] !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
[2020-06-15 15:21:30.416 15873] ----------------------------> FTL crashed! <----------------------------
[2020-06-15 15:21:30.416 15873] !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
[2020-06-15 15:21:30.416 15873] Please report a bug at https://github.com/pi-hole/FTL/issues
[2020-06-15 15:21:30.416 15873] and include in your report already the following details:
[2020-06-15 15:21:30.416 15873] FTL has been running for 1835 seconds
[2020-06-15 15:21:30.416 15873] FTL branch: fix/master-in_zone-and-iface_check_debugging
[2020-06-15 15:21:30.417 15873] FTL version: vDev-f2035de
[2020-06-15 15:21:30.417 15873] FTL commit: f2035de
[2020-06-15 15:21:30.417 15873] FTL date: 2020-06-10 09:43:13 +0200
[2020-06-15 15:21:30.418 15873] FTL user: started as pihole, ended as pihole
[2020-06-15 15:21:30.418 15873] Compiled for aarch64 (compiled on CI) using aarch64-linux-gnu-gcc (Debian 6.3.0-18) 6.3.0 20170516
[2020-06-15 15:21:30.418 15873] Received signal: Segmentation fault
[2020-06-15 15:21:30.418 15873] at address: 0x72610000aaaa36
[2020-06-15 15:21:30.418 15873] with code: SEGV_MAPERR (Address not mapped to object)
[2020-06-15 15:21:30.419 15873] Backtrace:
[2020-06-15 15:21:30.419 15873] B[0000]: 0xaaaaca0aa2b4, /usr/bin/pihole-FTL(+0x322b4) [0xaaaaca0aa2b4]
[2020-06-15 15:21:30.421 15873] B[0001]: 0xffff859aa688, linux-vdso.so.1(__kernel_rt_sigreturn+0) [0xffff859aa688]
[2020-06-15 15:21:30.421 15873] B[0002]: 0xaaaaca0d1c90, /usr/bin/pihole-FTL(answer_request+0xd8) [0xaaaaca0d1c90]
[2020-06-15 15:21:30.423 15873] B[0003]: 0xaaaaca0c4e90, /usr/bin/pihole-FTL(tcp_request+0x540) [0xaaaaca0c4e90]
[2020-06-15 15:21:30.424 15873] B[0004]: 0xaaaaca0db074, /usr/bin/pihole-FTL(+0x63074) [0xaaaaca0db074]
[2020-06-15 15:21:30.426 15873] B[0005]: 0xaaaaca0dcb08, /usr/bin/pihole-FTL(main_dnsmasq+0xe00) [0xaaaaca0dcb08]
[2020-06-15 15:21:30.428 15873] B[0006]: 0xaaaaca09d738, /usr/bin/pihole-FTL(main+0xf0) [0xaaaaca09d738]
[2020-06-15 15:21:30.430 15873] B[0007]: 0xffff857e3d24, /lib/aarch64-linux-gnu/libc.so.6(__libc_start_main+0xe4) [0xffff857e3d24]
[2020-06-15 15:21:30.430 15873] B[0008]: 0xaaaaca09d8cc, /usr/bin/pihole-FTL(+0x258cc) [0xaaaaca09d8cc]
[2020-06-15 15:21:30.432 15873] ------ Listing content of directory /dev/shm ------

@DL6ER dev build installed. On my platform happens randomly once a week. I will report back if any crash

@fradev
Copy link

fradev commented Jun 18, 2020

Hi there

here we go again

[2020-06-18 09:15:46.973 14287] Resizing "/FTL-dns-cache" from 286720 to 290816
[2020-06-18 09:32:01.754 14287] Resizing "/FTL-strings" from 708608 to 712704
[2020-06-18 09:51:07.052 14287] Resizing "/FTL-strings" from 712704 to 716800
[2020-06-18 09:53:17.041 11607] Note: FTL forked to handle TCP requests
[2020-06-18 09:53:17.098 11609] Note: FTL forked to handle TCP requests
[2020-06-18 09:53:17.120 11610] Note: FTL forked to handle TCP requests
[2020-06-18 09:53:17.169 11611] Note: FTL forked to handle TCP requests
[2020-06-18 09:53:17.251 11615] Note: FTL forked to handle TCP requests
[2020-06-18 09:53:17.290 11616] Note: FTL forked to handle TCP requests
[2020-06-18 09:53:17.307 11617] Note: FTL forked to handle TCP requests
[2020-06-18 09:53:17.324 11618] Note: FTL forked to handle TCP requests
[2020-06-18 09:53:17.367 11621] Note: FTL forked to handle TCP requests
[2020-06-18 09:53:17.387 11622] Note: FTL forked to handle TCP requests
[2020-06-18 09:53:17.554 11627] Note: FTL forked to handle TCP requests
[2020-06-18 09:53:17.597 14287] !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
[2020-06-18 09:53:17.597 14287] ----------------------------> FTL crashed! <----------------------------
[2020-06-18 09:53:17.597 14287] !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
[2020-06-18 09:53:17.597 14287] Please report a bug at https://github.com/pi-hole/FTL/issues
[2020-06-18 09:53:17.597 14287] and include in your report already the following details:
[2020-06-18 09:53:17.597 14287] FTL has been running for 225118 seconds
[2020-06-18 09:53:17.597 14287] FTL branch: development
[2020-06-18 09:53:17.597 14287] FTL version: vDev-6c32fb8
[2020-06-18 09:53:17.597 14287] FTL commit: 6c32fb8
[2020-06-18 09:53:17.597 14287] FTL date: 2020-06-04 15:39:51 +0200
[2020-06-18 09:53:17.599 14287] FTL user: started as pihole, ended as pihole
[2020-06-18 09:53:17.599 14287] Compiled for aarch64 (compiled on CI) using aarch64-linux-gnu-gcc (Debian 6.3.0-18) 6.3.0 20170516
[2020-06-18 09:53:17.599 14287] Received signal: Segmentation fault
[2020-06-18 09:53:17.599 14287] at address: 0xab607d390f30
[2020-06-18 09:53:17.599 14287] with code: SEGV_MAPERR (Address not mapped to object)
[2020-06-18 09:53:17.599 14287] Backtrace:
[2020-06-18 09:53:17.599 14287] B[0000]: 0xaaaac92c612c, /usr/bin/pihole-FTL(+0x3512c) [0xaaaac92c612c]
[2020-06-18 09:53:17.602 14287] B[0001]: 0xffffb806b688, linux-vdso.so.1(__kernel_rt_sigreturn+0) [0xffffb806b688]
[2020-06-18 09:53:17.602 14287] B[0002]: 0xaaaac92d763c, /usr/bin/pihole-FTL(cache_recv_insert+0x15c) [0xaaaac92d763c]
[2020-06-18 09:53:17.604 14287] B[0003]: 0xaaaac92e1730, /usr/bin/pihole-FTL(+0x50730) [0xaaaac92e1730]
[2020-06-18 09:53:17.607 14287] B[0004]: 0xaaaac92e32c0, /usr/bin/pihole-FTL(main_dnsmasq+0xe00) [0xaaaac92e32c0]
[2020-06-18 09:53:17.610 14287] B[0005]: 0xaaaac92b8f40, /usr/bin/pihole-FTL(main+0xf0) [0xaaaac92b8f40]
[2020-06-18 09:53:17.612 14287] B[0006]: 0xffffb7ea4d24, /lib/aarch64-linux-gnu/libc.so.6(__libc_start_main+0xe4) [0xffffb7ea4d24]
[2020-06-18 09:53:17.612 14287] B[0007]: 0xaaaac92b90d4, /usr/bin/pihole-FTL(+0x280d4) [0xaaaac92b90d4]
[2020-06-18 09:53:17.614 14287] ------ Listing content of directory /dev/shm ------
[2020-06-18 09:53:17.614 14287] File Mode User:Group Filesize Filename
[2020-06-18 09:53:17.616 14287] rwxrwxrwx root:root 260 .
[2020-06-18 09:53:17.616 14287] rwxr-xr-x root:root 3K ..
[2020-06-18 09:53:17.618 14287] rw------- pihole:pihole 4K FTL-per-client-regex
[2020-06-18 09:53:17.618 14287] rw------- pihole:pihole 291K FTL-dns-cache
[2020-06-18 09:53:17.619 14287] rw------- pihole:pihole 12K FTL-overTime
[2020-06-18 09:53:17.619 14287] rw------- pihole:pihole 7M FTL-queries
[2020-06-18 09:53:17.619 14287] rw------- pihole:pihole 4K FTL-upstreams
[2020-06-18 09:53:17.619 14287] rw------- pihole:pihole 332K FTL-clients
[2020-06-18 09:53:17.619 14287] rw------- pihole:pihole 492K FTL-domains
[2020-06-18 09:53:17.620 14287] rw------- pihole:pihole 717K FTL-strings
[2020-06-18 09:53:17.620 14287] rw------- pihole:pihole 12 FTL-settings
[2020-06-18 09:53:17.620 14287] rw------- pihole:pihole 124 FTL-counters
[2020-06-18 09:53:17.620 14287] rw------- pihole:pihole 56 FTL-lock
[2020-06-18 09:53:17.620 14287] ---------------------------------------------------
[2020-06-18 09:53:17.620 14287] Thank you for helping us to improve our FTL engine!
[2020-06-18 09:53:17.620 14287] FTL terminated!
[2020-06-18 10:20:41.012 495] Using log file /var/log/pihole-FTL.log
[2020-06-18 10:20:41.016 495] ########## FTL started! ##########
[2020-06-18 10:20:41.016 495] FTL branch: development
[2020-06-18 10:20:41.016 495] FTL version: vDev-6c32fb8
[2020-06-18 10:20:41.016 495] FTL commit: 6c32fb8
[2020-06-18 10:20:41.016 495] FTL date: 2020-06-04 15:39:51 +0200
[2020-06-18 10:20:41.016 495] FTL user: pihole

let me know if I can provide more info (seems consistent on my VM)

@DL6ER
Copy link
Member

DL6ER commented Jun 18, 2020

@fradev We have a potential fix for a rather severe dnsmasq-internal bug discovered by @fhriley

Please run

pihole checkout ftl fix/fhriley-fix_buf_overflow

and see if the crashes are resolved. As always, I very much appreciate testing as it is the only way to be sure we really got it fixed!

@fradev
Copy link

fradev commented Jun 19, 2020

Done.. will report in any case (currently the crash timespan is 2 days)

@jmarkert
Copy link

I'm not sure if it's the same bug, I'm running on a pi zero wh. Pihole crashed the 2nd time after running for 3 days, not sure how long it was up and running after the upgrade.

[2020-06-21 19:25:20.976 24776] Resizing "/FTL-dns-cache" from 143360 to 147456
[2020-06-21 19:59:01.392 24776] Notice: Database size is 755.39 MB, deleted 43 rows
[2020-06-21 20:24:55.030 25860] Note: FTL forked to handle TCP requests
[2020-06-21 20:42:32.365 26094] Note: FTL forked to handle TCP requests
[2020-06-21 20:54:55.124 26252] Note: FTL forked to handle TCP requests
[2020-06-21 21:00:01.423 24776] Notice: Database size is 755.59 MB, deleted 128 rows
[2020-06-21 21:24:55.786 24776] !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
[2020-06-21 21:24:55.786 24776] ---------------------------->  FTL crashed!  <----------------------------
[2020-06-21 21:24:55.786 24776] !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
[2020-06-21 21:24:55.787 24776] Please report a bug at https://github.com/pi-hole/FTL/issues
[2020-06-21 21:24:55.787 24776] and include in your report already the following details:
[2020-06-21 21:24:55.787 24776] FTL has been running for 294821 seconds
[2020-06-21 21:24:55.787 24776] FTL branch: master
[2020-06-21 21:24:55.787 24776] FTL version: v5.0
[2020-06-21 21:24:55.788 24776] FTL commit: 3d7c095
[2020-06-21 21:24:55.788 24776] FTL date: 2020-05-10 18:58:38 +0100
[2020-06-21 21:24:55.789 24776] FTL user: started as pihole, ended as pihole
[2020-06-21 21:24:55.789 24776] Compiled for arm (compiled on CI) using arm-linux-gnueabihf-gcc (crosstool-NG crosstool-ng-1.22.0-88-g8460611) 4.9.3
[2020-06-21 21:24:55.789 24776] Received signal: Segmentation fault
[2020-06-21 21:24:55.789 24776]      at address: 0x7a66652d
[2020-06-21 21:24:55.790 24776]      with code: SEGV_MAPERR (Address not mapped to object)
[2020-06-21 21:24:55.790 24776] Backtrace:
[2020-06-21 21:24:55.792 24776] B[0000]: 0x44f704, /usr/bin/pihole-FTL(+0x2c704) [0x44f704]
[2020-06-21 21:24:55.792 24776] B[0001]: 0xb6dd16c0, /lib/arm-linux-gnueabihf/libc.so.6(__default_rt_sa_restorer+0) [0xb6dd16c0]
[2020-06-21 21:24:55.792 24776] B[0002]: 0x485b2c, /usr/bin/pihole-FTL(+0x62b2c) [0x485b2c]
[2020-06-21 21:24:55.792 24776] B[0003]: 0x487cfc, /usr/bin/pihole-FTL(main_dnsmasq+0x1238) [0x487cfc]
[2020-06-21 21:24:55.793 24776] B[0004]: 0x441d04, /usr/bin/pihole-FTL(main+0xfc) [0x441d04]
[2020-06-21 21:24:55.793 24776] B[0005]: 0xb6dbb678, /lib/arm-linux-gnueabihf/libc.so.6(__libc_start_main+0x114) [0xb6dbb678]
[2020-06-21 21:24:55.793 24776] ------ Listing content of directory /dev/shm ------
[2020-06-21 21:24:55.794 24776] File Mode User:Group  Filesize Filename
[2020-06-21 21:24:55.795 24776] rwxrwxrwx root:root 260 .
[2020-06-21 21:24:55.796 24776] rwxr-xr-x root:root 4K ..
[2020-06-21 21:24:55.797 24776] rw------- pihole:pihole 4K FTL-per-client-regex
[2020-06-21 21:24:55.798 24776] rw------- pihole:pihole 147K FTL-dns-cache
[2020-06-21 21:24:55.799 24776] rw------- pihole:pihole 29K FTL-overTime
[2020-06-21 21:24:55.800 24776] rw------- pihole:pihole 3M FTL-queries
[2020-06-21 21:24:55.801 24776] rw------- pihole:pihole 20K FTL-upstreams
[2020-06-21 21:24:55.802 24776] rw------- pihole:pihole 643K FTL-clients
[2020-06-21 21:24:55.803 24776] rw------- pihole:pihole 131K FTL-domains
[2020-06-21 21:24:55.804 24776] rw------- pihole:pihole 193K FTL-strings
[2020-06-21 21:24:55.805 24776] rw------- pihole:pihole 12 FTL-settings
[2020-06-21 21:24:55.806 24776] rw------- pihole:pihole 124 FTL-counters
[2020-06-21 21:24:55.807 24776] rw------- pihole:pihole 28 FTL-lock
[2020-06-21 21:24:55.808 24776] ---------------------------------------------------
[2020-06-21 21:24:55.808 24776] Thank you for helping us to improve our FTL engine!
[2020-06-21 21:24:55.808 24776] FTL terminated!

@yik77
Copy link
Author

yik77 commented Jun 24, 2020

I have 2 more SEGFAULT crashes on new installation, in the meantime, my constant rebooting and hard power offs had destroyed my 1 year old SD card, so I had to buy new and reinstall it all. Crashes are on clean install of raspbian, on pi zero w, with pihole as the only application running. (on top of ssh deamons and default minimal debian)

[2020-06-23 20:34:21.946 20300] Note: FTL forked to handle TCP requests
[2020-06-23 20:39:27.317 6746] Resizing "/FTL-dns-cache" from 135168 to 139264
[2020-06-23 20:47:29.461 6746] Resizing "/FTL-strings" from 131072 to 135168
[2020-06-23 21:02:27.928 20532] Note: FTL forked to handle TCP requests
[2020-06-23 21:12:27.921 20646] Note: FTL forked to handle TCP requests
[2020-06-23 21:32:28.101 20778] Note: FTL forked to handle TCP requests
[2020-06-23 21:42:28.008 20901] Note: FTL forked to handle TCP requests
[2020-06-23 23:52:27.914 21958] Note: FTL forked to handle TCP requests
[2020-06-24 00:32:28.119 22292] Note: FTL forked to handle TCP requests
[2020-06-24 00:40:19.101 6746] Resizing "/FTL-dns-cache" from 139264 to 143360
[2020-06-24 03:02:28.194 23542] Note: FTL forked to handle TCP requests
[2020-06-24 03:32:28.593 6746] !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
[2020-06-24 03:32:28.593 6746] ----------------------------> FTL crashed! <----------------------------
[2020-06-24 03:32:28.594 6746] !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
[2020-06-24 03:32:28.594 6746] Please report a bug at https://github.com/pi-hole/FTL/issues
[2020-06-24 03:32:28.594 6746] and include in your report already the following details:
[2020-06-24 03:32:28.594 6746] FTL has been running for 111285 seconds
[2020-06-24 03:32:28.594 6746] FTL branch: master
[2020-06-24 03:32:28.595 6746] FTL version: v5.0
[2020-06-24 03:32:28.595 6746] FTL commit: 3d7c095
[2020-06-24 03:32:28.595 6746] FTL date: 2020-05-10 18:58:38 +0100
[2020-06-24 03:32:28.595 6746] FTL user: started as pihole, ended as pihole
[2020-06-24 03:32:28.596 6746] Compiled for arm (compiled on CI) using arm-linux-gnueabihf-gcc (crosstool-NG crosstool-ng-1.22.0-88-g8460611) 4.9.3
[2020-06-24 03:32:28.596 6746] Received signal: Segmentation fault
[2020-06-24 03:32:28.596 6746] at address: 0x636a7300
[2020-06-24 03:32:28.596 6746] with code: SEGV_MAPERR (Address not mapped to object)
[2020-06-24 03:32:28.597 6746] Backtrace:
[2020-06-24 03:32:28.598 6746] B[0000]: 0x4f3704, /usr/bin/pihole-FTL(+0x2c704) [0x4f3704]
[2020-06-24 03:32:28.598 6746] B[0001]: 0xb6dae130, /lib/arm-linux-gnueabihf/libc.so.6(__default_rt_sa_restorer+0) [0xb6dae130]
[2020-06-24 03:32:28.598 6746] B[0002]: 0x52e1b4, /usr/bin/pihole-FTL(iface_check+0x50) [0x52e1b4]
[2020-06-24 03:32:28.599 6746] B[0003]: 0x5134e0, /usr/bin/pihole-FTL(receive_query+0x268) [0x5134e0]
[2020-06-24 03:32:28.599 6746] B[0004]: 0x529d68, /usr/bin/pihole-FTL(+0x62d68) [0x529d68]
[2020-06-24 03:32:28.599 6746] B[0005]: 0x52bcfc, /usr/bin/pihole-FTL(main_dnsmasq+0x1238) [0x52bcfc]
[2020-06-24 03:32:28.599 6746] B[0006]: 0x4e5d04, /usr/bin/pihole-FTL(main+0xfc) [0x4e5d04]
[2020-06-24 03:32:28.599 6746] B[0007]: 0xb6d98718, /lib/arm-linux-gnueabihf/libc.so.6(__libc_start_main+0x10c) [0xb6d98718]
[2020-06-24 03:32:28.600 6746] ------ Listing content of directory /dev/shm ------
[2020-06-24 03:32:28.600 6746] File Mode User:Group Filesize Filename
[2020-06-24 03:32:28.600 6746] rwxrwxrwx root:root 260 .
[2020-06-24 03:32:28.601 6746] rwxr-xr-x root:root 4K ..
[2020-06-24 03:32:28.601 6746] rw------- pihole:pihole 4K FTL-per-client-regex
[2020-06-24 03:32:28.602 6746] rw------- pihole:pihole 143K FTL-dns-cache
[2020-06-24 03:32:28.603 6746] rw------- pihole:pihole 29K FTL-overTime
[2020-06-24 03:32:28.603 6746] rw------- pihole:pihole 4M FTL-queries
[2020-06-24 03:32:28.604 6746] rw------- pihole:pihole 20K FTL-upstreams
[2020-06-24 03:32:28.605 6746] rw------- pihole:pihole 643K FTL-clients
[2020-06-24 03:32:28.605 6746] rw------- pihole:pihole 131K FTL-domains
[2020-06-24 03:32:28.606 6746] rw------- pihole:pihole 135K FTL-strings
[2020-06-24 03:32:28.607 6746] rw------- pihole:pihole 12 FTL-settings
[2020-06-24 03:32:28.607 6746] rw------- pihole:pihole 124 FTL-counters
[2020-06-24 03:32:28.608 6746] rw------- pihole:pihole 28 FTL-lock
[2020-06-24 03:32:28.608 6746] ---------------------------------------------------
[2020-06-24 03:32:28.608 6746] Thank you for helping us to improve our FTL engine!
[2020-06-24 03:32:28.608 6746] FTL terminated!
[2020-06-24 06:41:52.755 25020] Using log file /var/log/pihole-FTL.log
[2020-06-24 06:41:52.756 25020] ########## FTL started! ##########
[2020-06-24 06:41:52.756 25020] FTL branch: master
[2020-06-24 06:41:52.756 25020] FTL version: v5.0
[2020-06-24 06:41:52.756 25020] FTL commit: 3d7c095
[2020-06-24 06:41:52.756 25020] FTL date: 2020-05-10 18:58:38 +0100
[2020-06-24 06:41:52.757 25020] FTL user: pihole
[2020-06-24 06:41:52.757 25020] Compiled for arm (compiled on CI) using arm-linux-gnueabihf-gcc (crosstool-NG crosstool-ng-1.22.0-88-g8460611) 4.9.3

and again,

[2020-06-24 15:46:17.300 29385] Note: FTL forked to handle TCP requests
[2020-06-24 15:56:17.261 29436] Note: FTL forked to handle TCP requests
[2020-06-24 15:57:20.686 25022] Resizing "/FTL-strings" from 126976 to 131072
[2020-06-24 15:58:20.068 25022] Resizing "/FTL-dns-cache" from 53248 to 57344
[2020-06-24 16:02:28.090 29491] Note: FTL forked to handle TCP requests
[2020-06-24 16:12:27.849 29597] Note: FTL forked to handle TCP requests
[2020-06-24 16:12:28.060 25022] !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
[2020-06-24 16:12:28.060 25022] ----------------------------> FTL crashed! <----------------------------
[2020-06-24 16:12:28.060 25022] !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
[2020-06-24 16:12:28.060 25022] Please report a bug at https://github.com/pi-hole/FTL/issues
[2020-06-24 16:12:28.061 25022] and include in your report already the following details:
[2020-06-24 16:12:28.061 25022] FTL has been running for 34236 seconds
[2020-06-24 16:12:28.061 25022] FTL branch: master
[2020-06-24 16:12:28.061 25022] FTL version: v5.0
[2020-06-24 16:12:28.061 25022] FTL commit: 3d7c095
[2020-06-24 16:12:28.062 25022] FTL date: 2020-05-10 18:58:38 +0100
[2020-06-24 16:12:28.062 25022] FTL user: started as pihole, ended as pihole
[2020-06-24 16:12:28.062 25022] Compiled for arm (compiled on CI) using arm-linux-gnueabihf-gcc (crosstool-NG crosstool-ng-1.22.0-88-g8460611) 4.9.3
[2020-06-24 16:12:28.062 25022] Received signal: Segmentation fault
[2020-06-24 16:12:28.062 25022] at address: 0x6432643d
[2020-06-24 16:12:28.063 25022] with code: SEGV_MAPERR (Address not mapped to object)
[2020-06-24 16:12:28.063 25022] Backtrace:
[2020-06-24 16:12:28.065 25022] B[0000]: 0x453704, /usr/bin/pihole-FTL(+0x2c704) [0x453704]
[2020-06-24 16:12:28.065 25022] B[0001]: 0xb6d66130, /lib/arm-linux-gnueabihf/libc.so.6(__default_rt_sa_restorer+0) [0xb6d66130]
[2020-06-24 16:12:28.065 25022] B[0002]: 0x48dacc, /usr/bin/pihole-FTL(+0x66acc) [0x48dacc]
[2020-06-24 16:12:28.065 25022] B[0003]: 0x489bd4, /usr/bin/pihole-FTL(+0x62bd4) [0x489bd4]
[2020-06-24 16:12:28.065 25022] B[0004]: 0x48bcfc, /usr/bin/pihole-FTL(main_dnsmasq+0x1238) [0x48bcfc]
[2020-06-24 16:12:28.065 25022] B[0005]: 0x445d04, /usr/bin/pihole-FTL(main+0xfc) [0x445d04]
[2020-06-24 16:12:28.066 25022] B[0006]: 0xb6d50718, /lib/arm-linux-gnueabihf/libc.so.6(__libc_start_main+0x10c) [0xb6d50718]
[2020-06-24 16:12:28.067 25022] ------ Listing content of directory /dev/shm ------
[2020-06-24 16:12:28.067 25022] File Mode User:Group Filesize Filename
[2020-06-24 16:12:28.068 25022] rwxrwxrwx root:root 260 .
[2020-06-24 16:12:28.068 25022] rwxr-xr-x root:root 4K ..
[2020-06-24 16:12:28.069 25022] rw------- pihole:pihole 4K FTL-per-client-regex
[2020-06-24 16:12:28.069 25022] rw------- pihole:pihole 57K FTL-dns-cache
[2020-06-24 16:12:28.070 25022] rw------- pihole:pihole 29K FTL-overTime
[2020-06-24 16:12:28.071 25022] rw------- pihole:pihole 3M FTL-queries
[2020-06-24 16:12:28.071 25022] rw------- pihole:pihole 20K FTL-upstreams
[2020-06-24 16:12:28.072 25022] rw------- pihole:pihole 643K FTL-clients
[2020-06-24 16:12:28.072 25022] rw------- pihole:pihole 131K FTL-domains
[2020-06-24 16:12:28.073 25022] rw------- pihole:pihole 131K FTL-strings
[2020-06-24 16:12:28.074 25022] rw------- pihole:pihole 12 FTL-settings
[2020-06-24 16:12:28.074 25022] rw------- pihole:pihole 124 FTL-counters
[2020-06-24 16:12:28.075 25022] rw------- pihole:pihole 28 FTL-lock
[2020-06-24 16:12:28.075 25022] ---------------------------------------------------
[2020-06-24 16:12:28.075 25022] Thank you for helping us to improve our FTL engine!
[2020-06-24 16:12:28.076 25022] FTL terminated!
[2020-06-24 16:49:15.602 30050] Using log file /var/log/pihole-FTL.log
[2020-06-24 16:49:15.602 30050] ########## FTL started! ##########
[2020-06-24 16:49:15.603 30050] FTL branch: master
[2020-06-24 16:49:15.603 30050] FTL version: v5.0
[2020-06-24 16:49:15.603 30050] FTL commit: 3d7c095

@yik77
Copy link
Author

yik77 commented Jun 24, 2020

Done.. will report in any case (currently the crash timespan is 2 days)

did the fix solved your problem?

@DL6ER
Copy link
Member

DL6ER commented Jun 24, 2020

@yik77 you are not using the fixed version

[2020-06-24 03:32:28.594 6746] FTL branch: master

Please run

pihole checkout ftl development

to get the fix

@yik77
Copy link
Author

yik77 commented Jun 24, 2020

@yik77 you are not using the fixed version
Please run

pihole checkout ftl development

to get the fix

Done. I just now did run pihole checkout ftl development.

@frollic
Copy link

frollic commented Jun 25, 2020

I just ran into this issue too, have two piholes, both set up in the same way, and getting the same
SEGV_MAPERR error.

Installed the fix_buf_overflow FTL just now on one of them, we'll see if it helps.

Log says vDev-3c0553e, is it proof of the beta binary used ?

@DL6ER
Copy link
Member

DL6ER commented Jun 26, 2020

@frollic Please run

pihole checkout ftl development

The other branch is already outdated.

@frollic
Copy link

frollic commented Jun 26, 2020

Done!

@fradev
Copy link

fradev commented Jun 26, 2020

after several days with FTL Version vDev (fix/fhriley-fix_buf_overflow, vDev-3c0553e) no crashes

@frollic
Copy link

frollic commented Jun 30, 2020

With vDev (development, vDev-3cf1889) mine crashed again after ~4days, on the 27th, didn't notice until today.

@DL6ER
Copy link
Member

DL6ER commented Jun 30, 2020

@frollic Can you give some details from /var/log/pihole-FTL.log ? (may be rotated after a few days, check the .1, .2, etc. files)

@frollic
Copy link

frollic commented Jun 30, 2020

Whole log attached, all the way back to before the v5 upgrade.

Not that many entries in it, since it's the secondary DNS server.

pihole-FTL.zip

@yik77
Copy link
Author

yik77 commented Jul 1, 2020

FYI after 7 days on vDev-bd405e9 there are no crashes. I am getting cautiously optimistic.

@DL6ER
Copy link
Member

DL6ER commented Jul 1, 2020

@frollic Please see #816 (comment)

@DL6ER DL6ER linked a pull request Jul 8, 2020 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants