Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Segmentation fault on raspberry pi 2 model b #38

Open
daniloncmayo opened this issue Aug 1, 2018 · 81 comments
Open

Segmentation fault on raspberry pi 2 model b #38

daniloncmayo opened this issue Aug 1, 2018 · 81 comments
Assignees

Comments

@daniloncmayo
Copy link

@daniloncmayo daniloncmayo commented Aug 1, 2018

File downloaded from https://bin.equinox.io/c/VdrWdbjqyF/cloudflared-stable-linux-arm.tgz

pi@raspberrypi:~ $ ./cloudflared -v
Segmentation fault
pi@raspberrypi:~ $ uname -a 
Linux raspberrypi 4.14.52+ #1123 Wed Jun 27 17:05:32 BST 2018 armv6l GNU/Linux
pi@raspberrypi:~ $ lsb_release -a
No LSB modules are available.
Distributor ID:	Raspbian
Description:	Raspbian GNU/Linux 9.4 (stretch)
Release:	9.4
Codename:	stretch
pi@raspberrypi:~ $ cat /proc/cpuinfo 
processor	: 0
model name	: ARMv6-compatible processor rev 7 (v6l)
BogoMIPS	: 697.95
Features	: half thumb fastmult vfp edsp java tls 
CPU implementer	: 0x41
CPU architecture: 7
CPU variant	: 0x0
CPU part	: 0xb76
CPU revision	: 7

Hardware	: BCM2835
Revision	: 000e
pi@raspberrypi:~ $ sha256sum cloudflared
b3730fd14bc7306b09eafefcef10025aca3f2e94a6059952426a5341ab6e4045  cloudflared
pi@raspberrypi:~ $ file cloudflared
cloudflared: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-armhf.so.3, for GNU/Linux 3.2.0, BuildID[sha1]=adf2825c51e543d9a36dea416573b70eeaa2ac8a, not stripped
@giusr
Copy link

@giusr giusr commented Aug 2, 2018

Same on Raspberry Pi Zero and Raspberry Pi Zero W.

@Pulikutti
Copy link

@Pulikutti Pulikutti commented Aug 4, 2018

Same on Pi Zero W. reverted to 2018.6.2

Program received signal SIGSEGV, Segmentation fault.
0x00060ea0 in runtime.moduledataverify1 (datap=0x36333b31) at /usr/local/go/src/runtime/symtab.go:517

@xyzulu
Copy link

@xyzulu xyzulu commented Aug 10, 2018

Exactly the same issue as well. It's also being reported elsewhere.
Reverting to 2018.7.2 works..

the issue is 2018.7.3

@giusr
Copy link

@giusr giusr commented Aug 26, 2018

@xyzulu how do you revert to 2018.7.2? I can't find the binaries for download.

@xyzulu
Copy link

@xyzulu xyzulu commented Aug 26, 2018

@angussidney
Copy link

@angussidney angussidney commented Sep 4, 2018

I have also reproduced this issue on my Raspberry Pi Model 1B (rev 2). Reverting to 2018.7.2 as xyzulu suggested works.

@rix1337
Copy link

@rix1337 rix1337 commented Sep 11, 2018

Same issue, same device

@cvocvo
Copy link

@cvocvo cvocvo commented Sep 11, 2018

+1 for same issue on a RPi Model B; 2018-7.2 works but I keep randomly running into this (#23) same error with 2018-7.2.
failed to connect to an HTTPS backend and then have to reboot the Pi.

Would love to get this fixed to try a later version of Cloudflared that hopefully resolves that problem :)

@dullroar
Copy link

@dullroar dullroar commented Sep 29, 2018

+1 on all of the above, including falling back to 2018-7.2 works.

@tconnz
Copy link

@tconnz tconnz commented Oct 2, 2018

Same issue here on RPi B - Segmentation Fault.

EDIT- 2018-7.2 only works if copied to usr/local/bin and then chmod +x is run.

@CBielstein
Copy link

@CBielstein CBielstein commented Oct 9, 2018

Same issue with a Raspberry Pi model A. Falling back to 2018-7.2 worked without any further steps.

@silvether
Copy link

@silvether silvether commented Oct 17, 2018

+1
Can repro on Raspberry Pi 1 B+ with Raspbian Minimal

@sssilver sssilver added the bug label Oct 24, 2018
@sssilver sssilver self-assigned this Oct 24, 2018
@jmvermeulen
Copy link

@jmvermeulen jmvermeulen commented Nov 11, 2018

Same here on Pi 2.

@Chlorus
Copy link

@Chlorus Chlorus commented Nov 27, 2018

+1 for a Raspberry Pi 1 Model B+, haven't tried reverting to an older version yet.

@fortunate-man
Copy link

@fortunate-man fortunate-man commented Dec 25, 2018

Falling back to 2018-7.2 does work on a Raspberry Pi Zero W.

@joehillen
Copy link

@joehillen joehillen commented Feb 3, 2019

I ended up building from source on my Pi Zero W, and it works like a charm. Had to add an extra swapfile in order for the build process to have enough RAM.

I suspect cloudflare's ARM build instance is to blame.

@ravvle
Copy link

@ravvle ravvle commented Feb 4, 2019

I also had this issue on the Pi Zero W, building from source fixed this for me. I had to add extra swap space to compile properly. Thanks @joehillen

Note: I compiled using Go 1.11.5. I noticed the build config seems to use the 1.9 stream which doesn't receive patches any more.

@dsadsa897897r
Copy link

@dsadsa897897r dsadsa897897r commented Feb 19, 2019

What command is needed to compile using Go?

sudo apt-get install golang
export GOPATH=$HOME/go
go install github.com/cloudflare/cloudflared/cmd/cloudflared

@sssilver
Copy link
Collaborator

@sssilver sssilver commented Feb 20, 2019

Hi,

The next cloudflared release will be built with Go 1.11.

@hexdra
Copy link

@hexdra hexdra commented Mar 5, 2019

Same error on Pi B Version 2019.2.1.
2018.7.2 works.

@ArniDagur
Copy link

@ArniDagur ArniDagur commented Apr 6, 2019

@sssilver Since there has been a new release, does that mean this issue is fixed?

@timothybrown
Copy link

@timothybrown timothybrown commented Apr 11, 2019

@sssilver Since there has been a new release, does that mean this issue is fixed?

Does not appear to have been fixed. I tried the current release today and it’s still segfaulting.

@Trikolon
Copy link

@Trikolon Trikolon commented May 6, 2019

cloudflared version 2019.4.1 (built 2019-04-19-2152 UTC) (official binary download))
Works on Raspberry Pi 3b+

@cleanev
Copy link

@cleanev cleanev commented May 16, 2019

Same issue on RPiZeroW with latest Rasberry Pi Stretch with all updates and name showing - Linux ZeroPiHole-I 4.19.42+ #1219 Tue May 14 21:16:38 BST 2019 armv6l GNU/Linux

Latest version 2019.4.1 only works with RPi3B+

pi@ZeroPiHole-I:~ $ dig yahoo.com

; <<>> DiG 9.10.3-P4-Raspbian <<>> yahoo.com
;; global options: +cmd
;; connection timed out; no servers could be reached
pi@ZeroPiHole-I:~ $ dig @127.0.0.1 -p 5053 yahoo.com

; <<>> DiG 9.10.3-P4-Raspbian <<>> @127.0.0.1 -p 5053 yahoo.com
; (1 server found)
;; global options: +cmd
;; connection timed out; no servers could be reached
pi@ZeroPiHole-I:~ $ ping 1.1.1.1
PING 1.1.1.1 (1.1.1.1) 56(84) bytes of data.
64 bytes from 1.1.1.1: icmp_seq=1 ttl=55 time=23.9 ms
64 bytes from 1.1.1.1: icmp_seq=2 ttl=55 time=26.9 ms
^C
--- 1.1.1.1 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1000ms
rtt min/avg/max/mdev = 23.930/25.415/26.900/1.485 ms

Name resolution with 2018.7.2 does not work at all

I referenced this article when setting up DoH - https://docs.pi-hole.net/guides/dns-over-https/

@billthefarmer
Copy link

@billthefarmer billthefarmer commented Jun 3, 2019

I think the segfault issue is due to released cloudflared being built on a later ARM processor, or cross compiled. The latest current version (2019.5.0) works fine on a Pi 3 B+, whether it stops working after a while (#23), I don't know, didn't test it for long enough, but segfault on a Pi 1B (BCM2835).
I installed go1.12.5 linux/arm on the Pi 1B and built from source, release 2019.5.0 plus a few commits (babcd9f), this appears to work, but stops working after a while (#23), it appears to run out of file handles.

Jun  3 06:33:14 jennifer cloudflared[261]: time="2019-06-03T06:33:14Z" level=error msg="failed to connect to an HTTPS backend \"https://1.1.1.1/dns-query\"" error="failed to perform an HTTPS request: Post https://1.1.1.1/dns-query: dial tcp 1.1.1.1:443: socket: too many open files"
Jun  3 06:33:14 jennifer cloudflared[261]: time="2019-06-03T06:33:14Z" level=error msg="failed to connect to an HTTPS backend \"https://1.0.0.1/dns-query\"" error="failed to perform an HTTPS request: Post https://1.0.0.1/dns-query: dial tcp 1.0.0.1:443: socket: too many open files"
Jun  3 06:33:14 jennifer cloudflared[261]: time="2019-06-03T06:33:14Z" level=error msg="failed to connect to an HTTPS backend \"https://1.1.1.1/dns-query\"" error="failed to perform an HTTPS request: Post https://1.1.1.1/dns-query: dial tcp 1.1.1.1:443: socket: too many open files"
Jun  3 06:33:14 jennifer cloudflared[261]: time="2019-06-03T06:33:14Z" level=error msg="failed to connect to an HTTPS backend \"https://1.0.0.1/dns-query\"" error="failed to perform an HTTPS request: Post https://1.0.0.1/dns-query: dial tcp 1.0.0.1:443: socket: too many open files"

Update:
Having looked at the Makefile, I built version 2019.5.0 using it...

pi@jennifer:~ $ cd gocode/src/github.com/cloudflare/cloudflared/
pi@jennifer:~/gocode/src/github.com/cloudflare/cloudflared $ git checkout 2019.5.0
Note: checking out '2019.5.0'.
...
HEAD is now at 4bff1ef... Release 2019.5.0
pi@jennifer:~/gocode/src/github.com/cloudflare/cloudflared $ export PATH=$PATH:/usr/local/go/bin
pi@jennifer:~/gocode/src/github.com/cloudflare/cloudflared $ export GOPATH=~/gocode
pi@jennifer:~/gocode/src/github.com/cloudflare/cloudflared $ go clean
pi@jennifer:~/gocode/src/github.com/cloudflare/cloudflared $ make cloudflared
go build -v -ldflags='-X "main.Version=2019.5.0" -X "main.BuildTime=2019-06-03-1717 UTC"' github.com/cloudflare/cloudflared/cmd/cloudflared
pi@jennifer:~/gocode/src/github.com/cloudflare/cloudflared $ ./cloudflared -v
cloudflared version 2019.5.0 (built 2019-06-03-1717 UTC)

This version may be ok, it's now been running for several hours including a Pi reboot and a router reboot.

@harishvishwakarma
Copy link

@harishvishwakarma harishvishwakarma commented Mar 25, 2020

Still happening in Pi 2 Model B

@charminULTRA
Copy link

@charminULTRA charminULTRA commented Apr 12, 2020

Still happening on Pi Zero W current version as of this date.

@charminULTRA
Copy link

@charminULTRA charminULTRA commented Apr 12, 2020

What command is needed to compile using Go?

sudo apt-get install golang
export GOPATH=$HOME/go
go install github.com/cloudflare/cloudflared/cmd/cloudflared

This no longer works, it's not located there anymore.

@charminULTRA
Copy link

@charminULTRA charminULTRA commented Apr 12, 2020

@deggers
Copy link

@deggers deggers commented Jun 14, 2020

Some problem raspberry pi zero W.

What Solution?

#38 (comment) still works on my rpi zero like from him suggestest. Just fetched with wget from hobin the latest

@queeup
Copy link

@queeup queeup commented Jun 21, 2020

Still happening in the pi 1B with latest version (2020.6.5)

@balu100
Copy link

@balu100 balu100 commented Sep 5, 2020

Still happening on Model B Rev 2

@joehillen
Copy link

@joehillen joehillen commented Sep 6, 2020

Since Cloudflare seems to ignore this issues. I've switched to NextDNS. It works great. Highly recommend it.

@TownLake
Copy link
Contributor

@TownLake TownLake commented Sep 6, 2020

Hi from Cloudflare. We've made some improvements to packaging and DoH issues. That said, we still haven't been able to track down the cause of the issues with Raspberry Pi devices. I know it's frustrating.

We have the PiHole DoH issues on our list, but don't have an ETA yet when you could expect a fix. We'll update this thread and the related ones when we do. Thanks for the patience and feedback - apologies for the delay here.

@abishekmuthian
Copy link

@abishekmuthian abishekmuthian commented Sep 6, 2020

I found that any version of cloudflared flaky on a Pi 1B, either an old build or a rebuilt recent version, in that it stops working after a while. I then tried DNSCrypt-Proxy, which also stopped working after a while. The common factor appears to be that they are both written in Go. I then tried unbound, which works and has the advantage that it's already in the Raspbian repository.

I recommend unbound as well as a remedy for issues with cloudflared on 32-bit ARM, as even if there is no segmentation fault in the natively built package, it stops working after a while. Here is a nice guide for setting it up on Pi Hole - https://bartonbytes.com/posts/configure-pi-hole-for-dns-over-tls/.

Using unbound for DNS over TLS specifically for secondary Pi Hole (as it's mostly the case for using older RPi) is especially useful as we can specify other encrypted DNS providers here as a fallback when Cloudflare goes down.

@brbergami
Copy link

@brbergami brbergami commented Dec 12, 2020

I can confirm this issue still happens. I'm using a Zero W. Had to find a 2018 build that works. But if you diff, there's a ton of commits added (which I guess includes features and bug fixes), can't be 2 years and this still persist.

root@DietPi:~# uname -a
Linux DietPi 5.4.79+ #1373 Mon Nov 23 13:18:15 GMT 2020 armv6l GNU/Linux
root@DietPi:~# lsb_release -a
No LSB modules are available.
Distributor ID: Raspbian
Description:    Raspbian GNU/Linux 10 (buster)
Release:        10
Codename:       buster
root@DietPi:~# cat /proc/cpuinfo
processor       : 0
model name      : ARMv6-compatible processor rev 7 (v6l)
BogoMIPS        : 1423.06
Features        : half thumb fastmult vfp edsp java tls
CPU implementer : 0x41
CPU architecture: 7
CPU variant     : 0x0
CPU part        : 0xb76
CPU revision    : 7

Hardware        : BCM2835
Revision        : 9000c1
Serial          : 000000001ea488ff
Model           : Raspberry Pi Zero W Rev 1.1

Working build: b2b46a483505babd15743d3743728ebf5a1b82ed8e47093a2009ae6e48c83034

@yolofy
Copy link

@yolofy yolofy commented Dec 18, 2020

Same here:

Linux pizero1 5.4.79+ #1373 Mon Nov 23 13:18:15 GMT 2020 armv6l GNU/Linux

Architecture:        armv6l
Byte Order:          Little Endian
CPU(s):              1
On-line CPU(s) list: 0
Thread(s) per core:  1
Core(s) per socket:  1
Socket(s):           1
Vendor ID:           ARM
Model:               7
Model name:          ARM1176
Stepping:            r0p7
CPU max MHz:         1000.0000
CPU min MHz:         700.0000
BogoMIPS:            697.95
Flags:               half thumb fastmult vfp edsp java tls

model name	: ARMv6-compatible processor rev 7 (v6l)
CPU implementer	: 0x41
CPU architecture: 7
CPU variant	: 0x0
CPU part	: 0xb76
CPU revision	: 7

Hardware	: BCM2835
Revision	: 9000c1
Model		: Raspberry Pi Zero W Rev 1.1

@brbergami
Copy link

@brbergami brbergami commented Jan 25, 2021

@sssilver Is there any kind of testing for armv6 or that's blocking the ticket? I could test it for some fix, but I have no way to compile to that architecture. PM me if you need a hand testing binaries.

@FridayJew
Copy link

@FridayJew FridayJew commented Jan 26, 2021

Same on rpi 1B , i use this https://hobin.ca/cloudflared/ builds for fix problem. Thanks for this!

@brbergami
Copy link

@brbergami brbergami commented Jan 27, 2021

Same on rpi 1B , i use this https://hobin.ca/cloudflared/ builds for fix problem. Thanks for this!

Who maintains this? I mean, this is supposed to use a secure protocol to make queries, how safe is this build archive?
The main page doesn't makes me feel precisely comfortable.

@hobindar
Copy link

@hobindar hobindar commented Jan 27, 2021

Who maintains this?

That would be me.

I mean, this is supposed to use a secure protocol to make queries, how safe is this build archive?

In my opinion, it's definitely safe. I left a comment earlier in this thread about how to build it yourself. If you follow all the instructions in that comment, I believe the resulting binaries should be bit-for-bit identical, allowing you to verify that I did not do anything malicious in my builds. If you ever find they are not identical, let me know and we can investigate why.

The main page doesn't makes me feel precisely comfortable.

Haha, yeah, I should probably change that page at some point. That's just a page that I hacked together many years ago to try out viewport units, back when viewport units were a brand new thing (and while I was still a university student). I'll replace the page when I think of something worthwhile to replace it with.

CC: @brbergami

@brbergami
Copy link

@brbergami brbergami commented Jan 27, 2021

@hobindar Thanks for clarifying.
Regarding building them by myself, I would love that, but on the device seems quite overkill to clone a repo, Golang and build, and outside I tried to achieve this using a VM and Arm Arch linux, quite a big deal..

'Lil big question there: In which these builds differs from the ones we can find in here? I can see that those are model targeted. Is just pull and build or some manual tweaking is done to deal with something extra?

@hobindar
Copy link

@hobindar hobindar commented Jan 27, 2021

Is just pull and build or some manual tweaking is done to deal with something extra?

There's no changes other than what's in that comment. It's just clone, tell go to build for ARMv6 and which cross-compiler to use, and build. No changes to source code, the binaries spat out by go, nor anything else.

@FridayJew
Copy link

@FridayJew FridayJew commented Jan 27, 2021

'Lil big question there: In which these builds differs from the ones we can find in here?

these builds working on rpi 1b. Official build for rpi can be installed but not started.

@brbergami
Copy link

@brbergami brbergami commented Jan 27, 2021

It's just clone, tell go to build for ARMv6 and which cross-compiler to use, and build.

Any nice guide to follow/recommend? Not a golang person tbh.

these builds working on rpi 1b. Official build for rpi can be installed but not started.

Will try for the sake of curiosity. Thanks!

@danielc-evans
Copy link

@danielc-evans danielc-evans commented Feb 1, 2021

Here's how to install 2018.7.2

cd ~
wget https://bin.equinox.io/a/4SUTAEmvqzB/cloudflared-2018.7.2-linux-arm.tar.gz
mkdir argo-tunnel
tar -xvzf cloudflared-2018.7.2-linux-arm.tar.gz -C ./argo-tunnel
rm cloudflared-2018.7.2-linux-arm.tar.gz
cd argo-tunnel
./cloudflared --version

@cshorler
Copy link

@cshorler cshorler commented Mar 9, 2021

Same issue on: Raspberry Pi Model B Rev 2 (/proc/cpuinfo)

On trying to compile from source, I think the golang version is too old in Debian Buster (reading some errors) - so I compiled the golang backports dpkg source version to ensure no ABI issue on the machine as I haven't set up or tested a cross compiler yet. This took more than one attempt and several attempts for tests to pass, and they never did during the build process so I overrided to create the deb anyway and manually verified the failing tests (due to timeout). I probably should have checked if Raspbian did anything to that package prior to starting... too late!

go version go1.14 linux/arm

Then compiling cloudflared
go install -work -v -buildmode=pie ./cmd/cloudflared

and I have a working executable... although I'm new to cloudflared, so I haven't really got to the stage of testing anything yet.

Did anyone else do this? Are my build settings different from the release package? can we git bisect? (once I have a cross compiler working... this is far too slow on the Pi!)

@cshorler
Copy link

@cshorler cshorler commented Mar 14, 2021

the reason this fails is the released executable is compiled to Debian armhf standards which are incompatible with the first Pi - you can see this with readelf -A

for the current release:

pi@raspberrypi:~/Downloads $ readelf -A ./cloudflared
Attribute Section: aeabi
File Attributes
  Tag_CPU_name: "7-A"
  Tag_CPU_arch: v7
  Tag_CPU_arch_profile: Application
  Tag_ARM_ISA_use: Yes
  Tag_THUMB_ISA_use: Thumb-2
  Tag_FP_arch: VFPv3-D16
  Tag_ABI_PCS_wchar_t: 4
  Tag_ABI_FP_rounding: Needed
  Tag_ABI_FP_denormal: Needed
  Tag_ABI_FP_exceptions: Needed
  Tag_ABI_FP_number_model: IEEE 754
  Tag_ABI_align_needed: 8-byte
  Tag_ABI_enum_size: int
  Tag_ABI_VFP_args: VFP registers
  Tag_CPU_unaligned_access: v6

note v7 and vfpv3, earlier Pi only have v6 and vfpv2.

here's the equivalent dump from my compilation result:

pi@raspberrypi:~/Downloads $ readelf -A /usr/local/bin/cloudflared 
Attribute Section: aeabi
File Attributes
  Tag_CPU_name: "6"
  Tag_CPU_arch: v6
  Tag_ARM_ISA_use: Yes
  Tag_THUMB_ISA_use: Thumb-1
  Tag_FP_arch: VFPv2
  Tag_ABI_PCS_wchar_t: 4
  Tag_ABI_FP_rounding: Needed
  Tag_ABI_FP_denormal: Needed
  Tag_ABI_FP_exceptions: Needed
  Tag_ABI_FP_number_model: IEEE 754
  Tag_ABI_align_needed: 8-byte
  Tag_ABI_enum_size: int
  Tag_ABI_VFP_args: VFP registers
  Tag_CPU_unaligned_access: v6

@FrancisTurner
Copy link

@FrancisTurner FrancisTurner commented Jun 7, 2021

Who maintains this?

That would be me.

I mean, this is supposed to use a secure protocol to make queries, how safe is this build archive?

In my opinion, it's definitely safe. I left a comment earlier in this thread about how to build it yourself. If you follow all the instructions in that comment, I believe the resulting binaries should be bit-for-bit identical, allowing you to verify that I did not do anything malicious in my builds. If you ever find they are not identical, let me know and we can investigate why.
CC: @brbergami

FWIW I have created a script that will check and update the armv6 cloudflared by scraping your page

https://gist.github.com/FrancisTurner/f8d53c40a201bc28328a0e8742d710dc

If you would like me to use a different way to get the data please ping me.

Usage is fairly obvious and it can be run in (root) cron as all the paths are fully specified - I added a documentation page https://francisturner.github.io/checkv6cfd.html

It assumes that cloudflared is in /usr/local/bin/ and if you ran cloudflared install to add it as a systemd service it will correctly restart that service with the new binary. Note that in such a case you will want to have disabled the systemd cloudflared-update service and timer because they pull from the cloudflare source and give you the dreaded segfault when they update.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet