Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dietpi-update failure #3555

Closed
jjmcmahon opened this issue May 22, 2020 · 33 comments
Closed

dietpi-update failure #3555

jjmcmahon opened this issue May 22, 2020 · 33 comments

Comments

@jjmcmahon
Copy link

Creating a bug report/issue

Required Information

Additional Information (if applicable)

Network Adapter Test Also Failing

Steps to reproduce

  1. run "dietpi-update" from 6.28 to 6.30

Expected behaviour

Update successful

Actual behaviour

Update Failure - Screenshot attached

Extra details

Ping google.com works
apt-update & apt-upgrade work
IPV6 is off
i attempted Joulinar's fix but that didn't work either
attempted changing repos, ntp, static/dhcp, wifi/eth

Annotation 2020-05-21 235256

Annotation 2020-05-21 235916

Annotation 2020-05-22 000212


root@DietPi:~# ping -4 -c 1 google.com
PING google.com (172.217.9.14) 56(84) bytes of data.
64 bytes from dfw28s02-in-f14.1e100.net (172.217.9.14): icmp_seq=1 ttl=56 time=20.9 ms

--- google.com ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 20.854/20.854/20.854/0.000 ms
root@DietPi:~#


@Joulinar
Copy link
Collaborator

Hi,

many thanks for your report. Your issue is not the internet connectivity in general. It looks like your have issue to resolve specific DNS like ssh.dietpi.com and github as shown on the screens above. can you try to ping them?

@jjmcmahon
Copy link
Author

jjmcmahon commented May 22, 2020

root@DietPi:~# ping -4 -c 1 ssh.dietpi.com
PING ssh.dietpi.com (185.101.93.93) 56(84) bytes of data.
64 bytes from dietpi.com (185.101.93.93): icmp_seq=1 ttl=52 time=148 ms

--- ssh.dietpi.com ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 147.641/147.641/147.641/0.000 ms
root@DietPi:~# root@DietPi:~# ping -4 -c 1 ssh.dietpi.com
PING ssh.dietpi.com (185.101.93.93) 56(84) bytes of data.
64 bytes from dietpi.com (185.101.93.93): icmp_seq=1 ttl=52 time=148 ms


root@DietPi:~# ping -4 -c 1 github.com
PING github.com (140.82.114.4) 56(84) bytes of data.
64 bytes from lb-140-82-114-4-iad.github.com (140.82.114.4): icmp_seq=1 ttl=52 time=46.8 ms

--- github.com ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 46.831/46.831/46.831/0.000 ms

@Joulinar
Copy link
Collaborator

can you try wget --spider https://raw.githubusercontent.com/MichaIng/DietPi/master/dietpi.txt

@jokoren
Copy link

jokoren commented May 24, 2020

Fixed it by:

dietpi-config
advanced options
Time sync mode
[custom]

Also trimming out on boot for tymesync, fixed.
Could be my network setup, behind 2 firewalls.

@MichaIng
Copy link
Owner

Wrong topic? #3558 (comment)

Disabling time sync without any alternative will break network access completely, at least all HTTPS traffic and that is ~99% of what we do. But if one has another time sync daemon (ntp, htpdate ntpdate (outdated), chrony, openntpd, ntpsec, ...) then systemd-timesyncd will detect it and fail to start and hence DietPi time sync mode 0/custom must be chosen indeed to not have the mentioned timeouts. I aim for a way to detect it within our script and switch to "custom" automatically.

@Joulinar
Copy link
Collaborator

yep wrong topic replay. Unfortunately it's not possible to move like on a forum 😄

@MichaIng
Copy link
Owner

Yes I am missing this feature as well.

@jjmcmahon
Copy link
Author

jjmcmahon commented May 26, 2020

can you try wget --spider https://raw.githubusercontent.com/MichaIng/DietPi/master/dietpi.txt

root@DietPi:~# wget --spider https://raw.githubusercontent.com/MichaIng/DietPi/master/dietpi.txt
Spider mode enabled. Check if remote file exists.
--2020-05-26 13:38:41-- https://raw.githubusercontent.com/MichaIng/DietPi/master/dietpi.txt
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.0.133, 151.101.64.133, 151.101.128.133, ...
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.0.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 12412 (12K) [text/plain]
Remote file exists.

@Joulinar
Copy link
Collaborator

@jjmcmahon
ok thats working. In theory the update should be able to reslove the hoste name as well. pls can you retry running dietpi-update

@jjmcmahon
Copy link
Author

[FAILED] DietPi-Set_software | Checking URL: https://raw.githubusercontent.com/MichaIng/DietPi/master/dietpi.txt



Details:

Steps to reproduce:

  1. ...
  2. ...

Expected behaviour:

  • ...

Actual behaviour:

  • ...

Extra details:

  • ...

Additional logs:

Spider mode enabled. Check if remote file exists.
--2020-05-26 15:35:58--  https://raw.githubusercontent.com/MichaIng/DietPi/master/dietpi.txt
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... failed: Connection timed out.
wget: unable to resolve host address ‘raw.githubusercontent.com’

[FAILED] DietPi-Set_software | Unable to continue, DietPi-Set_software will now terminate.
[FAILED] DietPi-Update | An error occured during dietpi.txt updates. Please check the above log or /var/tmp/dietpi/logs/dietpi-update.log for errors, and rerun "dietpi-update" after the cause has been solved.

Annotation 2020-05-26 154746

@Joulinar
Copy link
Collaborator

Joulinar commented May 26, 2020

quite strange, the command failing is
wget --spider -t 1 -T 5 https://raw.githubusercontent.com/MichaIng/DietPi/master/dietpi.txt. However if you execute it manually, it is working 🤔

Still looks like some DNS issues as the host can't be resolved
wget: unable to resolve host address ‘raw.githubusercontent.com’

But why is it working manually?

@jjmcmahon
Copy link
Author

Its really weird, i tried a lot of stuff before reverting all of my changes in an attempt to not have to post an issue. I actually did a full reinstall on a separate pi and card and i was able to install as 6.30 but was having a similar issue. One bit of info i noticed is that things seemed ok when i was on wifi but when i connected it to ethernet, That is when i saw the "internet connection test" start failing.

@Joulinar
Copy link
Collaborator

do you have connected both, WiFi as well as Ethernet connected same time (together)?

@jjmcmahon
Copy link
Author

I have tried the various methods:
Only Wifi
Only Ethernet
Both Connected
Both Enabled But Wifi Connected
Both Enabled but Ethernet Connected

When Reporting this issue here:
Ethernet: Available | On | Connected
WiFi: Not Found | On | Disconnected

After you asked, i did the following:
Ethernet: Available | On | Connected
Wifi: Not Found | Off | Disconnected

Results after disabling Wifi:

Checking URL: https://raw.githubusercontent.com/MichaIng/DietPi/master/dietpi.txt
│ - Command: wget --spider -t 1 -T 5 https://raw.githubusercontent.com/MichaIng/DietPi/master/dietpi.txt
│ - Exit code: 4 │
│ - DietPi version: v6.28.0 (MichaIng/master) | HW_MODEL: 4 | HW_ARCH: 2 | DISTRO: 5 │
│ - Image creator: DietPi Core Team │
│ - Pre-image: Raspbian Lite │
│ - Error log: │
│ Spider mode enabled. Check if remote file exists. │
│ --2020-05-27 12:50:19-- https://raw.githubusercontent.com/MichaIng/DietPi/master/dietpi.txt
│ Resolving raw.githubusercontent.com (raw.githubusercontent.com)... failed: Connection timed out. │
│ wget: unable to resolve host address ‘raw.githubusercontent.com’

Annotation 2020-05-26 154746

@jjmcmahon
Copy link
Author

jjmcmahon commented May 27, 2020

After the above i got the following:

----dietpi-update failure----



Details:

Steps to reproduce:

  1. ...
  2. ...

Expected behaviour:

  • ...

Actual behaviour:

  • ...

Extra details:

  • ...

Additional logs:

Spider mode enabled. Check if remote file exists.
--2020-05-27 12:50:19--  https://raw.githubusercontent.com/MichaIng/DietPi/master/dietpi.txt
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... failed: Connection timed out.
wget: unable to resolve host address ‘raw.githubusercontent.com’

[FAILED] DietPi-Set_software | Unable to continue, DietPi-Set_software will now terminate.
[FAILED] DietPi-Update | An error occured during dietpi.txt updates. Please check the above log or /var/tmp/dietpi/logs/dietpi-update.log for errors, and rerun "dietpi-update" after the cause has been solved.

----Ran Manually----
root@DietPi:~# wget --spider -t 1 -T 5 https://raw.githubusercontent.com/MichaIng/DietPi/master/dietpi.txt
Spider mode enabled. Check if remote file exists.
--2020-05-27 12:53:19-- https://raw.githubusercontent.com/MichaIng/DietPi/master/dietpi.txt
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... failed: Connection timed out.
wget: unable to resolve host address ‘raw.githubusercontent.com’

----Ran Manually----
root@DietPi:~# wget --spider https://raw.githubusercontent.com/MichaIng/DietPi/master/dietpi.txt
Spider mode enabled. Check if remote file exists.
--2020-05-27 12:53:49-- https://raw.githubusercontent.com/MichaIng/DietPi/master/dietpi.txt
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.0.133, 151.101.64.133, 151.101.128.133, ...
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.0.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 12412 (12K) [text/plain]
Remote file exists.

----Notes----
The second command worked but doesn't include the following: "-t 1 -T 5"

@MichaIng
Copy link
Owner

Since it "only" times out, how long does it actually take?

time wget --spider https://raw.githubusercontent.com/MichaIng/DietPi/master/dietpi.txt

Probably increasing timeout already solves the issue? dietpi-config > Network Options: Misc > G_CHECK_URL_TIMEOUT > 20 (20 seconds)

@Joulinar
Copy link
Collaborator

Joulinar commented May 27, 2020

or select change command once hit the error and remove -t 1 -T 5

maybe a latency issue on the Ethernet connection?

@MichaIng
Copy link
Owner

MichaIng commented May 27, 2020

Indeed, however since this will be not the only case likely, good to use a persistent solution.

Also, if resolving really takes more then 5 seconds, network + resolver choice should be checked since this should take less then 0.5 seconds. But I saw this quite often, not the network was the issue, but indeed the resolver itself in some cases, not sure why. But switching it could at least be tested, when using static IP already: dietpi-config > Network Options: Adapters > Ethernet > Static DNS

@prov3it
Copy link

prov3it commented May 28, 2020

Since it "only" times out, how long does it actually take?

time wget --spider https://raw.githubusercontent.com/MichaIng/DietPi/master/dietpi.txt

Probably increasing timeout already solves the issue? dietpi-config > Network Options: Misc > G_CHECK_URL_TIMEOUT > 20 (20 seconds)

I had the same issue.

Details:

Steps to reproduce:

  1. ...
  2. ...

Expected behaviour:

  • ...

Actual behaviour:

  • ...

Extra details:

  • ...

Additional logs:

Spider mode enabled. Check if remote file exists.
--2020-05-28 07:10:33--  https://raw.githubusercontent.com/MichaIng/DietPi/master/dietpi.txt
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... failed: Connection timed out.
wget: unable to resolve host address ‘raw.githubusercontent.com’

I performed all the tests jjmcmahon did, all with success. Changing the time out setting resolved the issue.

It takes:

real 0m5.359s
user 0m0.019s
sys 0m0.006s

@MichaIng
Copy link
Owner

MichaIng commented May 28, 2020

Thanks for testing. We changed the default to 10 seconds meanwhile, though this is active for new images only since we do not want to override user choices in dietpi.txt: https://github.com/MichaIng/DietPi/blob/master/dietpi.txt#L154

While this was a step to cover quite a significant number of cases where 5 seconds was/is not enough, I am still wondering where this long resolving times come from.
Do you use DHCP or static IP assignment? If static IP, which DNS nameserver did you choose, if DHCP, does name resolving take similar long on other clients, e.g. a Windows PC? To compare, the following can be used as well on Windows cmd (note the empty line to execute as well the last echo immediately):

echo %time%
curl -I raw.githubusercontent.com
echo %time%

On DietPi/Linux/UNIX shell:

time curl -I raw.githubusercontent.com

My output on Windows:

C:\WINDOWS\system32>echo %time%
10:49:59,48

C:\WINDOWS\system32>curl -I raw.githubusercontent.com
HTTP/1.1 301 Moved Permanently
Server: Varnish
Retry-After: 0
Location: https://raw.githubusercontent.com/
Content-Length: 0
Accept-Ranges: bytes
Date: Thu, 28 May 2020 08:49:59 GMT
Via: 1.1 varnish
Connection: close
X-Served-By: cache-fra19127-FRA
X-Cache: HIT
X-Cache-Hits: 0
X-Timer: S1590655800.691113,VS0,VE0
Access-Control-Allow-Origin: *
Expires: Thu, 28 May 2020 08:54:59 GMT
Vary: Authorization,Accept-Encoding

C:\WINDOWS\system32>echo %time%
10:49:59,62
  • 0.62-0.48=0.14 seconds

On DietPi:

2020-05-27 22:27:40 root@micha:/var/log# time curl -I raw.githubusercontent.com
HTTP/1.1 301 Moved Permanently
Server: Varnish
Retry-After: 0
Location: https://raw.githubusercontent.com/
Content-Length: 0
Accept-Ranges: bytes
Date: Thu, 28 May 2020 08:51:03 GMT
Via: 1.1 varnish
Connection: close
X-Served-By: cache-fra19179-FRA
X-Cache: HIT
X-Cache-Hits: 0
X-Timer: S1590655864.775779,VS0,VE0
Access-Control-Allow-Origin: *
Expires: Thu, 28 May 2020 08:56:03 GMT
Vary: Authorization,Accept-Encoding

real    0m0.128s
user    0m0.035s
sys     0m0.023s

@Joulinar
Copy link
Collaborator

Joulinar commented May 28, 2020

well usually there should not be any differences between DHCP and STATIC IP as if the DNS Server is know, the request will go to it. Doesn't matter how it was assigned. Or am I wrong? Interesting would be why resolution takes that long. And if changing DNS Server could improve situation.

BTW on my system it's as follow

real    0m0,141s
user    0m0,038s
sys     0m0,010s

@MichaIng
Copy link
Owner

MichaIng commented May 28, 2020

I just asked since with DHCP one cannot manually assign a DNS nameserver as this is assigned via DHCP then, likely the local ISP resolver or router cache. If one switched to static IP, the resolver can be switched easily to a different upstream DNS (or even using the local gateway/router instead) can be tested to solve the issue.

real 0m0,141s

Good to know that your results match mine very closely. Of your Ethernet vs WiFi plays a role as well.

@Joulinar
Copy link
Collaborator

I just asked since with DHCP one cannot manually assign a DNS nameserver as this is assigned via DHCP then

A yeah that's correct.

@prov3it
Copy link

prov3it commented May 28, 2020

Thanks for testing. We changed the default to 10 seconds meanwhile, though this is active for new images only since we do not want to override user choices in dietpi.txt: https://github.com/MichaIng/DietPi/blob/master/dietpi.txt#L154

While this was a step to cover quite a significant number of cases where 5 seconds was/is not enough, I am still wondering where this long resolving times come from.
Do you use DHCP or static IP assignment? If static IP, which DNS nameserver did you choose, if DHCP, does name resolving take similar long on other clients, e.g. a Windows PC? To compare, the following can be used as well on Windows cmd (note the empty line to execute as well the last echo immediately):

echo %time%
curl -I raw.githubusercontent.com
echo %time%

On DietPi/Linux/UNIX shell:

time curl -I raw.githubusercontent.com

Im running macOS. I use static IP assignment. Everything is wired. im connected by Fiber.

My DNS server on my DietPi is supposed to be pointing to cloudflare or my pihole, which is running on this machine as well.

Screen Shot 2020-05-28 at 12 08 06

But for whatever reason it says 10.16.0.1? Im also running a VPN server, but thats on a different subnet (172.16.0.0/24). I died try to re-apply the settings as specified and rebooted, but that didnt change.

timings on macOS:

real 0m0.376s
user 0m0.021s
sys 0m0.008s

@Joulinar
Copy link
Collaborator

what version of PiHole you are using?

@MichaIng
Copy link
Owner

MichaIng commented May 28, 2020

Please also paste the output of:

cat /etc/resolv.conf /etc/resolvconf/resolv.conf.d/*
ip a
ip r

But for whatever reason it says 10.16.0.1? Im also running a VPN server

10.* IPs are usually used by VPNs so I'm pretty sure that is your VPN-internal network, hence 10.16.0.1 is the VPN server. Depending on the VPN server/client software, it might override /etc/resolv.conf to prevent DNS leaks while connected. The menu shows the chosen DNS IPs that are applied to /etc/network/interfaces and from there applied to /etc/resolv.conf by ifup when configuring the interface. The info above the menu scrapes the first DNS nameserver entry from /etc/resolv.conf (which is the one that is effectively in use!), so a mismatch means that something else is overriding it after ifup.

@prov3it
Copy link

prov3it commented May 29, 2020

10.16. probably was the vpn server, when i changed it to 172.16. because of the usage of multiple VPN connections and overlapping subnets.

cat /etc/resolv.conf /etc/resolvconf/resolv.conf.d/*

# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
#     DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
nameserver 10.16.0.1
nameserver 1.1.1.1
nameserver 1.0.0.1
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
#     DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
domain fritz.box
search fritz.box
nameserver 192.168.178.1

I dont and never owned a fritz box router/modem.

ip a

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq state UP group default qlen 1000
    link/ether a8:a1:59:04:c7:b0 brd ff:ff:ff:ff:ff:ff
    inet 192.168.2.20/24 brd 192.168.2.255 scope global eth0
       valid_lft forever preferred_lft forever
3: tun0: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc fq state UNKNOWN group default qlen 100
    link/none 
    inet 172.16.0.1/24 brd 172.16.0.255 scope global tun0
       valid_lft forever preferred_lft forever

ip r

default via 192.168.2.254 dev eth0 onlink 
172.16.0.0/24 dev tun0 proto kernel scope link src 172.16.0.1 
192.168.2.0/24 dev eth0 proto kernel scope link src 192.168.2.20 

what version of PiHole you are using?

the latest.

@MichaIng
Copy link
Owner

MichaIng commented May 29, 2020

domain fritz.box
search fritz.box
nameserver 192.168.178.1

This must be the content of a backup file when resolvconf was installed, might be even from my building VM, since I had a Fritz!Box router 😅. However it has no effect, the effective entries are:

nameserver 10.16.0.1
nameserver 1.1.1.1
nameserver 1.0.0.1

Could you try:

ifup --force eth0
cat /etc/resolv.conf

This should update the resolver based on static DNS entry in /etc/network/interfaces.

@prov3it
Copy link

prov3it commented May 30, 2020

ifup --force eth0

RTNETLINK answers: File exists

I tried:

ifdown eth0 && ifup --force eth0

cat /etc/resolv.conf

Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)#     DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
nameserver 10.16.0.1
nameserver 1.1.1.1
nameserver 1.0.0.1

@MichaIng
Copy link
Owner

MichaIng commented May 30, 2020

@prov3it
Hmm, somehow resolv.conf is not touched, that is strange. Can you paste the output of:

cat /etc/network/interfaces{,.d/*}

@MichaIng
Copy link
Owner

I mark this as closed. Feel free to reopen if issue is still present.

@prov3it
Copy link

prov3it commented Nov 17, 2020

Okay, i haven't been contributing to this issue anymore, but its still present. In my case the issue is originating from having set up a openvpn server and alter the subnet in the /etc/openvpn/server.conf file. The old server address was still being used for dns. doing a grep -Ril "10.16.0.1" /etc gave me all the files containing this pattern:
/etc/resolvconf/run/interface/tun1.openvpn
/etc/resolvconf/run/resolv.conf
/etc/resolv.conf
/etc/netns/protected/resolv.conf

Removing /etc/resolvconf/run/interface/tun1.openvpn was not an option. Its recreated after reboot. /etc/resolvconf/run/resolv.conf is linked to /etc/resolv.conf.

Altering /etc/resolv.conf was my temp solution.

To answer your previous question:
cat /etc/network/interfaces{.d/*} doesnt give me any output. Directory {.d} is empty. /etc/network/interfaces:

# Location: /etc/network/interfaces
# Please modify network settings via: dietpi-config
# Or create your own drop-ins in: /etc/network/interfaces.d/

# Drop-in configs
source interfaces.d/*

# Local
auto lo
iface lo inet loopback

# Ethernet
allow-hotplug eth0
iface eth0 inet static
address 192.168.2.20
netmask 255.255.255.0
gateway 192.168.2.254
dns-nameservers 192.168.2.20

# WiFi
#allow-hotplug wlan0
iface wlan0 inet dhcp
address 0.0.0.0
netmask 0.0.0.0
gateway 0.0.0.0
#dns-nameservers 0.0.0.0
wpa-conf /etc/wpa_supplicant/wpa_supplicant.conf

The strangest thing is, 10.16.0.* is not the subnet used by pivpn, its 10.8.0.. I changed it to 172.16.0.. Where is this 10.16.0.1 coming from?!

@MichaIng
Copy link
Owner

/etc/resolvconf/run/interface/tun1.openvpn seems to be what induces the nameserver entry but that should be reset and reboot at latest and otherwise be part of the OpenVPN config somewhere. Or probably it has been placed by PiVPN installer somehow?

Not sure how this /etc/resolvconf/run/interface works at all, cannot find something about it in the man pages. Also I wonder why it is tun1 instead of tun0 (the default) 🤔.

And for the VPN server it does not make any sense to mess with the DNS entries at all, while this only makes sense for the clients to prevent DNS leaks... I'm confused here as well. I'd go through all OpenVPN and PiVPN settings and as well remove /etc/resolvconf/run/interface/tun1.openvpn to see if/when it reappears, e.g. if restarting OpenVPN does it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants