Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open Beta v6.26 | Please help testing and hardening the upcoming release #3131

Closed
MichaIng opened this issue Sep 29, 2019 · 26 comments
Closed
Assignees
Labels
Beta 🧪 Issues specific to the Beta branch testing Information ℹ️
Milestone

Comments

@MichaIng
Copy link
Owner

MichaIng commented Sep 29, 2019

RC version v6.26.3
Changelog https://github.com/Fourdee/DietPi/blob/beta/CHANGELOG.txt
Code changes Fourdee/DietPi@master...beta
v6.26.0 => v6.26.1 #3141
v6.26.1 => v6.26.2 #3145
v6.26.2 => v6.26.3 #3155
How to apply https://github.com/Fourdee/DietPi/blob/master/BRANCH_SYSTEM.md
Release planned Early October

Important testing cases:


Closes issues:

#1171 #2332 #2641 #2935 #2959 #2962 #2965 #2966 #2975 #2977 #2994 #2996 #2997 #2999 #3007 #3010 #3017 #3024 #3028 #3029 #3031 #3036 #3038 #3045 #3053 #3059 #3060 #3062 #3066 #3070 #3075 #3079 #3085 #3094 #3097 #3098 #3104 #3105 #3111 #3116 #3122 #3123 #3126 #3127 #3128 #3144 #3147 #3151


Known issues

DietPi functionality

  • DietPi-Config | Enabling WiFi + Ethernet adapters, both on different subnets, breaks WiFi connection in some cases: https://github.com/Fourdee/DietPi/issues/2103
    • Current workaround is to disable the adapter that is not in use (not for internet connection).
    • Otherwise a custom routing table is required.

SBC/device related

Software title related

@MichaIng MichaIng added Testing/testers required 🔽 Information ℹ️ Beta 🧪 Issues specific to the Beta branch testing labels Sep 29, 2019
@MichaIng MichaIng added this to the v6.26 milestone Sep 29, 2019
@MichaIng MichaIng pinned this issue Sep 29, 2019
@chinyongcy
Copy link

chinyongcy commented Sep 30, 2019

Currently running on Rpi3 . Keep getting error message on kodi installation.
But I am able to start kodi by typing "kodi" in my console.
However, if I want to install other stuff, the below error msg for kodi will pop up.
Seems like an small issue, but it is kind of irritating to have that message pop up every seconds.
Help appreciated! Thanks in advanced :)

  • EDIT: Also tried "apt remove kodi kodi-bin and apt autoremove --purge" before reinstalling. Doesn't work.

`#### Details:

  • Date | Mon 30 Sep 04:28:53 BST 2019
  • Bug report | N/A
  • DietPi version | v6.26.0 (MichaIng/beta)
  • Image creator | DietPi Core Team
  • Pre-image | Raspbian Lite
  • SBC device | RPi 3 Model B (armv7l) (index=3)
  • Kernel version | Linux Pi3 4.19.66-v7+ DietPi-Config | CPU performance benchmark #1253 SMP Thu Aug 15 11:49:46 BST 2019 armv7l GNU/Linux
  • Distro | buster (index=5)
  • Command | G_AGI libcec4 libcurl3-gnutls libnfs12 kodi
  • Exit code | 100
  • Software title | DietPi-Software

Steps to reproduce:

  1. ...
  2. ...

Expected behaviour:

  • ...

Actual behaviour:

  • ...

Extra details:

  • ...

Additional logs:

Setting up libjbig0:armhf (2.1-3.1+b2) ...
Setting up libmicrohttpd12 (0.9.62-1) ...
Setting up libwbclient0:armhf (2:4.9.5+dfsg-5+deb10u1+rpi1) ...
Setting up libjansson4:armhf (2.12-1) ...
Setting up libtalloc2:armhf (2.1.14-2) ...
Setting up mariadb-common (1:10.3.17-0+deb10u1) ...
update-alternatives: using /etc/mysql/mariadb.cnf to provide /etc/mysql/my.cnf (my.cnf) in auto mode
Setting up libjpeg62-turbo:armhf (1:1.5.2-2+b1) ...
Setting up libtevent0:armhf (0.9.37-1) ...
Setting up liblirc-client0:armhf (0.10.1-5.2) ...
Setting up libfribidi0:armhf (1.0.5-3.1) ...
Setting up libimagequant0:armhf (2.12.2-1.1) ...
Setting up libwebp6:armhf (0.6.1-2) ...
Setting up libmariadb3:armhf (1:10.3.17-0+deb10u1) ...
Setting up libaacs0:armhf (0.9.0-2) ...
Setting up libtinyxml2.6.2v5:armhf (2.6.2-4) ...
Setting up libcups2:armhf (2.2.10-6+deb10u1) ...
Setting up libsndio7.0:armhf (1.5.0-3) ...
Setting up libgif7:armhf (5.1.4-3) ...
Setting up python-bluez (0.22+really0.22-1) ...
Setting up libtag1v5:armhf (1.11.1+dfsg.1-0.3) ...
Setting up libasyncns0:armhf (0.8-6) ...
Setting up libxslt1.1:armhf (1.1.32-2.1~deb10u1) ...
Setting up libharfbuzz0b:armhf (2.3.1-1) ...
Setting up libtiff5:armhf (4.0.10-4) ...
Setting up libsndfile1:armhf (1.0.28-6) ...
Setting up libbluray2:armhf (1:1.1.0-1) ...
Setting up libshairplay0:armhf (0.9.0+git20180824.096b61a-1~buster) ...
Setting up python-talloc:armhf (2.1.14-2) ...
Setting up libwebpmux3:armhf (0.6.1-2) ...
Setting up libfstrcmp0 (0.7.D001-1.1) ...
Setting up libxkbcommon0:armhf (0.8.2-1) ...
Setting up libldb1:armhf (2:1.5.1+really1.4.6-3) ...
Setting up libiso9660-11:armhf (2.0.0-2) ...
Setting up libpulse0:armhf (12.2-4+deb10u1) ...
Setting up libwebpdemux2:armhf (0.6.1-2) ...
Setting up samba-libs:armhf (2:4.9.5+dfsg-5+deb10u1+rpi1) ...
Setting up python-pil:armhf (5.4.1-2) ...
Setting up libass9:armhf (1:0.14.0-2) ...
Setting up libpulse-mainloop-glib0:armhf (12.2-4+deb10u1) ...
Setting up libsmbclient:armhf (2:4.9.5+dfsg-5+deb10u1+rpi1) ...
Setting up kodi-bin (2:18.4-5~buster) ...
Setting up kodi (2:18.4-5~buster) ...
dpkg: error processing package kodi (--configure):
 installed kodi package post-installation script subprocess returned error exit status 1
Processing triggers for libc-bin (2.28-10+rpi1) ...
Processing triggers for mime-support (3.62) ...
Errors were encountered while processing:
 kodi
E: Sub-process /usr/bin/dpkg returned an error code (1)

`

@Joulinar
Copy link
Collaborator

Joulinar commented Sep 30, 2019

@MichaIng

I was running the update 2 times now in my HyperV VM. Both times I got the following error message

Details:

  • Date | Mon 30 Sep 12:16:58 BST 2019
  • Bug report | N/A
  • DietPi version | v6.25.3 (MichaIng/master)
  • Img creator | yumiris
  • Pre-image | DietPi for VMware
  • SBC device | Virtual Machine (x86_64) (index=20)
  • Kernel version | #1 SMP Debian 4.9.168-1+deb9u5 (2019-08-11)
  • Distro | stretch (index=4)
  • Command | G_AGUP
  • Exit code | 100
  • Software title | DietPi-Update

Steps to reproduce:

  1. G_CONFIG_INJECT 'DEV_GITBRANCH=' 'DEV_GITBRANCH=beta' /DietPi/dietpi.txt
  2. dietpi-update

Expected behaviour:

  • update to v6.26.0

Actual behaviour:

  • received error message as below

Additional logs:

Log file contents:
Get:52 https://cdn-aws.deb.debian.org/debian stretch-backports/main i386 Packages 2019-09-07-0214.23.pdiff [850 B]
Get:53 https://cdn-aws.deb.debian.org/debian stretch-backports/main i386 Packages 2019-09-10-1410.33.pdiff [218 B]
Get:54 https://cdn-aws.deb.debian.org/debian stretch-backports/main i386 Packages 2019-09-10-2013.46.pdiff [296 B]
Get:55 https://cdn-aws.deb.debian.org/debian stretch-backports/main i386 Packages 2019-09-11-0218.44.pdiff [19.2 kB]
Get:56 https://cdn-aws.deb.debian.org/debian stretch-backports/main i386 Packages 2019-09-13-0222.08.pdiff [627 B]
Get:57 https://cdn-aws.deb.debian.org/debian stretch-backports/main i386 Packages 2019-09-16-1418.37.pdiff [569 B]
Get:58 https://cdn-aws.deb.debian.org/debian stretch-backports/main i386 Packages 2019-09-20-0823.35.pdiff [4,303 B]
Get:59 https://cdn-aws.deb.debian.org/debian stretch-backports/main i386 Packages 2019-09-20-1410.38.pdiff [1,040 B]
Get:60 https://cdn-aws.deb.debian.org/debian stretch-backports/main i386 Packages 2019-09-20-2015.11.pdiff [652 B]
Get:61 https://cdn-aws.deb.debian.org/debian stretch-backports/main i386 Packages 2019-09-26-1413.34.pdiff [1,019 B]
Get:62 https://cdn-aws.deb.debian.org/debian stretch-backports/main i386 Packages 2019-09-28-2018.42.pdiff [945 B]
Get:62 https://cdn-aws.deb.debian.org/debian stretch-backports/main i386 Packages 2019-09-28-2018.42.pdiff [945 B]
Get:63 https://cdn-aws.deb.debian.org/debian stretch-backports/main Translation-en 2019-08-22-1415.40.pdiff [921 B]
Get:64 https://cdn-aws.deb.debian.org/debian stretch-backports/main Translation-en 2019-08-23-1430.55.pdiff [1,653 B]
Get:65 https://cdn-aws.deb.debian.org/debian stretch-backports/main Translation-en 2019-08-25-0819.10.pdiff [570 B]
Get:66 https://cdn-aws.deb.debian.org/debian stretch-backports/main Translation-en 2019-08-26-0220.54.pdiff [246 B]
Get:67 https://cdn-aws.deb.debian.org/debian stretch-backports/main Translation-en 2019-08-28-1426.17.pdiff [383 B]
Get:68 https://cdn-aws.deb.debian.org/debian stretch-backports/main Translation-en 2019-09-05-0816.52.pdiff [1,127 B]
Get:69 https://cdn-aws.deb.debian.org/debian stretch-backports/main Translation-en 2019-09-13-0830.13.pdiff [465 B]
Get:70 https://cdn-aws.deb.debian.org/debian stretch-backports/main Translation-en 2019-09-16-1418.37.pdiff [499 B]
Get:71 https://cdn-aws.deb.debian.org/debian stretch-backports/main Translation-en 2019-09-20-0823.35.pdiff [3,109 B]
Get:72 https://cdn-aws.deb.debian.org/debian stretch-backports/main Translation-en 2019-09-22-2014.03.pdiff [571 B]
Get:73 https://cdn-aws.deb.debian.org/debian stretch-backports/main Translation-en 2019-09-25-2025.12.pdiff [245 B]
Get:74 https://cdn-aws.deb.debian.org/debian stretch-backports/main Translation-en 2019-09-26-1413.34.pdiff [820 B]
Get:75 https://cdn-aws.deb.debian.org/debian stretch-backports/main Translation-en 2019-09-27-0212.46.pdiff [524 B]
Get:75 https://cdn-aws.deb.debian.org/debian stretch-backports/main Translation-en 2019-09-27-0212.46.pdiff [524 B]
Get:76 https://cdn-aws.deb.debian.org/debian stretch-backports/non-free i386 Packages 2019-09-16-1418.37.pdiff [2,523 B]
Get:76 https://cdn-aws.deb.debian.org/debian stretch-backports/non-free i386 Packages 2019-09-16-1418.37.pdiff [2,523 B]
Get:77 https://cdn-aws.deb.debian.org/debian stretch-backports/non-free amd64 Packages 2019-09-16-1418.37.pdiff [2,523 B]
Get:77 https://cdn-aws.deb.debian.org/debian stretch-backports/non-free amd64 Packages 2019-09-16-1418.37.pdiff [2,523 B]
Get:78 https://cdn-aws.deb.debian.org/debian stretch-backports/non-free Translation-en 2019-09-16-1418.37.pdiff [514 B]
Get:78 https://cdn-aws.deb.debian.org/debian stretch-backports/non-free Translation-en 2019-09-16-1418.37.pdiff [514 B]
Get:79 https://cdn-aws.deb.debian.org/debian-security stretch/updates/main amd64 Packages [499 kB]
Get:80 https://cdn-aws.deb.debian.org/debian-security stretch/updates/main i386 Packages [499 kB]
Get:81 https://cdn-aws.deb.debian.org/debian-security stretch/updates/main Translation-en [220 kB]
Get:82 https://cdn-aws.deb.debian.org/debian sid/main i386 Packages [8,181 kB]
Get:83 https://cdn-aws.deb.debian.org/debian sid/main amd64 Packages [8,247 kB]
Get:84 https://cdn-aws.deb.debian.org/debian stretch/main i386 Packages [7,066 kB]
Get:85 https://cdn-aws.deb.debian.org/debian stretch/main amd64 Packages [7,086 kB]
Get:86 https://cdn-aws.deb.debian.org/debian stretch/main Translation-en [5,385 kB]
Get:87 https://cdn-aws.deb.debian.org/debian stretch/contrib i386 Packages [48.0 kB]
Get:88 https://cdn-aws.deb.debian.org/debian stretch/contrib amd64 Packages [50.9 kB]
Get:89 https://cdn-aws.deb.debian.org/debian stretch/non-free i386 Packages [69.4 kB]
Get:90 https://cdn-aws.deb.debian.org/debian stretch/non-free amd64 Packages [78.3 kB]
Get:91 https://cdn-aws.deb.debian.org/debian stretch/non-free Translation-en [80.3 kB]
Get:92 https://cdn-aws.deb.debian.org/debian sid/main Translation-en [6,187 kB]
Fetched 44.9 MB in 18s (2,466 kB/s)
Reading package lists...
E: Failed to fetch https://deb.debian.org/debian/dists/sid/main/binary-amd64/PackagesIndex  Couldn't parse pdiff index
E: Some index files failed to download. They have been ignored, or old ones used instead.

However after I closed the error message dialog, the update continue and seems finalized

DietPi-Update
─────────────────────────────────────────────────────
 Mode: Completed

[ INFO ] DietPi-Update | Current version : v6.25.3 (branch: MichaIng/master)
[ INFO ] DietPi-Update | Latest version  : v6.26.0 (branch: MichaIng/beta)

@MichaIng
Copy link
Owner Author

@chinyongcy
Many thanks for testing. Yes this is APT trying to finish the failing post-install configuration step before doing any other installs.

Could you please try and paste the resolut of: dpkg --configure kodi
And if if fails, probably the first lines of journalctl -r give a hint. I will also run some test later today to see what this postinst script actually tries to do.

@Joulinar
Could you please paste the first lines of apt-get clean && apt-get update?
It seems the Sid repo comes with some structure that is not compatible with Stretch, at least when its about the PDiffs, not sure. You have WireGuard installed, right?
In this case, what you could try (what also the update does at later stage):

sed -i 's/sid/bullseye/g' /etc/apt/sources.list.d/dietpi-wireguard.list
apt-get clean
apt-get update

@Joulinar
Copy link
Collaborator

@MichaIng

short question: should I run the commands before or after updating to 6.26.0??

@chinyongcy
Copy link

chinyongcy commented Sep 30, 2019

@chinyongcy
Many thanks for testing. Yes this is APT trying to finish the failing post-install configuration step before doing any other installs.

Could you please try and paste the resolut of: dpkg --configure kodi
And if if fails, probably the first lines of journalctl -r give a hint. I will also run some test later today to see what this postinst script actually tries to do.

Thank you for your help! Before I went to sleep just now, pass my ssh access to my friend.
He told me manage to fix it by inserting this line

rm /var/lib/dpkg/info/kodi*
ls -l /var/lib/dpkg/info | grep kodi
G_AGI kodi

I have no idea what it meant, but it fixed the issues.
He told me "I think the package info is corrupted".

Afterwards, I go to dietpi-software, under software optimised, kodi is no registered as install.
I guess it have to do with my friend installing it in the manual way.
I install again via dietpi-software with no issues.

EDIT: I tried a clean install, it will still give me the same error as my first post. Have to go through this 3 lines to fix it.

@MichaIng
Copy link
Owner Author

MichaIng commented Sep 30, 2019

@chinyongcy
Okay thanks for feedback. The steps remove the package post-install/config scripts. The commands I provided earlier should give a hint which config step exactly failed and why. However I will run those tests later as well.
Since dietpi-software terminates due to this error, it does not save Kodi as installed. Rerunning the install indeed redoes this then.

Another question since you ran "kodi" from console. Does "start_kodi" or "kodi --standalone" work as well?

@Joulinar
The update fails if G_AGUP/apt-get update fail, thus you need to fix this before running the update. The first lines of the last command "apt-get update" should show more details about which repo (most likely "Sid") fails with which error. I will run some tests later as well.

@chinyongcy
Copy link

chinyongcy commented Sep 30, 2019

@MichaIng Typing "kodi --standalone" or "kodi" manage to run the kodi successfully but, it will give me /usr/bin/kodi: 1: /usr/bin/kodi: fbset: not found

However if I type "start_kodi"
it will return "-bash: start_kodi: command not found"

anyway what is the difference between "kodi --standalone" and "kodi"?

Also another update is, I'm trying to install Netflix on Kodi.
I ran this code to install the necessary stuff
sudo apt-get install python-pip python-crypto build-essential -y && sudo apt-get install python-all-dev python-setuptools python-wheel -y && sudo apt-get install python-crypto-dbg python-crypto-doc python-pip-whl -y

I am able to see
dpkg: warning: files list file for package 'kodi' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'kodi-bin' missing; assuming package has no files currently installed

will this be an issue?

and I uninstall kodi from the dietpi-software and just tried your command, I am not sure if it will still helps
dpkg --configure kodi

dpkg: error processing package kodi (--configure):
package kodi is already installed and configured
Errors were encountered while processing:
kodi

After which "journalctl -r"


> Sep 30 23:24:01 DietPi sudo[4320]: pam_unix(sudo:session): session closed for user root
> Sep 30 23:24:01 DietPi sudo[4320]: pam_unix(sudo:session): session opened for user root by root(uid=0)
> Sep 30 23:24:01 DietPi sudo[4320]:     root : TTY=pts/2 ; PWD=/root ; USER=root ; COMMAND=/usr/bin/dpkg --configure kodi
> Sep 30 23:19:17 DietPi sshd[3624]: pam_unix(sshd:session): session opened for user root by (uid=0)
> Sep 30 23:19:17 DietPi sshd[3624]: Accepted password for root from 192.168.0.58 port 54168 ssh2

@MichaIng
Copy link
Owner Author

MichaIng commented Sep 30, 2019

@chinyongcy
Ah yeah, indeed the commands from your friend removed the status files, thus APT and other scripts cannot find it installed anymore. Definitely not optimal, so we need to find a solution for the initial failure.

kodi --standalone should serve better performance since kodi always runs a desktop environment in the background, AFAIK. At least it is meant to run from desktop. However on RPi kodi --standalone can be used from console and desktop as well.

Good to know about fbset still required for this binary, or at least it creates this warnings. You can solve with G_AGI fbset.

To replicate the configuration/install error (re-create the files you removed), you need to run apt install --reinstall kodi.


For the start_kodi command one reboot (or relogging into fresh console session) is required, after dietpi-software tracked it as installed. However it simply is a wrapper for kodi --standalone on RPi.

@chinyongcy
Copy link

chinyongcy commented Sep 30, 2019

@MichaIng
Thank you very much for the detail explanation and solution.

I have a theory, not sure if it make sense... Previously when I was not in beta version, I saw your thread on #3031. Follow it last week and manage to install with no issues. Yesterday was fiddling with something and somehow corrupted my SD card. So did a fresh install anyway. Even the solution posted in #3031 will result in the same problem faced.

So I am guessing the problem lies with kodi newer not with dietpi? Currently is at version "kodi_18.4-5_buster_all". The previous version is "kodi_18.4-4_buster_all". Not sure if the "4-5" is causing an issue.

I tried installing older version by myself, I think I didn't do it correctly.

 wget http://pipplware.pplware.pt/pipplware/dists/buster/main/binary/kodi_18.4-4~buster_all.deb
 sudo dpkg -i kodi_18.4-4~buster_all.deb

After that I get back

dpkg: warning: downgrading kodi from 2:18.4-5~buster to 2:18.4-4~buster
(Reading database ... 24506 files and directories currently installed.)
Preparing to unpack kodi_18.4-4~buster_all.deb ...
Unpacking kodi (2:18.4-4~buster) over (2:18.4-5~buster) ...
dpkg: dependency problems prevent configuration of kodi:
 kodi depends on kodi-bin (<< 2:18.4-4~buster.1~); however:
  Version of kodi-bin on system is 2:18.4-5~buster.
  Version of kodi-bin on system, provided by kodi-bin:armhf, is <none>.

dpkg: error processing package kodi (--install):
 dependency problems - leaving unconfigured
Processing triggers for mime-support (3.62) ...
Errors were encountered while processing:
 kodi

Can't seem to get it to work as well.

@MichaIng
Copy link
Owner Author

MichaIng commented Sep 30, 2019

@chinyongcy
This is now due to dependencies. Since the kodi-bin package is version 18.4-5, the kodi package version 18.4-4 is not compatible with it. So you would need to pull as well the older package of kodi-bin and install both in one run.

However upgrading from an older to a newer package should never cause issues, as this would break every apt-get upgrade. If it failed as well on a fresh image (after upgrading to beta, before doing any other Kodi install attempt), then it is probably due to some missing dependency that is not listed in the package stats, and yes, probably something that changed with the newer version. However generally it should work, otherwise I would expect some error reports here: https://www.raspberrypi.org/forums/viewtopic.php?f=66&t=251645

@chinyongcy
Copy link

@MichaIng Yup, everything is working fine, just that the error message is there only. Thank you once again!

@Joulinar
Copy link
Collaborator

Joulinar commented Oct 1, 2019

@MichaIng Bad news, I'm not able to reproduce the error. Yesterday I had some issues with my HyperV VM and created a new one today, leading to the fact, that all Software title needed to be reinstall from scratch. Installing Wireguard, already forced G_AGUP. This finished without issues and I was able to run DietPi update to 6.26.0. Compare to yesterday, DietPi update was very quick and short. I guess installing Wireguard performed the major steps already.

Anyway, I now have another issue which is not related to the update. After the initial installation of Wireguard, the wg0 adapter is not coming up.

@MichaIng
Copy link
Owner Author

MichaIng commented Oct 1, 2019

@Joulinar
Okay just testing:

  • Buster VM => Sid repo works well
  • Stretch VM => Sid repo works fine as well 🤔
root@VM-Stretch:~# apt update
Ign:1 https://cdn-aws.deb.debian.org/debian stretch InRelease
Get:2 https://cdn-aws.deb.debian.org/debian sid InRelease [139 kB]
Get:3 https://cdn-aws.deb.debian.org/debian stretch-updates InRelease [91.0 kB]
Get:4 https://cdn-aws.deb.debian.org/debian-security stretch/updates InRelease [94.3 kB]
Get:5 https://cdn-aws.deb.debian.org/debian stretch Release [118 kB]
Get:6 https://cdn-aws.deb.debian.org/debian stretch Release.gpg [2,365 B]
Get:7 https://cdn-aws.deb.debian.org/debian sid/main amd64 Packages [8,249 kB]
Get:8 https://cdn-aws.deb.debian.org/debian stretch-updates/main amd64 Packages [27.4 kB]
Get:9 https://cdn-aws.deb.debian.org/debian-security stretch/updates/main amd64 Packages [499 kB]
Get:10 https://cdn-aws.deb.debian.org/debian-security stretch/updates/non-free amd64 Packages [1,596 B]
Get:11 https://cdn-aws.deb.debian.org/debian-security stretch/updates/contrib amd64 Packages [1,760 B]
Get:12 https://cdn-aws.deb.debian.org/debian stretch/main amd64 Packages [7,086 kB]
Get:13 https://cdn-aws.deb.debian.org/debian stretch/non-free amd64 Packages [78.3 kB]
Get:14 https://cdn-aws.deb.debian.org/debian stretch/contrib amd64 Packages [50.9 kB]
Fetched 16.4 MB in 9s (1,658 kB/s)
Reading package lists... Done
Building dependency tree
Reading state information... Done
173 packages can be upgraded. Run 'apt list --upgradable' to see them.
  • As you can see I disabled i386 arch and state translation downloads. Enabling both works fine as well.

But from your error it is about the PDiffs:

E: Failed to fetch https://deb.debian.org/debian/dists/sid/main/binary-amd64/PackagesIndex Couldn't parse pdiff index

This is hard to replicate since the required PDiffs (differential package lists) depend on the state of the currently loaded lists.
When comparing the different distros, it seems that indeed with Bullseye the way how those PDiffs are stored on the repo are different:

The last two do not contain this Packages.diff dir. I am not 100% sure but it would be bad but possible if Stretch and Buster APT would now be always incompatible to the Sid and Bullseye repo, when PDiffs are in use. The switch from Sid to Bullseye only solves the issue temporarily, since a complete new package list is downloaded instead of PDiffs.

I leave now two machines (Buster and Stretch) running until the end of the day and see if I can replicate the issue when some new PDiff is created. In case we need to check whether this is known and expected or otherwise report to Debian bug tracker.


So to lower the chance that users run into this during v6.26 update and give us some time to find a solution, I will now move the Sid => Bullseye switch for WireGuard to pre-patches, as well as disabling the package state downloads. Both is already done during regular incremental patches but those run after G_AGUP, thus the "fix" comes after the error terminates the updater.

@Joulinar
Copy link
Collaborator

Joulinar commented Oct 1, 2019

@MichaIng Yes it seems it depends on how the systems looks before running the update. Unfortunately, I lost my VM yesterday and I'm not able to reproduce the error as well.

@MichaIng
Copy link
Owner Author

MichaIng commented Oct 1, 2019

@Joulinar
What you mean by "lost my VM"? This was not due to the the APT error, right? Since this should only exit the update but not cause further issues.

I have moved some APT-affecting patches to pre-patches now, so they are applied before G_AGUP and G_AGUG are called. Since "sid" is replaced with "bullseye" for WireGuard, this resolves the issue for the initial call, however it might reappear on a future apt-get update/G_AGUP, so we need to keep eyes open.
81e4f57
b7d3c22


EDIT

Another PDiff has been created, apt-get update still succeeded on my test VMs:

root@VM-Stretch:~# apt-get update
Ign:1 https://cdn-aws.deb.debian.org/debian stretch InRelease
Get:2 https://cdn-aws.deb.debian.org/debian sid InRelease [139 kB]
Get:3 https://cdn-aws.deb.debian.org/debian stretch-updates InRelease [91.0 kB]
Hit:4 https://cdn-aws.deb.debian.org/debian-security stretch/updates InRelease
Hit:5 https://cdn-aws.deb.debian.org/debian stretch Release
Get:6 https://cdn-aws.deb.debian.org/debian sid/main i386 Packages.diff/Index [27.9 kB]
Get:7 https://cdn-aws.deb.debian.org/debian sid/main amd64 Packages.diff/Index [27.9 kB]
Get:8 https://cdn-aws.deb.debian.org/debian sid/main i386 Packages 2019-10-01-1409.16.pdiff [36.2 kB]
Get:8 https://cdn-aws.deb.debian.org/debian sid/main i386 Packages 2019-10-01-1409.16.pdiff [36.2 kB]
Get:9 https://cdn-aws.deb.debian.org/debian sid/main amd64 Packages 2019-10-01-1409.16.pdiff [30.6 kB]
Get:9 https://cdn-aws.deb.debian.org/debian sid/main amd64 Packages 2019-10-01-1409.16.pdiff [30.6 kB]
Fetched 352 kB in 8s (41.2 kB/s)
Reading package lists... Done
root@VM-Buster:/tmp# apt-get update
Hit:1 https://cdn-aws.deb.debian.org/debian buster InRelease
Get:2 https://cdn-aws.deb.debian.org/debian sid InRelease [139 kB]
Get:3 https://cdn-aws.deb.debian.org/debian buster-updates InRelease [49.3 kB]
Hit:4 https://cdn-aws.deb.debian.org/debian-security buster/updates InRelease
Get:5 https://cdn-aws.deb.debian.org/debian buster/main Translation-en [5,967 kB]
Get:6 https://cdn-aws.deb.debian.org/debian sid/main amd64 Packages.diff/Index [27.9 kB]
Get:7 https://cdn-aws.deb.debian.org/debian buster/contrib Translation-en [44.7 kB]
Get:8 https://cdn-aws.deb.debian.org/debian buster/non-free Translation-en [88.7 kB]
Get:9 https://cdn-aws.deb.debian.org/debian sid/main Translation-en [6,188 kB]
Get:10 https://cdn-aws.deb.debian.org/debian sid/contrib Translation-en [50.3 kB]
Get:11 https://cdn-aws.deb.debian.org/debian sid/non-free Translation-en [94.4 kB]
Get:12 https://cdn-aws.deb.debian.org/debian buster-updates/main Translation-en [3,655 B]
Get:13 https://cdn-aws.deb.debian.org/debian-security buster/updates/main Translation-en [58.5 kB]
Get:14 https://cdn-aws.deb.debian.org/debian sid/main amd64 Packages 2019-10-01-1409.16.pdiff [30.6 kB]
Get:14 https://cdn-aws.deb.debian.org/debian sid/main amd64 Packages 2019-10-01-1409.16.pdiff [30.6 kB]
Fetched 12.7 MB in 19s (669 kB/s)
Reading package lists... Done

So at least it is not a general incompatibility.

Another thing we can do is offering a full APT list reset on G_AGUP failures:

/DietPi/dietpi/func/dietpi-set_software apt-cache clean
  • This removes not only cache, but as well all list files. So no PDiffs will be downloaded afterwards but only the full list files.

@Joulinar
Copy link
Collaborator

Joulinar commented Oct 1, 2019

@MichaIng ahh no no. It was my fault. I deleted my save points on my VM, instead of restoring it. Therefore I lost my v6.25.3 backups. To get back to v6.25.3, it was needed to create new VM from scratch, unfortunately. Means no chance to test on the previous scenario, where it failed.

Thx for looking into it :)

@MichaIng
Copy link
Owner Author

MichaIng commented Oct 7, 2019

Beta v6.26.1 has been merged: #3141

@MichaIng
Copy link
Owner Author

Beta v6.26.2 has been merged: #3145

@MichaIng
Copy link
Owner Author

MichaIng commented Oct 14, 2019

Beta v6.26.3 has been merged: #3155

@Vansinnet
Copy link

Working quite fine with 6.26.3. However, one of my RPi 4s won’t boot with overclock set to 1900mhz and overvoltage 4. It booted without problems with 6.25.3. I think the new boot setting where it boots on max mhz for 20 seconds is the culprit. The same machine boots fine when I set the overclock to 1750 MHz and overvoltage 2.

I suppose this specific machine wasn’t as rock stable as I previously thought.

@MichaIng
Copy link
Owner Author

MichaIng commented Oct 15, 2019

@Vansinnet
Many thanks for testing and reporting.

Could you play around a bid (after doing a backup!), e.g.:

  • Disable initial_turbo and see if it boots fine then
  • Run a stress test to see if with 1900 MHz + overvoltage 4 it works fine (does not freeze) after boot
  • Increase overvoltage to 5 (/DietPi/config.txt) and see if initial_turbo boot and stress test work stable then

I hope that stress test and initial_turbo both have the same result, that it indeed requires a higher voltage to run stable with 1900 MHz, otherwise we need to report to RPi devs to have some info what initial_turbo does different, e.g. CPU states are applied in a different order or something.
In case we need to raise the overvoltage for the "high" profiles, although for most boards this means an unnecessary increase of power usage and heat level 🤔.

@Vansinnet
Copy link

Vansinnet commented Oct 15, 2019

@MichaIng
Tried your suggestions:

I disabled initial_turbo (set it to 0) and the RPi booted just fine with over_voltage 4 @ 1900 mhz. After boot I stress tested the RPi for 30 minutes (with DietPi's built in stress test). At 1900 and over_voltage set to 4 the result was successfull, no freeze or crash during stress test. Just like in 6.25.3 - PROBLEM SOLVED.

However, I really like the initial_turbo option. So I raised the overvoltage to 5 like you suggested. No luck. Doesn't boot.

On my other RPi 4, there's no problem booting with over_voltage 4 @ 1900 mhz even with initial_turbo enabled

In the meantime, I think I'll just disable initial_turbo for the machine that can't handle it and run with 1900 mhz. It's running 24/7 so I figure it's not that big of a deal. Could be for others though! Maybe a warning message when one selects the 1900 mhz option in the overclocking profiles?

@Vansinnet
Copy link

Vansinnet commented Oct 15, 2019

@MichaIng
Post 2:
I decided to test the other RPi, which runs fine at 1900 mhz with initial_turbo enabled. Tried raising its frequency to 2000 mhz, which works flawlessly on 6.25.3. It won't boot like it does with initial_turbo disabled.

Conclusion: There's something going on during early boot which makes the RPi sensitive to high overclocks (initial_turbo enabled). Result is a dead client. Only way to revert is to remove the SD-card and edit the config.txt on another machine.

Is there anyone else that can verify my finding?

@MichaIng
Copy link
Owner Author

@Vansinnet
Do you have a screen attached and in case is there any output?

I just reported the issue to RPi devs: raspberrypi/firmware#1005

@popcornmix
Copy link

I have seen it reported.
Overclocking is a bit of a niche use case (especially on the faster Pi4) so this isn't high priority, but I'll try to look into it.

@MichaIng
Copy link
Owner Author

v6.26 has been released: #3162

Many thanks to all testers, I will mark this issue as closed now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Beta 🧪 Issues specific to the Beta branch testing Information ℹ️
Projects
None yet
Development

No branches or pull requests

6 participants