New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update to NCP 0.62.2 fails, stuck removing PHP7.0-fpm #669

Closed
XDA-Bam opened this Issue Oct 3, 2018 · 19 comments

Comments

Projects
None yet
8 participants
@XDA-Bam

XDA-Bam commented Oct 3, 2018

The update from NCP 0.60.8 to 0.62.2 via the NCP web interface fails. It runs up to the point where php7.0 should be uninstalled, then nothing happens. The web interface shows

Reading state information...
Package 'php7.0-bcmath' is not installed, so not removed
Package 'php7.0-bz2' is not installed, so not removed
Package 'php7.0-cgi' is not installed, so not removed
Package 'php7.0-dba' is not installed, so not removed
Package 'php7.0-dev' is not installed, so not removed
Package 'php7.0-enchant' is not installed, so not removed
Package 'php7.0-gmp' is not installed, so not removed
Package 'php7.0-imap' is not installed, so not removed
Package 'php7.0-interbase' is not installed, so not removed
Package 'php7.0-odbc' is not installed, so not removed
Package 'php7.0-pgsql' is not installed, so not removed
Package 'php7.0-phpdbg' is not installed, so not removed
Package 'php7.0-pspell' is not installed, so not removed
Package 'php7.0-recode' is not installed, so not removed
Package 'php7.0-snmp' is not installed, so not removed
Package 'php7.0-soap' is not installed, so not removed
Package 'php7.0-sqlite3' is not installed, so not removed
Package 'php7.0-sybase' is not installed, so not removed
Package 'php7.0-tidy' is not installed, so not removed
Package 'php7.0-xmlrpc' is not installed, so not removed
Package 'php7.0-xsl' is not installed, so not removed
The following packages were automatically installed and are no longer required:
libmagic-mgc libmagic1 libmcrypt4 libzip4 php-common
Use 'sudo apt autoremove' to remove them.
The following packages will be REMOVED:
php7.0-cli* php7.0-common* php7.0-curl* php7.0-fpm* php7.0-gd* php7.0-intl*
php7.0-json* php7.0-ldap* php7.0-mbstring* php7.0-mcrypt* php7.0-mysql*
php7.0-opcache* php7.0-readline* php7.0-xml* php7.0-zip*
0 upgraded, 0 newly installed, 15 to remove and 0 not upgraded.
After this operation, 16.1 MB disk space will be freed.
(Reading database ...
(Reading database ... 5%
(Reading database ... 75%
(Reading database ... 85%
(Reading database ... 100%
Removing php7.0-fpm (7.0.30-0+deb9u1) ...

and hangs. The system is idle at this point, nothing seems to be going on. Also, the web interface says

Something went wrong. Try refreshing the page

A refresh results in

503 Service Unavailable

The commands

dpkg --configure -a
apt-get upgrade
service php7.0-fpm start

fix the php problems and the NCP web-interface comes back to life. It still shows version 0.60.8 and prompts me to upgrade (which fails in the same way if attempted). The nextcloud server is down and responds with

Internal Server Error

The server encountered an internal error and was unable to complete your request.

Disabling the NC maintenance mode has no effect. Neither does rebooting.

@Drialus

This comment has been minimized.

Drialus commented Oct 3, 2018

Hello,

i had the same issue which resulted in this after a new installation. Have you experienced the same?

@XDA-Bam

This comment has been minimized.

XDA-Bam commented Oct 3, 2018

@Drialus if you are referencing the btrfs-issues: No. I'm running ext4 and the system itself seems fine. Just NC is down.

@derdershat

This comment has been minimized.

derdershat commented Oct 3, 2018

I am in too but on 0.62.2

[ nc-update ]
Downloading updates
Performing updates
Installing nc-backup
Installing nc-restore
previewgenerator already installed
previewgenerator enabled
Maintenance mode enabled
Get:1 http://raspbian.raspberrypi.org/raspbian stretch InRelease [15.0 kB]
Hit:2 http://archive.raspberrypi.org/debian stretch InRelease
Fetched 15.0 kB in 1s (14.4 kB/s)
Reading package lists...

Reading package lists...

Building dependency tree...

Reading state information...

apt-transport-https is already the newest version (1.4.8).
The following package was automatically installed and is no longer required:
libluajit-5.1-2
Use 'sudo apt autoremove' to remove it.
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Hit:1 http://raspbian.raspberrypi.org/raspbian stretch InRelease
Hit:2 http://archive.raspberrypi.org/debian stretch InRelease
Get:3 https://cdn-aws.deb.debian.org/debian buster InRelease [150 kB]
Ign:3 https://cdn-aws.deb.debian.org/debian buster InRelease
Get:4 https://cdn-aws.deb.debian.org/debian buster/main armhf Packages [7396 kB]
Get:5 https://cdn-aws.deb.debian.org/debian buster/main Translation-en [5725 kB]
Get:6 https://cdn-aws.deb.debian.org/debian buster/contrib armhf Packages [42.1 kB]
Get:7 https://cdn-aws.deb.debian.org/debian buster/contrib Translation-en [43.6 kB]
Get:8 https://cdn-aws.deb.debian.org/debian buster/non-free armhf Packages [67.1 kB]
Get:9 https://cdn-aws.deb.debian.org/debian buster/non-free Translation-en [83.9 kB]
Fetched 13.5 MB in 22s (593 kB/s)
Reading package lists...

W: GPG error: https://cdn-aws.deb.debian.org/debian buster InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 8B48AD6246925553 NO_PUBKEY 7638D0442B90D010 NO_PUBKEY 04EE7237B7D453EC
W: The repository 'https://deb.debian.org/debian buster InRelease' is not signed.
Reading package lists...
Building dependency tree...

Reading state information...
The following package was automatically installed and is no longer required:
libluajit-5.1-2
Use 'sudo apt autoremove' to remove it.
The following NEW packages will be installed:
debian-archive-keyring
0 upgraded, 1 newly installed, 0 to remove and 972 not upgraded.
Need to get 56.9 kB of archives.
After this operation, 117 kB of additional disk space will be used.
WARNING: The following packages cannot be authenticated!
debian-archive-keyring
Authentication warning overridden.
Get:1 https://cdn-aws.deb.debian.org/debian buster/main armhf debian-archive-keyring all 2017.7 [56.9 kB]
Fetched 56.9 kB in 1s (32.6 kB/s)
Selecting previously unselected package debian-archive-keyring.
(Reading database ...
(Reading database ... 5%
(Reading database ... 65%
(Reading database ... 75%
(Reading database ... 80%
(Reading database ... 90%
(Reading database ... 95%
(Reading database ... 100%
Preparing to unpack .../debian-archive-keyring_2017.7_all.deb ...
Unpacking debian-archive-keyring (2017.7) ...
Setting up debian-archive-keyring (2017.7) ...
Unable to get Terminal Size. The TIOCGWINSZ ioctl didn't work. The COLUMNS and LINES environment variables didn't work. The resize program didn't work. The stty program didn't work. at /usr/share/perl5/NeedRestart/UI.pm line 50.
Running kernel seems to be up-to-date.
Unable to get Terminal Size. The TIOCGWINSZ ioctl didn't work. The COLUMNS and LINES environment variables didn't work. The resize program didn't work. The stty program didn't work. at /usr/share/perl5/NeedRestart/UI.pm line 50.
Services to be restarted:
systemctl restart fail2ban.service
Unable to get Terminal Size. The TIOCGWINSZ ioctl didn't work. The COLUMNS and LINES environment variables didn't work. The resize program didn't work. The stty program didn't work. at /usr/share/perl5/NeedRestart/UI.pm line 50.
Services being skipped:
systemctl restart rc-local.service
Unable to get Terminal Size. The TIOCGWINSZ ioctl didn't work. The COLUMNS and LINES environment variables didn't work. The resize program didn't work. The stty program didn't work. at /usr/share/perl5/NeedRestart/UI.pm line 50.
No containers need to be restarted.
Unable to get Terminal Size. The TIOCGWINSZ ioctl didn't work. The COLUMNS and LINES environment variables didn't work. The resize program didn't work. The stty program didn't work. at /usr/share/perl5/NeedRestart/UI.pm line 50.
No user sessions are running outdated binaries.
Hit:1 http://raspbian.raspberrypi.org/raspbian stretch InRelease
Get:2 http://deb.debian.org/debian buster InRelease [150 kB]
Hit:3 http://archive.raspberrypi.org/debian stretch InRelease
Fetched 150 kB in 1s (87.7 kB/s)
Reading package lists...

Reading package lists...

Building dependency tree...

Reading state information...
Package 'php7.0-bcmath' is not installed, so not removed
Package 'php7.0-bz2' is not installed, so not removed
Package 'php7.0-cgi' is not installed, so not removed
Package 'php7.0-dba' is not installed, so not removed
Package 'php7.0-dev' is not installed, so not removed
Package 'php7.0-enchant' is not installed, so not removed
Package 'php7.0-gmp' is not installed, so not removed
Package 'php7.0-imap' is not installed, so not removed
Package 'php7.0-interbase' is not installed, so not removed
Package 'php7.0-intl' is not installed, so not removed
Package 'php7.0-ldap' is not installed, so not removed
Package 'php7.0-odbc' is not installed, so not removed
Package 'php7.0-pgsql' is not installed, so not removed
Package 'php7.0-phpdbg' is not installed, so not removed
Package 'php7.0-pspell' is not installed, so not removed
Package 'php7.0-recode' is not installed, so not removed
Package 'php7.0-snmp' is not installed, so not removed
Package 'php7.0-soap' is not installed, so not removed
Package 'php7.0-sqlite3' is not installed, so not removed
Package 'php7.0-sybase' is not installed, so not removed
Package 'php7.0-tidy' is not installed, so not removed
Package 'php7.0-xmlrpc' is not installed, so not removed
Package 'php7.0-xsl' is not installed, so not removed
The following packages were automatically installed and are no longer required:
liblqr-1-0 libluajit-5.1-2 libmagickcore-6.q16-3 libmagickwand-6.q16-3
libmcrypt4 libzip4 php-common
Use 'sudo apt autoremove' to remove them.
The following packages will be REMOVED:
php-igbinary* php-imagick* php-redis* php-smbclient* php7.0* php7.0-cli*
php7.0-common* php7.0-curl* php7.0-fpm* php7.0-gd* php7.0-json*
php7.0-mbstring* php7.0-mcrypt* php7.0-mysql* php7.0-opcache*
php7.0-readline* php7.0-xml* php7.0-zip*
0 upgraded, 0 newly installed, 18 to remove and 0 not upgraded.
After this operation, 14.2 MB disk space will be freed.
(Reading database ...
(Reading database ... 75%
Removing php-redis (3.1.1-1) ...
Removing php-igbinary (2.0.1-1) ...
Removing php-imagick (3.4.3~rc2-2) ...
Removing php-smbclient (0.8.0-3) ...
Removing php7.0 (7.0.30-0+deb9u1) ...
Removing php7.0-fpm (7.0.30-0+deb9u1) ...`

ncp.log

@TomTurnschuh

This comment has been minimized.

Contributor

TomTurnschuh commented Oct 3, 2018

Hi there

After the update, please run the following code from terminal (e.g. via SSH) as superuser:

ncc maintenance:mode --off

bash -c "sleep 5 && service apache2 restart" &>/dev/null &
bash -c " sleep 3
      service php7.2-fpm stop
      service mysql      stop
      sleep 0.5
      service php7.2-fpm start
      service mysql      start
      " &>/dev/null &

Start the execution with Enter on your keyboard, wait for at least 8 seconds, and press Enter again.

@TomTurnschuh

This comment has been minimized.

Contributor

TomTurnschuh commented Oct 3, 2018

@nachoparker: It seems that the above mentioned last part of the update code for PHP 7.2 is not being executed. Maybe it should be executed before the autoremove command?
Running this code after the update solved the problem on my two currently running installations.

@XDA-Bam

This comment has been minimized.

XDA-Bam commented Oct 3, 2018

Hi @TomTurnschuh, thanks for trying to help! In my case, executing those commands would fail, because PHP 7.2 has not been installed during the update. PHP 7.0 is still running.

@TomTurnschuh

This comment has been minimized.

Contributor

TomTurnschuh commented Oct 3, 2018

@nachoparker edit: See this solution, it is the same thing but easier to type -> #669 (comment)

many thanks @TomTurnschuh


@XDA-Bam, how about running this whole code?:

sudo su
PHPVER=7.2
APTINSTALL="apt-get install -y --no-install-recommends"
export DEBIAN_FRONTEND=noninteractive

    $APTINSTALL -t buster php${PHPVER} php${PHPVER}-curl php${PHPVER}-gd php${PHPVER}-fpm php${PHPVER}-cli php${PHPVER}-opcache \
                          php${PHPVER}-mbstring php${PHPVER}-xml php${PHPVER}-zip php${PHPVER}-fileinfo php${PHPVER}-ldap \
                          php${PHPVER}-intl php${PHPVER}-bz2 php${PHPVER}-json
 
      $APTINSTALL php${PHPVER}-mysql
      $APTINSTALL -t buster php${PHPVER}-redis
      $APTINSTALL -t buster php-smbclient                                         # for external storage
      $APTINSTALL -t buster imagemagick php${PHPVER}-imagick php${PHPVER}-exif    # for gallery

      cat > /etc/php/${PHPVER}/mods-available/opcache.ini <<EOF
zend_extension=opcache.so
opcache.enable=1
opcache.enable_cli=1
opcache.fast_shutdown=1
opcache.interned_strings_buffer=8
opcache.max_accelerated_files=10000
opcache.memory_consumption=128
opcache.save_comments=1
opcache.revalidate_freq=1
opcache.file_cache=/tmp;
EOF
      a2enconf  php${PHPVER}-fpm


      DATADIR="$( grep datadirectory /var/www/nextcloud/config/config.php | awk '{ print $3 }' | grep -oP "[^']*[^']" | head -1 )"
      UPLOADTMPDIR="$DATADIR"/tmp
      sed -i "s|^;\?upload_tmp_dir =.*$|upload_tmp_dir = $UPLOADTMPDIR|" /etc/php/${PHPVER}/cli/php.ini
      sed -i "s|^;\?upload_tmp_dir =.*$|upload_tmp_dir = $UPLOADTMPDIR|" /etc/php/${PHPVER}/fpm/php.ini
      sed -i "s|^;\?sys_temp_dir =.*$|sys_temp_dir = $UPLOADTMPDIR|"     /etc/php/${PHPVER}/fpm/php.ini

      OPCACHEDIR="$DATADIR"/.opcache
      sed -i "s|^opcache.file_cache=.*|opcache.file_cache=$OPCACHEDIR|" /etc/php/${PHPVER}/mods-available/opcache.ini

      apt-get autoremove -y

      ncc maintenance:mode --off

      bash -c "sleep 5 && service apache2 restart" &>/dev/null &
      bash -c " sleep 3
              service php${PHPVER}-fpm stop
              service mysql      stop
              sleep 0.5
              service php${PHPVER}-fpm start
              service mysql      start
              " &>/dev/null &

This should be the part after the last executed command in your log.

@XDA-Bam

This comment has been minimized.

XDA-Bam commented Oct 3, 2018

@TomTurnschuh I executed the commands and PHP 7.2 installed correctly. After this, NC was still down. I then remembered that I switched from
'memcache.local' => '\\OC\\Memcache\\Redis',
to
'memcache.local' => '\\OC\\Memcache\\APCu',
some time ago and the APCu package might be missing. I went back to Redis, restarted PHP and NC is now back up again. So thanks for the help!

However, neither NC nor NCP did update. I am still running NCP 0.60.8 and NC 13.0.6.1 according to the web interface. So there's still work to do. Maybe I'm gonna try the NCP update script again, but right now, it's time to sleep :)

@nachoparker

This comment has been minimized.

Member

nachoparker commented Oct 4, 2018

Hi,

The update got stuck for no reason while removing PHP7.0? that's really weird. Probably it was a matter of waiting more.

Also running the upgrade again would have worked.

Thanks @TomTurnschuh

@nachoparker

This comment has been minimized.

Member

nachoparker commented Oct 4, 2018

Anybody else seeing this? there is no reason why it would get stuck here

apt-get purge -y php7.0-*

It's just an apt-get purge command

@hurrikan77

This comment has been minimized.

hurrikan77 commented Oct 4, 2018

Hi

I had the same problem after starting the update via web interface.

After using

dpkg --configure -a

and starting the update from shell the updates works.
But although there are error messages depending the autoremove.

After updating the ncp-wb and rebooting the update of the Nextcloud works without any problem.

@derdershat

This comment has been minimized.

derdershat commented Oct 4, 2018

I tried again and restored a sd-card backup

first via ncp web update but stuck on same line "Removing php7.0-fpm (7.0.30-0+deb9u1) ..."
"Something went wrong .. please update page" but I didn`t updated as quick as last time.
waited about 20min. but nothing changed

than via shell
dpkg --configure -a
sudo ncp-update

than via ncp web
nc-update-nextcloud Version 14.0.1

and it worked

@XDA-Bam

This comment has been minimized.

XDA-Bam commented Oct 4, 2018

Hi,

The update got stuck for no reason while removing PHP7.0? that's really weird. Probably it was a matter of waiting more.

Also running the upgrade again would have worked.

Thanks @TomTurnschuh

Yes, seems like it got stuck because the PHP removal process threw some kind of error. Waiting longer or running again would not have worked, though. I waited for 10 minutes on the first update run, then checked htop and observed it for a couple of minutes - the system was idle. I then cleaned up everything with

dpkg --configure -a
apt-get upgrade
service php7.0-fpm start

, restarted PHP 7.0 and tried the update again via the web interface and it got stuck again. Same problem on a third try. It just failed cleaning up PHP 7.0.

Only the manual solution from @TomTurnschuh solved it. After installing and configuring PHP 7.2, I just updated to NCP 0.62.5 via the web interface, which worked totally fine. NC is still on 13.06.1, though, so the script didn't update NC automatically.

EDIT:
Could all this be related to the fact that I made some changes to my PHP 7.0 install? These included

  • Install of php7.0-apcu package
  • Config change in var/www/nextcloud/config/config.php to set memory caching to use APCu instead of Redis
  • A couple of backup config files in the PHP directory (which prevents apt-get purge -y php7.0-* from deleting the /7.0 directory)
@devilsharp

This comment has been minimized.

devilsharp commented Oct 4, 2018

EDIT:
Could all this be related to the fact that I made some changes to my PHP 7.0 install? These included

* Install of php7.0-apcu package

* Config change in var/www/nextcloud/config/config.php to set memory caching to use APCu instead of Redis

* A couple of backup config files in the PHP directory (which prevents apt-get purge -y php7.0-* from deleting the /7.0 directory)

I don't think so, I don't made any change on the PHP configuration set by Nextcloud instance. And I've got the issue

@devilsharp

This comment has been minimized.

devilsharp commented Oct 4, 2018

Moreover, I am not able to use the @TomTurnschuh solution because I've got also the issue #667 .
When I try to execute the commands provided by @TomTurnschuh I get the libc6 and libc6-dev in error which is blocking totally the update process.

At each step I'm trying to back to the working version of libc6 and libc6-dev by these commands:
sudo apt --fix-broken install
sudo apt-get install libc6 libc6-dev --reinstall

Which is temporarily fixed the #667 .
But the issue coming back at each try to install PHP 7.2 or module corresponding.
I'm totally blocked

@TomTurnschuh

This comment has been minimized.

Contributor

TomTurnschuh commented Oct 4, 2018

Just for information:

After the (more or less) successful update, the service php7.2-fpm may have to be started manually after every reboot, cf. #672.

On another installation I also have to start the service mysql.

@nachoparker

This comment has been minimized.

Member

nachoparker commented Oct 5, 2018

Still surprised that it gets stuck there... inexplicable stuff like this always smells to me like a HW issue (see FAQ).

Anyhow, glad that there's a solution

@nachoparker nachoparker closed this Oct 5, 2018

@nachoparker nachoparker changed the title from Update to NCP 0.62.2 fails to Update to NCP 0.62.2 fails, stuck removing PHP7.0-fpm Oct 5, 2018

@nachoparker nachoparker self-assigned this Oct 5, 2018

@nachoparker

This comment has been minimized.

Member

nachoparker commented Oct 6, 2018

Anyone else seeing this try this fix

sudo su
apt-get purge -y php7.0* php7.2*
rm -rf /etc/php/7.2
ncp-update

This will retry the upgrade to PHP7.2

@Henriktor

This comment has been minimized.

Henriktor commented Oct 6, 2018

Anyone else seeing this try this fix

sudo su
apt-get purge -y php7.0* php7.2*
rm -rf /etc/php/7.2
ncp-update

This will retry the upgrade to PHP7.2

This is the soulution! It worked like a charm.

Thx for this!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment