Upgrade fails, synapse no longer usable

[matlag]

Upgraded today, had the following error:

Warning: dh_install: Compatibility levels before 9 are deprecated (level 7 in use) Warning: dh_installdocs: Compatibility levels before 9 are deprecated (level 7 in use) Warning: dh_installdeb: Compatibility levels before 9 are deprecated (level 7 in use) Warning: E: Packages need to be removed but remove is disabled. Warning: /etc/init.d/matrix-synapse wasn't deleted because it doesn't exist. Warning: /lib/systemd/system/matrix-synapse.service wasn't deleted because it doesn't exist. Warning: /etc/systemd/system/matrix-synapse.service wasn't deleted because it doesn't exist. Warning: Created symlink from /etc/systemd/system/multi-user.target.wants/matrix-synapse.service to /etc/systemd/system/matrix-synapse.service. Warning: No passwd entry for user 'postgres' Warning: !! Warning: synapse's script has encountered an error. Its execution was cancelled. Warning: !! Warning: Error: Unable to upgrade synapse Error: No app to upgrade

Would I be correct to say that the install fails if postgre is not already installed?
Should it not be in the dependencies?

[Josue-T]

Hello,

Thank you for your report, postgresql is already as a dependence. I think the problem is more about the migration script.

Have you any more log about this upgrade because with that it's difficult to know were is the bug. Normally you could find a more verbose report in : /var/log/yunohost/yunohost-cli.log

[matlag]

Hello,

Here it is!

synapse_failed_upgrade.txt

You're right. Actually it looks like the issue is elsewhere, but I'm not sure.
2017-08-16 21:54:21,919 WARNING yunohost.hook <lambda> - [11012.1] No passwd entry for user 'postgres'

Would that interrupt the whole thing?

[Josue-T]

I see that :

2017-08-16 21:40:55,986 INFO     yunohost.hook <lambda> - [11012.1] + sudo apt-get -y -qq --no-remove -o Dpkg::Options::=--force-confdef -o Dpkg::Options::=--force-confold install -f
2017-08-16 21:40:58,857 WARNING  yunohost.hook <lambda> - [11012.1] E: Packages need to be removed but remove is disabled.

Really strange, a think that the install of dependance fail. could you try to lunch this command : sudo apt install -f

[matlag]

Ok, here the output, but it may have been convoluted with my later actions.
Since synapse was no longer available, I tried to restore it from backup. I wouldn't work because the app was already installed. So I tried to uninstall it before using the backup. Unfortunately the backup also did not work.

I don't think I had postgresql installed before, but I've tried multiple apps, and one of them may have installed it as a dep at some points. So it may have left behind some config files.

So, with all this taken in account, here's what sudo apt -f install gives me:

Reading package lists... Done Building dependency tree Reading state information... Done Correcting dependencies... Done The following packages were automatically installed and are no longer required: libgssrpc4 libkadm5clnt-mit9 libkadm5srv-mit9 libkdb5-7 postgresql postgresql-9.4 postgresql-client-9.4 postgresql-client-common postgresql-common Use 'apt-get autoremove' to remove them. The following extra packages will be installed: postgresql postgresql-9.4 postgresql-client-9.4 postgresql-client-common postgresql-common Suggested packages: postgresql-doc oidentd ident-server locales-all postgresql-doc-9.4 The following packages will be REMOVED: synapse-ynh-deps The following NEW packages will be installed: postgresql postgresql-9.4 postgresql-client-9.4 postgresql-client-common postgresql-common 0 upgraded, 5 newly installed, 1 to remove and 2 not upgraded. Need to get 5098 kB of archives. After this operation, 23.8 MB of additional disk space will be used. Do you want to continue? [Y/n]

For now, I won't proceed, in case that destroys clues or confuses further the picture.

The good news is that I was using it for rooms only, and the other users on my server had not touched it yet. So not really anything was lost for me but a bit of time.

[Josue-T]

Ok,

It look like that the synapse dependance can't be installed. Could you try this command to see witch package is a problem : sudo apt-get install -s coturn build-essential python2.7-dev libffi-dev python-pip python-setuptools sqlite3 libssl-dev python-virtualenv libjpeg-dev libpq-dev postgresql

It's also possible that postgresql wasn't cleanly removed because when you remove or purge the package there are a lot of file witch stay at sometime it could be a problem on the next install.

[matlag]

Here's the output:
Reading package lists... Done Building dependency tree Reading state information... Done libffi-dev is already the newest version. libssl-dev is already the newest version. python-setuptools is already the newest version. python-setuptools set to manually installed. build-essential is already the newest version. coturn is already the newest version. libjpeg-dev is already the newest version. libjpeg-dev set to manually installed. python-pip is already the newest version. python-pip set to manually installed. python-virtualenv is already the newest version. python2.7-dev is already the newest version. python2.7-dev set to manually installed. sqlite3 is already the newest version. You might want to run 'apt-get -f install' to correct these: The following packages have unmet dependencies: libpq-dev : Depends: krb5-multidev but it is not going to be installed Depends: comerr-dev but it is not going to be installed postgresql : Depends: postgresql-9.4 but it is not going to be installed E: Unmet dependencies. Try 'apt-get -f install' with no packages (or specify a solution).

[Josue-T]

I see that :

You might want to run 'apt-get -f install' to correct these:
The following packages have unmet dependencies:
 libpq-dev : Depends: krb5-multidev but it is not going to be installed
             Depends: comerr-dev but it is not going to be installed
 postgresql : Depends: postgresql-9.4 but it is not going to be installed
E: Unmet dependencies. Try 'apt-get -f install' with no packages (or specify a solution).

It look like that some package have a problem about dependance, and not only postgresql.
Witch architecture do you have. Could you share me your /etc/apt/source.list file to see the repository.
I pupose to you to lunch apt-get install ⁻f and accept the purpose. After retry : sudo apt-get install -s coturn build-essential python2.7-dev libffi-dev python-pip python-setuptools sqlite3 libssl-dev python-virtualenv libjpeg-dev libpq-dev postgresql and share me the result.

You could also try after apt-get install postgresql-9.4 krb5-multidev comerr-dev to know why this package can't be installed.

[matlag]

Ok! I found the problem!
Indeed, there was an issue with comerr-dev: it would not install because of my libcomerr2 installed from... the backports!
I don't know why I had that package from the backports, but yet again: I installed and removed quite some apps, so for now I won't be able to trace it back.

Once that was done, I also had to rename the directory in /opt/yunohost.
As said previously: I tried to remove synapse and restore from the backup and it didn't work. I found quite some "residus" here and there. But the good news is after that, I could install again Synapse.
I've lost the settings and rooms subscription, but it's no big deal to re-subscribe.

That case is closed for me (I re-install Synapse and it worked). But for others, it may be worth having a less damaging upgrade when some issues are encountered during packages installation?
On the other hand, I don't know how many people using Yunohost have Synapse and didn't do the upgrade and will have the same issue, and based on what I've seen, you've moved away from packages anyway. So might not be worth the effort. I let you judge!

I can still dig through log files if there is anything else you need.

Thanks for your help!

[Josue-T]

Hello,

Thank you really much for your answer. I know that this upgrade could break the install, so I'm happy to know the issue for you. I will search a way to do the upgrade more secure. If your have any idea to increase the upgrade I'm open.
Just a question : Did you said that the restore from the backup didn't run for you ?

[matlag]

We're getting off topic, but I traced down what went wrong with the backup restore. After reviewing the log, here's the sequence:

-Attempt to upgrade: FAIL (that was this very ticket)
App is unusable
-Attempt to restore from backup: FAIL
That's because I had not removed the app. It was unusable but it was still there
-Remove app: SUCCESS
-Attempt to restore from backup: FAIL
Error message is:
ynh_die 'There is already a nginx conf file at this path: /etc/nginx/conf.d/chtisurel.net.d/synapse.conf'

But in the app removal process, I have:
REMOVE_NGINX_CONF '[' -e /etc/nginx/conf.d/.d/synapse.conf ']'
and no indication that had failed

I should have been more curious at that time, but I wasn't. So I can't tell which happened:

1. The file was not properly removed when I removed the app
2. The file was prematurely created at re-install and then tested for presence
3. The file was wrongly detected at re-install but was not there

I would bet on the 1st, though. I assume the restoration and installation share as much code as possible, right?

[Josue-T]

I think that it look like that the yunohost app settings has been removed but not the file. I think that the yunohost cli have removed the settings (on the restore script when it fail) but the config file has not been removed. I these thing like that could append when a lot of things fail. But this problem could be with every apps when a lot of command fail...

Since the last yunohost version less code is shared between the restoration and install script because with the last version a lot of file are automatically reinstalled by the special yunohost helper. So the restore script is really lighter than the install script.

I will see (when I have time) to improve the upgrade script (when we do the migration) with more security.