Database upgrade failed after upgrading to new package

[kikuhama]

Steps to reproduce

1. I had installed the owncloud-files package to a Ubuntu 16.04 server. (Using PostgreSQL database)
2. Last week, I prepared a new Ubuntu 18.04 server to migrate the server.
3. I installed the owncloud-files package to the new server, copied /vaw/www/owncloud/data directory to new server and dumped and restored the PostgreSQL database.
4. The new owncloud server works fine after the migration.
5. Today, I realized that a new owncloud-files package exists in the repository.
6. sudo apt upgrade
7. Installing the package finished correctly.
8. I accessed to my ownCloud site and I started upgrading the database.
9. The database upgrading failed and these message was shown.

アップデートの準備中
ログをデバッグレベルに設定
メンテナンスモードがオンになりました
データベースのスキーマを更新
データベース更新済み
Doctrine\DBAL\Exception\InvalidFieldNameException: An exception occurred while executing 'SELECT min_value, increment_by FROM "oc_account_terms_id_seq"': SQLSTATE[42703]: Undefined column: 7 ERROR: column "min_value" does not exist LINE 1: SELECT min_value, increment_by FROM "oc_account_terms_id_seq... ^

Actual behaviour

The database upgrade was failed and these message was shown.

アップデートの準備中
ログをデバッグレベルに設定
メンテナンスモードがオンになりました
データベースのスキーマを更新
データベース更新済み
Doctrine\DBAL\Exception\InvalidFieldNameException: An exception occurred while executing 'SELECT min_value, increment_by FROM "oc_account_terms_id_seq"': SQLSTATE[42703]: Undefined column: 7 ERROR: column "min_value" does not exist LINE 1: SELECT min_value, increment_by FROM "oc_account_terms_id_seq... ^

Server configuration

Operating system:
Ubuntu 18.04

Web server:
Apache 2.4.29

Database:
PostgreSQL 10.4

PHP version:
PHP 7.2.7

ownCloud version: (see ownCloud admin page)
10.0.9

Updated from an older ownCloud or fresh install:
Updated from previous version.

Where did you install ownCloud from:
http://download.owncloud.org/download/repositories/production/Ubuntu_18.04/
(using apt)

Signing status (ownCloud 9.0 and above):
I cannot login now.

The content of config/config.php:

ownCloud or one of the apps require upgrade - only a limited number of commands are available
You may use your browser or the occ upgrade command to do the upgrade
{
    "system": {
        "updatechecker": false,
        "instanceid": "ocmymeilgaeq",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "www.hamajima.co.jp"
        ],
        "datadirectory": "\/var\/www\/owncloud\/data",
        "overwrite.cli.url": "https:\/\/www.hamajima.co.jp\/owncloud",
        "dbtype": "pgsql",
        "version": "10.0.8.5",
        "dbname": "owncloud",
        "dbhost": "localhost",
        "dbtableprefix": "oc_",
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "logtimezone": "UTC",
        "installed": true,
        "maintenance": false,
        "theme": "",
        "loglevel": 2,
        "memcache.local": "\\OC\\Memcache\\Redis",
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "redis": {
            "host": "localhost",
            "port": 6379,
            "timeout": 0,
            "password": "***REMOVED SENSITIVE VALUE***"
        },
        "mail_smtpmode": "smtp",
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***"
    }
}

**List of activated apps:**
ownCloud or one of the apps require upgrade - only a limited number of commands are available
You may use your browser or the occ upgrade command to do the upgrade
Enabled:
  - activity: 2.3.7
  - comments: 0.3.0
  - configreport: 0.1.1
  - dav: 0.3.2
  - federatedfilesharing: 0.3.1
  - federation: 0.1.0
  - files: 1.5.1
  - files_external: 0.7.1
  - files_pdfviewer: 0.9.0
  - files_sharing: 0.10.1
  - files_texteditor: 2.2.1
  - files_trashbin: 0.9.1
  - files_versions: 1.3.0
  - files_videoplayer: 0.9.8
  - firstrunwizard: 1.1
  - gallery: 16.1.0
  - market: 0.2.5
  - notifications: 0.3.3
  - provisioning_api: 0.5.0
  - systemtags: 0.3.0
  - templateeditor: 0.3.1
  - updatenotification: 0.2.1
Disabled:
  - encryption
  - external
  - user_external

Are you using external storage, if yes which one: local/smb/sftp/...
No

Are you using encryption: yes/no
No

Are you using an external user-backend, if yes which one: LDAP/ActiveDirectory/Webdav/...
No

Client configuration

Browser:
Chrome 68

Operating system:
Windows 10 Pro

Logs

Web server error log

No error log exists.

ownCloud log (data/owncloud.log)

{"reqId":"OBR24fNzOWrEmJHNH30g","level":3,"time":"2018-07-29T23:59:28+00:00","remoteAddr":"110.3.31.143","user":"--","app":"PHP","method":"GET","url":"\/owncloud\/","message":"ini_set(): A session is active. You cannot change the session module's ini settings at this time at \/var\/www\/owncloud\/lib\/base.php#603"}
{"reqId":"brZuyntBwYgIW6DLIewB","level":3,"time":"2018-07-29T23:59:29+00:00","remoteAddr":"110.3.31.143","user":"--","app":"PHP","method":"GET","url":"\/owncloud\/index.php\/core\/js\/oc.js?v=abab09a545f502b9e6b0730faaf3e978","message":"ini_set(): A session is active. You cannot change the session module's ini settings at this time at \/var\/www\/owncloud\/lib\/base.php#603"}
{"reqId":"IsghTKfbNMA2GhZAbFn8","level":3,"time":"2018-07-29T23:59:31+00:00","remoteAddr":"110.3.31.143","user":"--","app":"PHP","method":"GET","url":"\/owncloud\/core\/ajax\/update.php?requesttoken=fRF6NQw9H01HLFQsBQByXBYUJnJjYgEbdQM0bQ1aHBk%3D%3AKbLlmntz2U6YKT6neQMB00dcD3m9uiLr6SQrFbmZODE%3D","message":"ini_set(): A session is active. You cannot change the session module's ini settings at this time at \/var\/www\/owncloud\/lib\/base.php#603"}
{"reqId":"IsghTKfbNMA2GhZAbFn8","level":0,"time":"2018-07-29T23:59:32+00:00","remoteAddr":"110.3.31.143","user":"--","app":"core","method":"GET","url":"\/owncloud\/core\/ajax\/update.php?requesttoken=fRF6NQw9H01HLFQsBQByXBYUJnJjYgEbdQM0bQ1aHBk%3D%3AKbLlmntz2U6YKT6neQMB00dcD3m9uiLr6SQrFbmZODE%3D","message":"starting upgrade from 10.0.8.5 to 10.0.9.5"}
{"reqId":"IsghTKfbNMA2GhZAbFn8","level":3,"time":"2018-07-29T23:59:57+00:00","remoteAddr":"110.3.31.143","user":"--","app":"core","method":"GET","url":"\/owncloud\/core\/ajax\/update.php?requesttoken=fRF6NQw9H01HLFQsBQByXBYUJnJjYgEbdQM0bQ1aHBk%3D%3AKbLlmntz2U6YKT6neQMB00dcD3m9uiLr6SQrFbmZODE%3D","message":"Exception: {\"Exception\":\"Doctrine\\\\DBAL\\\\Exception\\\\InvalidFieldNameException\",\"Message\":\"An exception occurred while executing 'SELECT min_value, increment_by FROM \\\"oc_account_terms_id_seq\\\"':\\n\\nSQLSTATE[42703]: Undefined column: 7 ERROR:  column \\\"min_value\\\" does not exist\\nLINE 1: SELECT min_value, increment_by FROM \\\"oc_account_terms_id_seq...\\n               ^\",\"Code\":0,\"Trace\":\"#0 \\\/var\\\/www\\\/owncloud\\\/lib\\\/composer\\\/doctrine\\\/dbal\\\/lib\\\/Doctrine\\\/DBAL\\\/DBALException.php(128): Doctrine\\\\DBAL\\\\Driver\\\\AbstractPostgreSQLDriver->convertException('An exception oc...', Object(Doctrine\\\\DBAL\\\\Driver\\\\PDOException))\\n#1 \\\/var\\\/www\\\/owncloud\\\/lib\\\/composer\\\/doctrine\\\/dbal\\\/lib\\\/Doctrine\\\/DBAL\\\/Connection.php(855): Doctrine\\\\DBAL\\\\DBALException::driverExceptionDuringQuery(Object(Doctrine\\\\DBAL\\\\Driver\\\\PDOPgSql\\\\Driver), Object(Doctrine\\\\DBAL\\\\Driver\\\\PDOException), 'SELECT min_valu...', Array)\\n#2 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/DB\\\/Connection.php(189): Doctrine\\\\DBAL\\\\Connection->executeQuery('SELECT min_valu...', Array, Array, NULL)\\n#3 \\\/var\\\/www\\\/owncloud\\\/lib\\\/composer\\\/doctrine\\\/dbal\\\/lib\\\/Doctrine\\\/DBAL\\\/Connection.php(785): OC\\\\DB\\\\Connection->executeQuery('SELECT min_valu...', Array, Array)\\n#4 \\\/var\\\/www\\\/owncloud\\\/lib\\\/composer\\\/doctrine\\\/dbal\\\/lib\\\/Doctrine\\\/DBAL\\\/Schema\\\/PostgreSqlSchemaManager.php(292): Doctrine\\\\DBAL\\\\Connection->fetchAll('SELECT min_valu...')\\n#5 \\\/var\\\/www\\\/owncloud\\\/lib\\\/composer\\\/doctrine\\\/dbal\\\/lib\\\/Doctrine\\\/DBAL\\\/Schema\\\/PostgreSqlSchemaManager.php(267): Doctrine\\\\DBAL\\\\Schema\\\\PostgreSqlSchemaManager->_getPortableSequenceDefinition(Array)\\n#6 \\\/var\\\/www\\\/owncloud\\\/lib\\\/composer\\\/doctrine\\\/dbal\\\/lib\\\/Doctrine\\\/DBAL\\\/Schema\\\/AbstractSchemaManager.php(147): Doctrine\\\\DBAL\\\\Schema\\\\PostgreSqlSchemaManager->_getPortableSequencesList(Array)\\n#7 \\\/var\\\/www\\\/owncloud\\\/lib\\\/composer\\\/doctrine\\\/dbal\\\/lib\\\/Doctrine\\\/DBAL\\\/Schema\\\/AbstractSchemaManager.php(1037): Doctrine\\\\DBAL\\\\Schema\\\\AbstractSchemaManager->listSequences()\\n#8 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/DB\\\/Migrator.php(138): Doctrine\\\\DBAL\\\\Schema\\\\AbstractSchemaManager->createSchema()\\n#9 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/DB\\\/Connection.php(427): OC\\\\DB\\\\Migrator->createSchema()\\n#10 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/DB\\\/MigrationService.php(402): OC\\\\DB\\\\Connection->createSchema()\\n#11 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/DB\\\/MigrationService.php(358): OC\\\\DB\\\\MigrationService->executeStep(20180604132522)\\n#12 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/legacy\\\/app.php(1030): OC\\\\DB\\\\MigrationService->migrate()\\n#13 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/Updater.php(307): OC_App::updateApp('notifications')\\n#14 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/Updater.php(233): OC\\\\Updater->doAppUpgrade()\\n#15 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/Updater.php(108): OC\\\\Updater->doUpgrade('10.0.9.5', '10.0.8.5')\\n#16 \\\/var\\\/www\\\/owncloud\\\/core\\\/ajax\\\/update.php(190): OC\\\\Updater->upgrade()\\n#17 {main}\",\"File\":\"\\\/var\\\/www\\\/owncloud\\\/lib\\\/composer\\\/doctrine\\\/dbal\\\/lib\\\/Doctrine\\\/DBAL\\\/Driver\\\/AbstractPostgreSQLDriver.php\",\"Line\":72}"}
{"reqId":"0LI80ufzjo7hmPB7z9gW","level":3,"time":"2018-07-30T00:00:45+00:00","remoteAddr":"110.3.31.143","user":"--","app":"PHP","method":"GET","url":"\/owncloud\/","message":"ini_set(): A session is active. You cannot change the session module's ini settings at this time at \/var\/www\/owncloud\/lib\/base.php#603"}
{"reqId":"zKyCe9KMAWX2FQlOs7A9","level":3,"time":"2018-07-30T00:00:45+00:00","remoteAddr":"110.3.31.143","user":"--","app":"PHP","method":"GET","url":"\/owncloud\/index.php\/core\/js\/oc.js?v=3a0c8db619c7824d700db01941f7ec3f","message":"ini_set(): A session is active. You cannot change the session module's ini settings at this time at \/var\/www\/owncloud\/lib\/base.php#603"}
{"reqId":"VVRzmKGvQ8tCihTRFnz7","level":3,"time":"2018-07-30T00:00:49+00:00","remoteAddr":"110.3.31.143","user":"--","app":"PHP","method":"GET","url":"\/owncloud\/core\/ajax\/update.php?requesttoken=VSUFLwkXWG8QEypMfDAIexcLCQk4JUpTdV4hPUp9CDE%3D%3AcV3vhD3XejH92dLIdNb9kw%2F%2BDnxi2NXZEGJOwDij%2Bsk%3D","message":"ini_set(): A session is active. You cannot change the session module's ini settings at this time at \/var\/www\/owncloud\/lib\/base.php#603"}
{"reqId":"VVRzmKGvQ8tCihTRFnz7","level":0,"time":"2018-07-30T00:00:49+00:00","remoteAddr":"110.3.31.143","user":"--","app":"core","method":"GET","url":"\/owncloud\/core\/ajax\/update.php?requesttoken=VSUFLwkXWG8QEypMfDAIexcLCQk4JUpTdV4hPUp9CDE%3D%3AcV3vhD3XejH92dLIdNb9kw%2F%2BDnxi2NXZEGJOwDij%2Bsk%3D","message":"starting upgrade from 10.0.8.5 to 10.0.9.5"}
{"reqId":"VVRzmKGvQ8tCihTRFnz7","level":3,"time":"2018-07-30T00:00:54+00:00","remoteAddr":"110.3.31.143","user":"--","app":"core","method":"GET","url":"\/owncloud\/core\/ajax\/update.php?requesttoken=VSUFLwkXWG8QEypMfDAIexcLCQk4JUpTdV4hPUp9CDE%3D%3AcV3vhD3XejH92dLIdNb9kw%2F%2BDnxi2NXZEGJOwDij%2Bsk%3D","message":"Exception: {\"Exception\":\"Doctrine\\\\DBAL\\\\Exception\\\\InvalidFieldNameException\",\"Message\":\"An exception occurred while executing 'SELECT min_value, increment_by FROM \\\"oc_account_terms_id_seq\\\"':\\n\\nSQLSTATE[42703]: Undefined column: 7 ERROR:  column \\\"min_value\\\" does not exist\\nLINE 1: SELECT min_value, increment_by FROM \\\"oc_account_terms_id_seq...\\n               ^\",\"Code\":0,\"Trace\":\"#0 \\\/var\\\/www\\\/owncloud\\\/lib\\\/composer\\\/doctrine\\\/dbal\\\/lib\\\/Doctrine\\\/DBAL\\\/DBALException.php(128): Doctrine\\\\DBAL\\\\Driver\\\\AbstractPostgreSQLDriver->convertException('An exception oc...', Object(Doctrine\\\\DBAL\\\\Driver\\\\PDOException))\\n#1 \\\/var\\\/www\\\/owncloud\\\/lib\\\/composer\\\/doctrine\\\/dbal\\\/lib\\\/Doctrine\\\/DBAL\\\/Connection.php(855): Doctrine\\\\DBAL\\\\DBALException::driverExceptionDuringQuery(Object(Doctrine\\\\DBAL\\\\Driver\\\\PDOPgSql\\\\Driver), Object(Doctrine\\\\DBAL\\\\Driver\\\\PDOException), 'SELECT min_valu...', Array)\\n#2 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/DB\\\/Connection.php(189): Doctrine\\\\DBAL\\\\Connection->executeQuery('SELECT min_valu...', Array, Array, NULL)\\n#3 \\\/var\\\/www\\\/owncloud\\\/lib\\\/composer\\\/doctrine\\\/dbal\\\/lib\\\/Doctrine\\\/DBAL\\\/Connection.php(785): OC\\\\DB\\\\Connection->executeQuery('SELECT min_valu...', Array, Array)\\n#4 \\\/var\\\/www\\\/owncloud\\\/lib\\\/composer\\\/doctrine\\\/dbal\\\/lib\\\/Doctrine\\\/DBAL\\\/Schema\\\/PostgreSqlSchemaManager.php(292): Doctrine\\\\DBAL\\\\Connection->fetchAll('SELECT min_valu...')\\n#5 \\\/var\\\/www\\\/owncloud\\\/lib\\\/composer\\\/doctrine\\\/dbal\\\/lib\\\/Doctrine\\\/DBAL\\\/Schema\\\/PostgreSqlSchemaManager.php(267): Doctrine\\\\DBAL\\\\Schema\\\\PostgreSqlSchemaManager->_getPortableSequenceDefinition(Array)\\n#6 \\\/var\\\/www\\\/owncloud\\\/lib\\\/composer\\\/doctrine\\\/dbal\\\/lib\\\/Doctrine\\\/DBAL\\\/Schema\\\/AbstractSchemaManager.php(147): Doctrine\\\\DBAL\\\\Schema\\\\PostgreSqlSchemaManager->_getPortableSequencesList(Array)\\n#7 \\\/var\\\/www\\\/owncloud\\\/lib\\\/composer\\\/doctrine\\\/dbal\\\/lib\\\/Doctrine\\\/DBAL\\\/Schema\\\/AbstractSchemaManager.php(1037): Doctrine\\\\DBAL\\\\Schema\\\\AbstractSchemaManager->listSequences()\\n#8 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/DB\\\/Migrator.php(138): Doctrine\\\\DBAL\\\\Schema\\\\AbstractSchemaManager->createSchema()\\n#9 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/DB\\\/Connection.php(427): OC\\\\DB\\\\Migrator->createSchema()\\n#10 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/DB\\\/MigrationService.php(402): OC\\\\DB\\\\Connection->createSchema()\\n#11 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/DB\\\/MigrationService.php(358): OC\\\\DB\\\\MigrationService->executeStep(20180604132522)\\n#12 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/legacy\\\/app.php(1030): OC\\\\DB\\\\MigrationService->migrate()\\n#13 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/Updater.php(307): OC_App::updateApp('notifications')\\n#14 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/Updater.php(233): OC\\\\Updater->doAppUpgrade()\\n#15 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/Updater.php(108): OC\\\\Updater->doUpgrade('10.0.9.5', '10.0.8.5')\\n#16 \\\/var\\\/www\\\/owncloud\\\/core\\\/ajax\\\/update.php(190): OC\\\\Updater->upgrade()\\n#17 {main}\",\"File\":\"\\\/var\\\/www\\\/owncloud\\\/lib\\\/composer\\\/doctrine\\\/dbal\\\/lib\\\/Doctrine\\\/DBAL\\\/Driver\\\/AbstractPostgreSQLDriver.php\",\"Line\":72}"}
{"reqId":"zdbAO2U39sy3uaAfAynC","level":3,"time":"2018-07-30T00:12:46+00:00","remoteAddr":"110.3.31.143","user":"--","app":"PHP","method":"POST","url":"\/owncloud\/index.php\/heartbeat","message":"ini_set(): A session is active. You cannot change the session module's ini settings at this time at \/var\/www\/owncloud\/lib\/base.php#603"}
{"reqId":"k6DgZEiPIvljM8i9ylXI","level":3,"time":"2018-07-30T00:19:18+00:00","remoteAddr":"110.3.31.143","user":"--","app":"PHP","method":"GET","url":"\/owncloud\/","message":"ini_set(): A session is active. You cannot change the session module's ini settings at this time at \/var\/www\/owncloud\/lib\/base.php#603"}
{"reqId":"FrYsaKCVvfzmNdvxsdAt","level":3,"time":"2018-07-30T00:19:18+00:00","remoteAddr":"110.3.31.143","user":"--","app":"PHP","method":"GET","url":"\/owncloud\/index.php\/core\/js\/oc.js?v=3a0c8db619c7824d700db01941f7ec3f","message":"ini_set(): A session is active. You cannot change the session module's ini settings at this time at \/var\/www\/owncloud\/lib\/base.php#603"}
{"reqId":"5t4VAWr1op1LwOUUhYrM","level":3,"time":"2018-07-30T00:31:19+00:00","remoteAddr":"110.3.31.143","user":"--","app":"PHP","method":"POST","url":"\/owncloud\/index.php\/heartbeat","message":"ini_set(): A session is active. You cannot change the session module's ini settings at this time at \/var\/www\/owncloud\/lib\/base.php#603"}

Browser log

No errors exists.

[ownclouders]

GitMate.io thinks possibly related issues are #12125 (Automatic upgrade after package upgrade), #28236 (Upgrade failed), #216 (Database corruption after upgrade), #16949 (Unable to upgrade Debian packages), and #25589 (After Upgrade Code integrity check fails).

[kikuhama]

The upgrade log said:

SQLSTATE[42703]: Undefined column: 7 ERROR: column "min_value" does not exist
LINE 1: SELECT min_value, increment_by FROM "oc_account_terms_id_seq...
^

It seems that "oc_account_terms_id_seq" is a sequence relation.
And according to PostgreSQL 10 release notes (https://www.postgresql.org/docs/10/static/release-10.html), metadata fields of sequenses were moved to a new pg_sequence system catalog.

Move sequences' metadata fields into a new pg_sequence system catalog (Peter Eisentraut)

A sequence relation now stores only the fields that can be modified by nextval(), that is last_value, log_cnt, and is_called. Other sequence properties, such as the starting value and increment, are kept in a corresponding row of the pg_sequence catalog. ALTER SEQUENCE updates are now fully transactional, implying that the sequence is locked until commit. The nextval() and setval() functions remain nontransactional.

The main incompatibility introduced by this change is that selecting from a sequence relation now returns only the three fields named above. To obtain the sequence's other properties, applications must look into pg_sequence. The new system view pg_sequences can also be used for this purpose; it provides column names that are more compatible with existing code.

The output of psql's \d command for a sequence has been redesigned, too.

I suppose that the database migration test of owncloud-files package is not compatible with PostgreSQL 10.

[kikuhama]

Sorry, I found previous issue #31334.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.