Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automated tagging breaks occ:scan - PostreSQL #75

Closed
DecaTec opened this issue May 16, 2019 · 8 comments · Fixed by nextcloud/server#16822
Closed

Automated tagging breaks occ:scan - PostreSQL #75

DecaTec opened this issue May 16, 2019 · 8 comments · Fixed by nextcloud/server#16822

Comments

@DecaTec
Copy link

DecaTec commented May 16, 2019

Just updated to NC 16.0.1 and wanted to do a rescan of all files.

Steps to reproduce

  1. Create a hidden tag, e.g. "Folder documents"
  2. Assign this tag to a folder.
  3. Create a rule: Assign tag "document" when file system tag is "folder documents"
  4. Now do a occ files:scan --all (or just the affected folder)

Expected behaviour

files:scan should finish without problems.

Actual behaviour

I get an error:

Exception during scan: An exception occurred while executing 'SELECT * FROM "oc_systemtag" WHERE "id" IN (?) ORDER BY "name" ASC, "visibility" ASC, "editable" ASC' with params ["12"]:

SQLSTATE[25P02]: In failed sql transaction: 7 FEHLER:  aktuelle Transaktion wurde abgebrochen, Befehle werden bis zum Ende der Transaktion ignoriert
#0 /var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/DBALException.php(184): Doctrine\DBAL\Driver\AbstractPostgreSQLDriver->convertException('An exception oc...', Object(Doctrine\DBAL\Driver\PDOException))
#1 /var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/DBALException.php(158): Doctrine\DBAL\DBALException::wrapException(Object(Doctrine\DBAL\Driver\PDOPgSql\Driver), Object(Doctrine\DBAL\Driver\PDOException), 'An exception oc...')
#2 /var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Connection.php(938): Doctrine\DBAL\DBALException::driverExceptionDuringQuery(Object(Doctrine\DBAL\Driver\PDOPgSql\Driver), Object(Doctrine\DBAL\Driver\PDOException), 'SELECT * FROM "...', Array)
#3 /var/www/nextcloud/lib/private/DB/Connection.php(195): Doctrine\DBAL\Connection->executeQuery('SELECT * FROM "...', Array, Array, NULL)
#4 /var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Query/QueryBuilder.php(219): OC\DB\Connection->executeQuery('SELECT * FROM "...', Array, Array)
#5 /var/www/nextcloud/lib/private/DB/QueryBuilder/QueryBuilder.php(214): Doctrine\DBAL\Query\QueryBuilder->execute()
#6 /var/www/nextcloud/lib/private/SystemTag/SystemTagManager.php(115): OC\DB\QueryBuilder\QueryBuilder->execute()
#7 /var/www/nextcloud/apps/systemtags/lib/Activity/Listener.php(161): OC\SystemTag\SystemTagManager->getTagsByIds(Array)
#8 /var/www/nextcloud/apps/systemtags/appinfo/app.php(56): OCA\SystemTags\Activity\Listener->mapperEvent(Object(OCP\SystemTag\MapperEvent))
#9 /var/www/nextcloud/3rdparty/symfony/event-dispatcher/EventDispatcher.php(212): OC_App::{closure}(Object(OCP\SystemTag\MapperEvent), 'OCP\\SystemTag\\I...', Object(Symfony\Component\EventDispatcher\EventDispatcher))
#10 /var/www/nextcloud/3rdparty/symfony/event-dispatcher/EventDispatcher.php(44): Symfony\Component\EventDispatcher\EventDispatcher->doDispatch(Array, 'OCP\\SystemTag\\I...', Object(OCP\SystemTag\MapperEvent))
#11 /var/www/nextcloud/lib/private/SystemTag/SystemTagObjectMapper.php(170): Symfony\Component\EventDispatcher\EventDispatcher->dispatch('OCP\\SystemTag\\I...', Object(OCP\SystemTag\MapperEvent))
#12 /var/www/nextcloud/apps/files_automatedtagging/lib/Operation.php(76): OC\SystemTag\SystemTagObjectMapper->assignTags('94757', 'files', Array)
#13 /var/www/nextcloud/apps/files_automatedtagging/lib/CacheListener.php(45): OCA\FilesAutomatedTagging\Operation->checkOperations(Object(OCA\Files_Trashbin\Storage), 94757, 'files/Dokumente...')
#14 /var/www/nextcloud/3rdparty/symfony/event-dispatcher/EventDispatcher.php(212): OCA\FilesAutomatedTagging\CacheListener->onCacheEvent(Object(OCP\Files\Cache\CacheUpdateEvent), 'OCP\\Files\\Cache...', Object(Symfony\Component\EventDispatcher\EventDispatcher))
#15 /var/www/nextcloud/3rdparty/symfony/event-dispatcher/EventDispatcher.php(44): Symfony\Component\EventDispatcher\EventDispatcher->doDispatch(Array, 'OCP\\Files\\Cache...', Object(OCP\Files\Cache\CacheUpdateEvent))
#16 /var/www/nextcloud/lib/private/Files/Cache/Cache.php(345): Symfony\Component\EventDispatcher\EventDispatcher->dispatch('OCP\\Files\\Cache...', Object(OCP\Files\Cache\CacheUpdateEvent))
#17 /var/www/nextcloud/lib/private/Files/Cache/Scanner.php(417): OC\Files\Cache\Cache->update(94757, Array)
#18 /var/www/nextcloud/lib/private/Files/Cache/Scanner.php(409): OC\Files\Cache\Scanner->scanChildren('files/Dokumente...', true, 3, 94757, true)
#19 /var/www/nextcloud/lib/private/Files/Cache/Scanner.php(409): OC\Files\Cache\Scanner->scanChildren('files/Dokumente...', true, 3, 94752, true)
#20 /var/www/nextcloud/lib/private/Files/Cache/Scanner.php(338): OC\Files\Cache\Scanner->scanChildren('files/Dokumente...', true, 3, 346, true)
#21 /var/www/nextcloud/lib/private/Files/Utils/Scanner.php(245): OC\Files\Cache\Scanner->scan('files/Dokumente...', true, 3)
#22 /var/www/nextcloud/apps/files/lib/Command/Scan.php(145): OC\Files\Utils\Scanner->scan('/Jan/files/Doku...', true, NULL)
#23 /var/www/nextcloud/apps/files/lib/Command/Scan.php(201): OCA\Files\Command\Scan->scanFiles('Jan', '/Jan/files/Doku...', Object(Symfony\Component\Console\Output\ConsoleOutput), false, true, false)
#24 /var/www/nextcloud/3rdparty/symfony/console/Command/Command.php(255): OCA\Files\Command\Scan->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#25 /var/www/nextcloud/core/Command/Base.php(166): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#26 /var/www/nextcloud/3rdparty/symfony/console/Application.php(901): OC\Core\Command\Base->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#27 /var/www/nextcloud/3rdparty/symfony/console/Application.php(262): Symfony\Component\Console\Application->doRunCommand(Object(OCA\Files\Command\Scan), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#28 /var/www/nextcloud/3rdparty/symfony/console/Application.php(145): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#29 /var/www/nextcloud/lib/private/Console/Application.php(213): Symfony\Component\Console\Application->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#30 /var/www/nextcloud/console.php(97): OC\Console\Application->run()
#31 /var/www/nextcloud/occ(11): require_once('/var/www/nextcl...')
#32 {main}

When I delete the automated tagging rule, the scan works without problems.

Server configuration

Operating system: Ubuntu server 18.04

Web server: nginx

Database: PostgreSQL 10

PHP version: 7.2

Nextcloud version: 16.0.1

Are you using encryption: no

Logs

Nextcloud log (data/owncloud.log)

No logs in nextcloud.log

@DecaTec
Copy link
Author

DecaTec commented May 19, 2019

I've tested it again with a fresh install of NC.
Maybe this is related to PostgreSQL, @rullzer ?

In the PostgreSQL log I can find this:

2019-05-19 09:30:10.423 CEST [2005] nextcloud_db_user@nextcloud_db FEHLER:  doppelter Schlüsselwert verletzt Unique-Constraint »mapping«
2019-05-19 09:30:10.423 CEST [2005] nextcloud_db_user@nextcloud_db DETAIL:  Schlüssel »(objecttype, objectid, systemtagid)=(files, 171, 2)« existiert bereits.
2019-05-19 09:30:10.423 CEST [2005] nextcloud_db_user@nextcloud_db ANWEISUNG:  INSERT INTO "oc_systemtag_object_mapping" ("objectid", "objecttype", "systemtagid") VALUES($1, $2, $3)
2019-05-19 09:30:10.426 CEST [2005] nextcloud_db_user@nextcloud_db FEHLER:  aktuelle Transaktion wurde abgebrochen, Befehle werden bis zum Ende der Transaktion ignoriert
2019-05-19 09:30:10.426 CEST [2005] nextcloud_db_user@nextcloud_db ANWEISUNG:  SELECT * FROM "oc_systemtag" WHERE "id" IN ($1) ORDER BY "name" ASC, "visibility" ASC, "editable" ASC

Can I provide any more information?

@stratege1401
Copy link

i just spend a few minutes on a virtual server to replicate your issue.

Bug replicate with 10.8 and 11.3.

Bug not replicate with mariaDB 10.3

@DecaTec DecaTec changed the title Automated tagging breaks occ:scan Automated tagging breaks occ:scan - PostreSQL May 27, 2019
@DecaTec
Copy link
Author

DecaTec commented May 27, 2019

@stratege1401 thanks for verifying this. Seems to be a problem with PostgreSQL then.

@nickvergessen
Copy link
Member

nickvergessen commented Aug 21, 2019

https://github.com/nextcloud/server/pull/16822/files should fix the issue.
Can you try it?

@DecaTec
Copy link
Author

DecaTec commented Aug 21, 2019

I think you meant this one: https://github.com/nextcloud/server/pull/16822/files

And yes, it seems to work, occ files:scan --all doesn't produce an exception now.

@mudi0
Copy link

mudi0 commented Mar 29, 2023

I have the same issue now, also using PostgreSQL and Nextcloud 25.0.5. I have the same issue with the automated tagging breaking occ:scan

@mudi0
Copy link

mudi0 commented Mar 29, 2023

this is the PostgreSQL logfile, similar to above.

2023-03-29 23:57:10.794 CEST [845034] nextclouduser@nextcloud_db FEHLER: doppelter Schlüsselwert verletzt Unique-Constraint »oc_systemtag_object_mapping_pkey«
2023-03-29 23:57:10.794 CEST [845034] nextclouduser@nextcloud_db DETAIL: Schlüssel »(objecttype, objectid, systemtagid)=(files, 4187909, 6)« existiert bereits.
2023-03-29 23:57:10.794 CEST [845034] nextclouduser@nextcloud_db ANWEISUNG: INSERT INTO "oc_systemtag_object_mapping" ("objectid", "objecttype", "systemtagid") VALUES($1, $2, $3)
2023-03-29 23:57:10.795 CEST [845034] nextclouduser@nextcloud_db FEHLER: aktuelle Transaktion wurde abgebrochen, Befehle werden bis zum Ende der Transaktion ignoriert
2023-03-29 23:57:10.795 CEST [845034] nextclouduser@nextcloud_db ANWEISUNG: UPDATE "oc_filecache" SET "size" = $1 WHERE ("fileid" = $2) AND (("size" <> $3) OR ("size" IS NULL))
2023-03-29 23:57:37.676 CEST [845341] nextclouduser@nextcloud_db FEHLER: doppelter Schlüsselwert verletzt Unique-Constraint »oc_systemtag_object_mapping_pkey«
2023-03-29 23:57:37.676 CEST [845341] nextclouduser@nextcloud_db DETAIL: Schlüssel »(objecttype, objectid, systemtagid)=(files, 4187909, 6)« existiert bereits.
2023-03-29 23:57:37.676 CEST [845341] nextclouduser@nextcloud_db ANWEISUNG: INSERT INTO "oc_systemtag_object_mapping" ("objectid", "objecttype", "systemtagid") VALUES($1, $2, $3)
2023-03-29 23:57:37.679 CEST [845341] nextclouduser@nextcloud_db FEHLER: aktuelle Transaktion wurde abgebrochen, Befehle werden bis zum Ende der Transaktion ignoriert
2023-03-29 23:57:37.679 CEST [845341] nextclouduser@nextcloud_db ANWEISUNG: UPDATE "oc_filecache" SET "size" = $1 WHERE ("fileid" = $2) AND (("size" <> $3) OR ("size" IS NULL))

@blizzz
Copy link
Member

blizzz commented Mar 30, 2023

This is closed almost 3.5 yrs ago. Please open a new issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants