Upgrade from 9.1.3 to master :bomb: #26907

Open
PVince81 opened this Issue Jan 9, 2017 · 4 comments

Projects

None yet

1 participant

@PVince81
Collaborator
PVince81 commented Jan 9, 2017

Steps:

  1. Setup v9.1.3 from git
  2. Checkout master (5a10d04) and run make clean; make
  3. Run occ upgrade from the CLI

Expected

Upgrade success.

Actual

Upgrade failed:

Set log level to debug
Turned on maintenance mode
Updating database schema
Updated database
Disabled 3rd-party app: admin_migrate
Disabled 3rd-party app: files_odfviewer
Disabled incompatible app: files_texteditor
Disabled incompatible app: files_videoviewer
Disabled 3rd-party app: files_videoviewer
Disabled 3rd-party app: user_migrate
Drop old database tables

 Done
 28/28 [============================] 100%
Remove old (< 9.0) calendar/contact shares
 Done
 4/4 [============================] 100%
Fix permissions so avatars can be stored again
 Done
 2/2 [============================] 100%
Move user avatars outside the homes to the new location
 Starting ...
 0/1 [>---------------------------]   0%Doctrine\DBAL\Exception\TableNotFoundException: An exception occurred while executing 'SELECT m.`mount_id`, `mount_point`, `storage_backend`, `auth_backend`, `priority`, m.`type` FROM `oc_external_mounts` m INNER JOIN `oc_external_applicable` a ON m.`mount_id` = a.`mount_id` WHERE (a.`type` = ?) AND (a.`value` = ?) AND (m.`type` = '1')' with params [3, "admin"]:

SQLSTATE[42S02]: Base table or view not found: 1146 Table 'owncloud.oc_external_mounts' doesn't exist
Update failed
Maintenance mode is kept active
Reset log level

Log

{"reqId":"tAyO1k42KrQVZ0rHQNAC","remoteAddr":"","app":"core","message":"Exception: {\"Exception\":\"Doctrine\\\\DBAL\\\\Exception\\\\TableNotFoundException\",\"Message\":\"An exception occurred while executing 'SELECT m.`mount_id`, `mount_point`, `storage_backend`, `auth_backend`, `priority`, m.`type` FROM `oc_external_mounts` m INNER JOIN `oc_external_applicable` a ON m.`mount_id` = a.`mount_id` WHERE (a.`type` = ?) AND (a.`value` = ?) AND (m.`type` = '1')' with params [3, \\\"admin\\\"]:\\n\\nSQLSTATE[42S02]: Base table or view not found: 1146 Table 'owncloud.oc_external_mounts' doesn't exist\",\"Code\":0,\"Trace\":\"#0 \\\/srv\\\/www\\\/htdocs\\\/owncloud\\\/lib\\\/composer\\\/doctrine\\\/dbal\\\/lib\\\/Doctrine\\\/DBAL\\\/DBALException.php(128): Doctrine\\\\DBAL\\\\Driver\\\\AbstractMySQLDriver->convertException('An exception oc...', Object(Doctrine\\\\DBAL\\\\Driver\\\\PDOException))\\n#1 \\\/srv\\\/www\\\/htdocs\\\/owncloud\\\/lib\\\/composer\\\/doctrine\\\/dbal\\\/lib\\\/Doctrine\\\/DBAL\\\/Connection.php(836): Doctrine\\\\DBAL\\\\DBALException::driverExceptionDuringQuery(Object(Doctrine\\\\DBAL\\\\Driver\\\\PDOMySql\\\\Driver), Object(Doctrine\\\\DBAL\\\\Driver\\\\PDOException), 'SELECT m.`mount...', Array)\\n#2 \\\/srv\\\/www\\\/htdocs\\\/owncloud\\\/lib\\\/private\\\/DB\\\/Connection.php(188): Doctrine\\\\DBAL\\\\Connection->executeQuery('SELECT m.`mount...', Array, Array, NULL)\\n#3 \\\/srv\\\/www\\\/htdocs\\\/owncloud\\\/lib\\\/composer\\\/doctrine\\\/dbal\\\/lib\\\/Doctrine\\\/DBAL\\\/Query\\\/QueryBuilder.php(206): OC\\\\DB\\\\Connection->executeQuery('SELECT m.`mount...', Array, Array)\\n#4 \\\/srv\\\/www\\\/htdocs\\\/owncloud\\\/lib\\\/private\\\/DB\\\/QueryBuilder\\\/QueryBuilder.php(141): Doctrine\\\\DBAL\\\\Query\\\\QueryBuilder->execute()\\n#5 \\\/srv\\\/www\\\/htdocs\\\/owncloud\\\/lib\\\/private\\\/Files\\\/External\\\/Service\\\/DBConfigService.php(335): OC\\\\DB\\\\QueryBuilder\\\\QueryBuilder->execute()\\n#6 \\\/srv\\\/www\\\/htdocs\\\/owncloud\\\/lib\\\/private\\\/Files\\\/External\\\/Service\\\/DBConfigService.php(145): OC\\\\Files\\\\External\\\\Service\\\\DBConfigService->getMountsFromQuery(Object(OC\\\\DB\\\\QueryBuilder\\\\QueryBuilder))\\n#7 \\\/srv\\\/www\\\/htdocs\\\/owncloud\\\/lib\\\/private\\\/Files\\\/External\\\/Service\\\/UserGlobalStoragesService.php(76): OC\\\\Files\\\\External\\\\Service\\\\DBConfigService->getAdminMountsFor(3, 'admin')\\n#8 \\\/srv\\\/www\\\/htdocs\\\/owncloud\\\/lib\\\/private\\\/Files\\\/External\\\/Service\\\/StoragesService.php(128): OC\\\\Files\\\\External\\\\Service\\\\UserGlobalStoragesService->readDBConfig()\\n#9 \\\/srv\\\/www\\\/htdocs\\\/owncloud\\\/lib\\\/private\\\/Files\\\/External\\\/Service\\\/StoragesService.php(178): OC\\\\Files\\\\External\\\\Service\\\\StoragesService->readConfig()\\n#10 \\\/srv\\\/www\\\/htdocs\\\/owncloud\\\/lib\\\/private\\\/Files\\\/External\\\/Service\\\/StoragesService.php(187): OC\\\\Files\\\\External\\\\Service\\\\StoragesService->getAllStorages()\\n#11 \\\/srv\\\/www\\\/htdocs\\\/owncloud\\\/lib\\\/private\\\/Files\\\/External\\\/Service\\\/UserGlobalStoragesService.php(109): OC\\\\Files\\\\External\\\\Service\\\\StoragesService->getStorages()\\n#12 \\\/srv\\\/www\\\/htdocs\\\/owncloud\\\/lib\\\/private\\\/Files\\\/External\\\/ConfigAdapter.php(119): OC\\\\Files\\\\External\\\\Service\\\\UserGlobalStoragesService->getUniqueStorages()\\n#13 \\\/srv\\\/www\\\/htdocs\\\/owncloud\\\/lib\\\/private\\\/Files\\\/Config\\\/MountProviderCollection.php(76): OC\\\\Files\\\\External\\\\ConfigAdapter->getMountsForUser(Object(OC\\\\User\\\\User), Object(OC\\\\Files\\\\Storage\\\\StorageFactory))\\n#14 [internal function]: OC\\\\Files\\\\Config\\\\MountProviderCollection->OC\\\\Files\\\\Config\\\\{closure}(Object(OC\\\\Files\\\\External\\\\ConfigAdapter))\\n#15 \\\/srv\\\/www\\\/htdocs\\\/owncloud\\\/lib\\\/private\\\/Files\\\/Config\\\/MountProviderCollection.php(77): array_map(Object(Closure), Array)\\n#16 \\\/srv\\\/www\\\/htdocs\\\/owncloud\\\/lib\\\/private\\\/Files\\\/Filesystem.php(443): OC\\\\Files\\\\Config\\\\MountProviderCollection->getMountsForUser(Object(OC\\\\User\\\\User))\\n#17 \\\/srv\\\/www\\\/htdocs\\\/owncloud\\\/lib\\\/private\\\/Repair\\\/MoveAvatarOutsideHome.php(106): OC\\\\Files\\\\Filesystem::initMountPoints('admin')\\n#18 \\\/srv\\\/www\\\/htdocs\\\/owncloud\\\/lib\\\/private\\\/Repair\\\/MoveAvatarOutsideHome.php(134): OC\\\\Repair\\\\MoveAvatarOutsideHome->moveAvatars(Object(OC\\\\Repair), Object(OC\\\\User\\\\User))\\n#19 \\\/srv\\\/www\\\/htdocs\\\/owncloud\\\/lib\\\/private\\\/User\\\/Manager.php(411): OC\\\\Repair\\\\MoveAvatarOutsideHome->OC\\\\Repair\\\\{closure}(Object(OC\\\\User\\\\User))\\n#20 \\\/srv\\\/www\\\/htdocs\\\/owncloud\\\/lib\\\/private\\\/Repair\\\/MoveAvatarOutsideHome.php(140): OC\\\\User\\\\Manager->callForSeenUsers(Object(Closure))\\n#21 \\\/srv\\\/www\\\/htdocs\\\/owncloud\\\/lib\\\/private\\\/Repair.php(89): OC\\\\Repair\\\\MoveAvatarOutsideHome->run(Object(OC\\\\Repair))\\n#22 \\\/srv\\\/www\\\/htdocs\\\/owncloud\\\/lib\\\/private\\\/Updater.php(254): OC\\\\Repair->run()\\n#23 \\\/srv\\\/www\\\/htdocs\\\/owncloud\\\/lib\\\/private\\\/Updater.php(121): OC\\\\Updater->doUpgrade('9.2.0.3', '9.1.2.5')\\n#24 \\\/srv\\\/www\\\/htdocs\\\/owncloud\\\/core\\\/Command\\\/Upgrade.php(259): OC\\\\Updater->upgrade()\\n#25 \\\/srv\\\/www\\\/htdocs\\\/owncloud\\\/lib\\\/composer\\\/symfony\\\/console\\\/Command\\\/Command.php(256): OC\\\\Core\\\\Command\\\\Upgrade->execute(Object(Symfony\\\\Component\\\\Console\\\\Input\\\\ArgvInput), Object(Symfony\\\\Component\\\\Console\\\\Output\\\\ConsoleOutput))\\n#26 \\\/srv\\\/www\\\/htdocs\\\/owncloud\\\/lib\\\/composer\\\/symfony\\\/console\\\/Application.php(818): Symfony\\\\Component\\\\Console\\\\Command\\\\Command->run(Object(Symfony\\\\Component\\\\Console\\\\Input\\\\ArgvInput), Object(Symfony\\\\Component\\\\Console\\\\Output\\\\ConsoleOutput))\\n#27 \\\/srv\\\/www\\\/htdocs\\\/owncloud\\\/lib\\\/composer\\\/symfony\\\/console\\\/Application.php(186): Symfony\\\\Component\\\\Console\\\\Application->doRunCommand(Object(OC\\\\Core\\\\Command\\\\Upgrade), Object(Symfony\\\\Component\\\\Console\\\\Input\\\\ArgvInput), Object(Symfony\\\\Component\\\\Console\\\\Output\\\\ConsoleOutput))\\n#28 \\\/srv\\\/www\\\/htdocs\\\/owncloud\\\/lib\\\/composer\\\/symfony\\\/console\\\/Application.php(117): Symfony\\\\Component\\\\Console\\\\Application->doRun(Object(Symfony\\\\Component\\\\Console\\\\Input\\\\ArgvInput), Object(Symfony\\\\Component\\\\Console\\\\Output\\\\ConsoleOutput))\\n#29 \\\/srv\\\/www\\\/htdocs\\\/owncloud\\\/lib\\\/private\\\/Console\\\/Application.php(158): Symfony\\\\Component\\\\Console\\\\Application->run(NULL, NULL)\\n#30 \\\/srv\\\/www\\\/htdocs\\\/owncloud\\\/console.php(99): OC\\\\Console\\\\Application->run()\\n#31 \\\/srv\\\/www\\\/htdocs\\\/owncloud\\\/occ(11): require_once('\\\/srv\\\/www\\\/htdocs...')\\n#32 {main}\",\"File\":\"\\\/srv\\\/www\\\/htdocs\\\/owncloud\\\/lib\\\/composer\\\/doctrine\\\/dbal\\\/lib\\\/Doctrine\\\/DBAL\\\/Driver\\\/AbstractMySQLDriver.php\",\"Line\":53}","level":3,"time":"2017-01-09T10:21:04+00:00","method":"--","url":"--","user":"--"}

Seems to be a clash between the "move avatars outside repair code" and the "move files_external to core" code. For some reason the table might not exist yet at the time this repair step is executed. Need to check why...

@PVince81 PVince81 added this to the 10.0 milestone Jan 9, 2017
@PVince81 PVince81 self-assigned this Jan 9, 2017
@PVince81
Collaborator

Oh... looks like now that we have Doctrine migrations ownCloud ignores changes to db_structure.xml, so basically the files_external tables that are now in core are not getting created automatically 😢

So this means I have to rewrite the whole thing as a conditional Doctrine migration (conditional because the tables might already exist before).

@DeepDiver1975 correct ?

@PVince81
Collaborator

@DeepDiver1975 I wish there was a script to convert XML migrations to an actual Doctrine code migration to save me some time. Sounds like I'll have to lose another day or two for that.

@PVince81
Collaborator
PVince81 commented Jan 11, 2017 edited

Structure to be implemented as Doctrine migration:

	<!-- External storage definition -->
	<table>
		<name>*dbprefix*external_mounts</name>
		<declaration>
			<field>
				<name>mount_id</name>
				<type>integer</type>
				<default>0</default>
				<notnull>true</notnull>
				<autoincrement>1</autoincrement>
				<length>6</length>
			</field>
			<field>
				<name>mount_point</name>
				<type>text</type>
				<length>128</length>
				<notnull>true</notnull>
			</field>
			<field>
				<name>storage_backend</name>
				<type>text</type>
				<length>64</length>
				<notnull>true</notnull>
			</field>
			<field>
				<name>auth_backend</name>
				<type>text</type>
				<length>64</length>
				<notnull>true</notnull>
			</field>
			<field>
				<name>priority</name>
				<type>integer</type>
				<default>100</default>
				<length>4</length>
				<notnull>true</notnull>
			</field>
			<!-- admin = 1, personal = 2-->
			<field>
				<name>type</name>
				<type>integer</type>
				<length>4</length>
				<notnull>true</notnull>
			</field>
		</declaration>
	</table>
	<table>
		<name>*dbprefix*external_applicable</name>
		<declaration>
			<field>
				<name>applicable_id</name>
				<type>integer</type>
				<default>0</default>
				<notnull>true</notnull>
				<autoincrement>1</autoincrement>
				<length>6</length>
			</field>
			<field>
				<!--foreign key: external_mounts.mount_id-->
				<name>mount_id</name>
				<type>integer</type>
				<notnull>true</notnull>
				<length>6</length>
			</field>
			<field>
				<!-- possible mount types: global = 1, group = 2, user = 3 -->
				<name>type</name>
				<type>integer</type>
				<length>4</length>
				<notnull>true</notnull>
			</field>
			<field>
				<!-- user_id, group_id or null for global mounts -->
				<name>value</name>
				<type>text</type>
				<length>64</length>
			</field>

			<index>
				<name>applicable_mount</name>
				<field>
					<name>mount_id</name>
					<sorting>ascending</sorting>
				</field>
			</index>
			<index>
				<name>applicable_type_value</name>
				<field>
					<name>type</name>
					<sorting>ascending</sorting>
				</field>
				<field>
					<name>value</name>
					<sorting>ascending</sorting>
				</field>
			</index>
			<index>
				<name>applicable_type_value_mount</name>
				<unique>true</unique>
				<field>
					<name>type</name>
					<sorting>ascending</sorting>
				</field>
				<field>
					<name>value</name>
					<sorting>ascending</sorting>
				</field>
				<field>
					<name>mount_id</name>
					<sorting>ascending</sorting>
				</field>
			</index>
		</declaration>
	</table>

	<table>
		<name>*dbprefix*external_config</name>
		<declaration>
			<field>
				<name>config_id</name>
				<type>integer</type>
				<default>0</default>
				<notnull>true</notnull>
				<autoincrement>1</autoincrement>
				<length>6</length>
			</field>
			<field>
				<!--foreign key: external_mounts.mount_id-->
				<name>mount_id</name>
				<type>integer</type>
				<notnull>true</notnull>
				<length>6</length>
			</field>
			<field>
				<name>key</name>
				<type>text</type>
				<notnull>true</notnull>
				<length>64</length>
			</field>
			<field>
				<name>value</name>
				<type>text</type>
				<notnull>true</notnull>
				<length>4096</length>
			</field>

			<index>
				<name>config_mount</name>
				<field>
					<name>mount_id</name>
					<sorting>ascending</sorting>
				</field>
			</index>
			<index>
				<name>config_mount_key</name>
				<unique>true</unique>
				<field>
					<name>mount_id</name>
					<sorting>ascending</sorting>
				</field>
				<field>
					<name>key</name>
					<sorting>ascending</sorting>
				</field>
			</index>
		</declaration>
	</table>

	<table>
		<name>*dbprefix*external_options</name>
		<declaration>
			<field>
				<name>option_id</name>
				<type>integer</type>
				<default>0</default>
				<notnull>true</notnull>
				<autoincrement>1</autoincrement>
				<length>6</length>
			</field>
			<field>
				<!--foreign key: external_mounts.mount_id-->
				<name>mount_id</name>
				<type>integer</type>
				<notnull>true</notnull>
				<length>6</length>
			</field>
			<field>
				<name>key</name>
				<type>text</type>
				<notnull>true</notnull>
				<length>64</length>
			</field>
			<field>
				<name>value</name>
				<type>text</type>
				<notnull>true</notnull>
				<length>256</length>
			</field>

			<index>
				<name>option_mount</name>
				<field>
					<name>mount_id</name>
					<sorting>ascending</sorting>
				</field>
			</index>
			<index>
				<name>option_mount_key</name>
				<unique>true</unique>
				<field>
					<name>mount_id</name>
					<sorting>ascending</sorting>
				</field>
				<field>
					<name>key</name>
					<sorting>ascending</sorting>
				</field>
			</index>
		</declaration>
	</table>
@PVince81
Collaborator

PR here: #26923

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment