S3 files external mountpoints are no longer available when upgrading from oC 10.2.1 to 10.3 alpha2

[davitol]

Steps to reproduce

1. Create aws bucket in, e.g. eu-west-1 (ireland). Set up access controls etc.
2. In oC 10.2.1 add external storage, giving bucket key and secret (global and personal)
3. mount external storage
4. Upgrade from oC 10.2.1 to 10.3
5. Install and enable files_external_s3 1.0.0 and tick S3 checkbox for personal mounts in Admin settings

Expected behaviour

Mounts are accesible as they were before

Actual behaviour

Mounts are no longer available. They are shown as Red in Files View.

{"reqId":"eXVnVwwPUmjdzWaiB8fL","level":3,"time":"2019-09-05T09:11:32+00:00","remoteAddr":"172.18.0.1","user":"pepe","app":"files_external","method":"GET","url":"\/index.php\/apps\/files_external\/userglobalstorages\/1?testOnly=false","message":"Exception: {\"Exception\":\"OCP\\\\Files\\\\StorageNotAvailableException\",\"Message\":\"\",\"Code\":1,\"Trace\":\"#0 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/Files\\\/External\\\/LegacyUtil.php(204): OC\\\\Files\\\\External\\\\InvalidStorage->__construct(Array)\\n#1 \\\/var\\\/www\\\/owncloud\\\/apps\\\/files_external\\\/lib\\\/Controller\\\/StoragesController.php(254): OC\\\\Files\\\\External\\\\LegacyUtil::getBackendStatus('\\\\\\\\OC\\\\\\\\Files\\\\\\\\Exter...', Array, false, false)\\n#2 \\\/var\\\/www\\\/owncloud\\\/apps\\\/files_external\\\/lib\\\/Controller\\\/UserGlobalStoragesController.php(123): OCA\\\\Files_External\\\\Controller\\\\StoragesController->updateStorageStatus(Object(OC\\\\Files\\\\External\\\\StorageConfig), false)\\n#3 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/AppFramework\\\/Http\\\/Dispatcher.php(153): OCA\\\\Files_External\\\\Controller\\\\UserGlobalStoragesController->show(1, false)\\n#4 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/AppFramework\\\/Http\\\/Dispatcher.php(85): OC\\\\AppFramework\\\\Http\\\\Dispatcher->executeController(Object(OCA\\\\Files_External\\\\Controller\\\\UserGlobalStoragesController), 'show')\\n#5 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/AppFramework\\\/App.php(100): OC\\\\AppFramework\\\\Http\\\\Dispatcher->dispatch(Object(OCA\\\\Files_External\\\\Controller\\\\UserGlobalStoragesController), 'show')\\n#6 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/AppFramework\\\/Routing\\\/RouteActionHandler.php(47): OC\\\\AppFramework\\\\App::main('OCA\\\\\\\\Files_Exter...', 'show', Object(OC\\\\AppFramework\\\\DependencyInjection\\\\DIContainer), Array)\\n#7 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/Route\\\/Router.php(342): OC\\\\AppFramework\\\\Routing\\\\RouteActionHandler->__invoke(Array)\\n#8 \\\/var\\\/www\\\/owncloud\\\/lib\\\/base.php(920): OC\\\\Route\\\\Router->match('\\\/apps\\\/files_ext...')\\n#9 \\\/var\\\/www\\\/owncloud\\\/index.php(54): OC::handleRequest()\\n#10 {main}\",\"File\":\"\\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/Files\\\/External\\\/InvalidStorage.php\",\"Line\":31}"}
{"reqId":"AwwjL31AtCuuoYy6r4IL","level":3,"time":"2019-09-05T09:11:33+00:00","remoteAddr":"172.18.0.1","user":"pepe","app":"webdav","method":"PROPFIND","url":"\/remote.php\/dav\/files\/pepe\/AmazonS3","message":"Caused by: {\"Exception\":\"OCP\\\\Files\\\\StorageNotAvailableException\",\"Message\":\"\",\"Code\":1,\"Trace\":\"#0 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/Files\\\/Storage\\\/Wrapper\\\/Wrapper.php(525): OC\\\\Files\\\\Storage\\\\FailedStorage->getAvailability()\\n#1 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/Files\\\/Storage\\\/Wrapper\\\/Wrapper.php(525): OC\\\\Files\\\\Storage\\\\Wrapper\\\\Wrapper->getAvailability()\\n#2 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/Files\\\/Storage\\\/Wrapper\\\/Availability.php(63): OC\\\\Files\\\\Storage\\\\Wrapper\\\\Wrapper->getAvailability()\\n#3 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/Files\\\/Storage\\\/Wrapper\\\/Availability.php(74): OC\\\\Files\\\\Storage\\\\Wrapper\\\\Availability->isAvailable()\\n#4 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/Files\\\/Storage\\\/Wrapper\\\/Availability.php(392): OC\\\\Files\\\\Storage\\\\Wrapper\\\\Availability->checkAvailability()\\n#5 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/Files\\\/Storage\\\/Wrapper\\\/Wrapper.php(378): OC\\\\Files\\\\Storage\\\\Wrapper\\\\Availability->hasUpdated('', NULL)\\n#6 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/Files\\\/Storage\\\/Wrapper\\\/Wrapper.php(378): OC\\\\Files\\\\Storage\\\\Wrapper\\\\Wrapper->hasUpdated('', NULL)\\n#7 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/Files\\\/Storage\\\/Wrapper\\\/Wrapper.php(378): OC\\\\Files\\\\Storage\\\\Wrapper\\\\Wrapper->hasUpdated('', NULL)\\n#8 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/Files\\\/Cache\\\/Watcher.php(125): OC\\\\Files\\\\Storage\\\\Wrapper\\\\Wrapper->hasUpdated('', NULL)\\n#9 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/Files\\\/View.php(1389): OC\\\\Files\\\\Cache\\\\Watcher->needsUpdate('', Object(OC\\\\Files\\\\Cache\\\\CacheEntry))\\n#10 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/Files\\\/View.php(1427): OC\\\\Files\\\\View->getCacheEntry(Object(OCA\\\\Files_Trashbin\\\\Storage), '', 'AmazonS3')\\n#11 \\\/var\\\/www\\\/owncloud\\\/apps\\\/dav\\\/lib\\\/Connector\\\/Sabre\\\/ObjectTree.php(179): OC\\\\Files\\\\View->getFileInfo('\\\/pepe\\\/files\\\/Ama...')\\n#12 \\\/var\\\/www\\\/owncloud\\\/apps\\\/dav\\\/lib\\\/Tree.php(70): OCA\\\\DAV\\\\Connector\\\\Sabre\\\\ObjectTree->getNodeForPath('AmazonS3')\\n#13 \\\/var\\\/www\\\/owncloud\\\/lib\\\/composer\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(966): OCA\\\\DAV\\\\Tree->getNodeForPath('files\\\/pepe\\\/Amaz...')\\n#14 \\\/var\\\/www\\\/owncloud\\\/lib\\\/composer\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(1649): Sabre\\\\DAV\\\\Server->getPropertiesIteratorForPath('files\\\/pepe\\\/Amaz...', Array, 1)\\n#15 \\\/var\\\/www\\\/owncloud\\\/lib\\\/composer\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/CorePlugin.php(363): Sabre\\\\DAV\\\\Server->generateMultiStatus(Object(Generator), false)\\n#16 \\\/var\\\/www\\\/owncloud\\\/lib\\\/composer\\\/sabre\\\/event\\\/lib\\\/WildcardEmitterTrait.php(96): Sabre\\\\DAV\\\\CorePlugin->httpPropFind(Object(Sabre\\\\HTTP\\\\Request), Object(Sabre\\\\HTTP\\\\Response))\\n#17 \\\/var\\\/www\\\/owncloud\\\/lib\\\/composer\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(464): Sabre\\\\DAV\\\\Server->emit('method:PROPFIND', Array)\\n#18 \\\/var\\\/www\\\/owncloud\\\/lib\\\/composer\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(241): Sabre\\\\DAV\\\\Server->invokeMethod(Object(Sabre\\\\HTTP\\\\Request), Object(Sabre\\\\HTTP\\\\Response))\\n#19 \\\/var\\\/www\\\/owncloud\\\/apps\\\/dav\\\/lib\\\/Server.php(320): Sabre\\\\DAV\\\\Server->start()\\n#20 \\\/var\\\/www\\\/owncloud\\\/apps\\\/dav\\\/appinfo\\\/v2\\\/remote.php(31): OCA\\\\DAV\\\\Server->exec()\\n#21 \\\/var\\\/www\\\/owncloud\\\/remote.php(165): require_once('\\\/var\\\/www\\\/ownclo...')\\n#22 {main}\",\"File\":\"\\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/Files\\\/Storage\\\/FailedStorage.php\",\"Line\":206}”}

Server configuration

Operating system: Ubuntu 18

Web server: Apache

Database: MySQL

ownCloud version: 10.2.1 to 10.3. alpha2

files_external_s3 version: 1.0.0 RC2

(cc @sharidas )

[davitol]

We should keep the previous configuration before the upgrade. If in 10.2.1 it was checked as enable it should be in 10.3 too

[sharidas]

Investigating this issue.

[sharidas]

With a minor change in the external_mounts table storage_backend is updated to files_external_s3 from amazons3 , the storages page now shows green button in my instance ( as if it is connected ). But then when I go to the root folder the files/folders are not found. I am currently looking into this issue.

[sharidas]

Created PRs:

• Core Add migration to change the storage backend for amazons3 core#36171
• files_external_s3 Use updated API to get the keys from the bucket and include a migration step to fix the storage backend #237 ( merged )

[sharidas]

The PR #237 is merged.

[micbar]

@pmaier1 @settermjd
We would need a good explanation in the Release notes, maybe a warning which imforms the admins of the migration from core to files_external_s3

[micbar]

Fixed.