Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

nextcloud does not use s3 external storrage #5516

Open
johansmitsnl opened this issue Jun 22, 2017 · 9 comments
Open

nextcloud does not use s3 external storrage #5516

johansmitsnl opened this issue Jun 22, 2017 · 9 comments

Comments

@johansmitsnl
Copy link

@johansmitsnl johansmitsnl commented Jun 22, 2017

Steps to reproduce

  1. do a clean install
  2. open the config
  3. configure the s3 endpoint with tokens

Expected behaviour

The page should load

Actual behaviour

Internal Server Error on the login page

Server configuration

Operating system:
docker nextcloud 12-apache 7b2ad2ebe922

Web server:
docker nextcloud 12-apache 7b2ad2ebe922

Database:
mariadb

PHP version:
docker nextcloud 12-apache 7b2ad2ebe922

Nextcloud version: (see Nextcloud admin page)
12

Updated from an older Nextcloud/ownCloud or fresh install:
fresh

Where did you install Nextcloud from:
docker

Login as admin user into your Nextcloud and access 
http://example.com/index.php/settings/integrity/failed 
paste the results here.

Unable to login

List of activated apps:

App list External storrage + ldap ``` If you have access to your command line run e.g.: sudo -u www-data php occ app:list from within your Nextcloud installation folder ```

Nextcloud configuration:

Config report
<?php
$CONFIG = array (
  'apps_paths' =>
  array (
    0 =>
    array (
      'path' => '/var/www/html/apps',
      'url' => '/apps',
      'writable' => false,
    ),
    1 =>
    array (
      'path' => '/var/www/html/custom_apps',
      'url' => '/custom_apps',
      'writable' => true,
    ),
  ),
  'instanceid' => 'xxx',
  'passwordsalt' => 'xxx',
  'secret' => 'xxx+xxx',
  'trusted_domains' =>
  array (
    0 => 'nextcloud.xxx.net',
  ),
  'datadirectory' => '/var/www/html/data',
  'overwrite.cli.url' => 'https://nextcloud.xxx.net',
  'dbtype' => 'mysql',
  'version' => '12.0.0.29',
  'dbname' => 'nextcloud',
  'dbhost' => 'nextcloud_db:3306',
  'dbport' => '',
  'dbtableprefix' => 'oc_',
  'dbuser' => 'nextcloud',
  'dbpassword' => 'xxx',
  'installed' => true,
  'ldapIgnoreNamingRules' => false,
  'ldapProviderFactory' => '\\OCA\\User_LDAP\\LDAPProviderFactory',
  'mail_smtpmode' => 'smtp',
  'mail_smtpauthtype' => 'LOGIN',
  'mail_from_address' => 'cloud',
  'mail_domain' => 'xxx.net',
  'mail_smtpauth' => 1,
  'mail_smtphost' => 'mail',
  'mail_smtpport' => '25',
  'mail_smtpname' => 'nextcloud',
  'mail_smtppassword' => 'xxx',
  'objectstore' => array(
        'class' => 'OC\\Files\\ObjectStore\\S3',
        'arguments' => array(
                'bucket' => 'smi',
                'autocreate' => true,
                'key'    => 'xxx',
                'secret' => 'xxx',
                'use_ssl' => true,
                'region' => 'eu-central-1',
                // required for some non amazon s3 implementations
                'use_path_style'=>false
        ),
  ),
);

Are you using external storage, if yes which one: S3

Are you using encryption: no

Are you using an external user-backend, if yes which one: LDAP

Client configuration

Browser:
chrome

Operating system:

Logs

Nextcloud log (data/nextcloud.log)

Nextcloud log
{"reqId":"UBSD5omBgvlmOKydpQau","level":3,"time":"2017-06-22T20:16:09+00:00","remoteAddr":"172.16.38.4","user":"c6920486-eaff-1036-9f81-bb71a5790f66","app":"index","method":"GET","url":"\/index.php\/apps\/files\/","message":"Exception: {\"Exception\":\"OCP\\\\Files\\\\NotFoundException\",\"Message\":\"\",\"Code\":0,\"Trace\":\"#0 \\\/var\\\/www\\\/html\\\/apps\\\/files\\\/lib\\\/Controller\\\/ViewController.php(130): OC_Helper::getStorageInfo('\\\/', false)\\n#1 \\\/var\\\/www\\\/html\\\/apps\\\/files\\\/lib\\\/Controller\\\/ViewController.php(159): OCA\\\\Files\\\\Controller\\\\ViewController->getStorageInfo()\\n#2 [internal function]: OCA\\\\Files\\\\Controller\\\\ViewController->index('', '', NULL, false)\\n#3 \\\/var\\\/www\\\/html\\\/lib\\\/private\\\/AppFramework\\\/Http\\\/Dispatcher.php(160): call_user_func_array(Array, Array)\\n#4 \\\/var\\\/www\\\/html\\\/lib\\\/private\\\/AppFramework\\\/Http\\\/Dispatcher.php(90): OC\\\\AppFramework\\\\Http\\\\Dispatcher->executeController(Object(OCA\\\\Files\\\\Controller\\\\ViewController), 'index')\\n#5 \\\/var\\\/www\\\/html\\\/lib\\\/private\\\/AppFramework\\\/App.php(114): OC\\\\AppFramework\\\\Http\\\\Dispatcher->dispatch(Object(OCA\\\\Files\\\\Controller\\\\ViewController), 'index')\\n#6 \\\/var\\\/www\\\/html\\\/lib\\\/private\\\/AppFramework\\\/Routing\\\/RouteActionHandler.php(47): OC\\\\AppFramework\\\\App::main('ViewController', 'index', Object(OC\\\\AppFramework\\\\DependencyInjection\\\\DIContainer), Array)\\n#7 [internal function]: OC\\\\AppFramework\\\\Routing\\\\RouteActionHandler->__invoke(Array)\\n#8 \\\/var\\\/www\\\/html\\\/lib\\\/private\\\/Route\\\/Router.php(299): call_user_func(Object(OC\\\\AppFramework\\\\Routing\\\\RouteActionHandler), Array)\\n#9 \\\/var\\\/www\\\/html\\\/lib\\\/base.php(1000): OC\\\\Route\\\\Router->match('\\\/apps\\\/files\\\/')\\n#10 \\\/var\\\/www\\\/html\\\/index.php(40): OC::handleRequest()\\n#11 {main}\",\"File\":\"\\\/var\\\/www\\\/html\\\/lib\\\/private\\\/legacy\\\/helper.php\",\"Line\":543}","userAgent":"Mozilla\/5.0 (Macintosh; Intel Mac OS X 10_12_5) AppleWebKit\/537.36 (KHTML, like Gecko) Chrome\/59.0.3071.104 Safari\/537.36","version":"12.0.0.29"}
{"reqId":"UBSD5omBgvlmOKydpQau","level":3,"time":"2017-06-22T20:16:09+00:00","remoteAddr":"172.16.38.4","user":"c6920486-eaff-1036-9f81-bb71a5790f66","app":"objectstore","method":"GET","url":"\/index.php\/apps\/files\/","message":"Could not create object: A header you provided implies functionality that is not implemented","userAgent":"Mozilla\/5.0 (Macintosh; Intel Mac OS X 10_12_5) AppleWebKit\/537.36 (KHTML, like Gecko) Chrome\/59.0.3071.104 Safari\/537.36","version":"12.0.0.29"}
{"reqId":"UBSD5omBgvlmOKydpQau","level":3,"time":"2017-06-22T20:16:10+00:00","remoteAddr":"172.16.38.4","user":"c6920486-eaff-1036-9f81-bb71a5790f66","app":"objectstore","method":"GET","url":"\/index.php\/apps\/files\/","message":"Could not create object: A header you provided implies functionality that is not implemented","userAgent":"Mozilla\/5.0 (Macintosh; Intel Mac OS X 10_12_5) AppleWebKit\/537.36 (KHTML, like Gecko) Chrome\/59.0.3071.104 Safari\/537.36","version":"12.0.0.29"}
{"reqId":"UBSD5omBgvlmOKydpQau","level":3,"time":"2017-06-22T20:16:10+00:00","remoteAddr":"172.16.38.4","user":"c6920486-eaff-1036-9f81-bb71a5790f66","app":"objectstore","method":"GET","url":"\/index.php\/apps\/files\/","message":"Could not create object: A header you provided implies functionality that is not implemented","userAgent":"Mozilla\/5.0 (Macintosh; Intel Mac OS X 10_12_5) AppleWebKit\/537.36 (KHTML, like Gecko) Chrome\/59.0.3071.104 Safari\/537.36","version":"12.0.0.29"}
{"reqId":"UBSD5omBgvlmOKydpQau","level":3,"time":"2017-06-22T20:16:10+00:00","remoteAddr":"172.16.38.4","user":"c6920486-eaff-1036-9f81-bb71a5790f66","app":"core","method":"GET","url":"\/index.php\/apps\/files\/","message":"Exception: {\"Exception\":\"OCP\\\\Files\\\\NotFoundException\",\"Message\":\"\",\"Code\":0,\"Trace\":\"#0 \\\/var\\\/www\\\/html\\\/apps\\\/files\\\/lib\\\/Controller\\\/ViewController.php(130): OC_Helper::getStorageInfo('\\\/', false)\\n#1 \\\/var\\\/www\\\/html\\\/apps\\\/files\\\/lib\\\/Controller\\\/ViewController.php(159): OCA\\\\Files\\\\Controller\\\\ViewController->getStorageInfo()\\n#2 [internal function]: OCA\\\\Files\\\\Controller\\\\ViewController->index('', '', NULL, false)\\n#3 \\\/var\\\/www\\\/html\\\/lib\\\/private\\\/AppFramework\\\/Http\\\/Dispatcher.php(160): call_user_func_array(Array, Array)\\n#4 \\\/var\\\/www\\\/html\\\/lib\\\/private\\\/AppFramework\\\/Http\\\/Dispatcher.php(90): OC\\\\AppFramework\\\\Http\\\\Dispatcher->executeController(Object(OCA\\\\Files\\\\Controller\\\\ViewController), 'index')\\n#5 \\\/var\\\/www\\\/html\\\/lib\\\/private\\\/AppFramework\\\/App.php(114): OC\\\\AppFramework\\\\Http\\\\Dispatcher->dispatch(Object(OCA\\\\Files\\\\Controller\\\\ViewController), 'index')\\n#6 \\\/var\\\/www\\\/html\\\/lib\\\/private\\\/AppFramework\\\/Routing\\\/RouteActionHandler.php(47): OC\\\\AppFramework\\\\App::main('ViewController', 'index', Object(OC\\\\AppFramework\\\\DependencyInjection\\\\DIContainer), Array)\\n#7 [internal function]: OC\\\\AppFramework\\\\Routing\\\\RouteActionHandler->__invoke(Array)\\n#8 \\\/var\\\/www\\\/html\\\/lib\\\/private\\\/Route\\\/Router.php(299): call_user_func(Object(OC\\\\AppFramework\\\\Routing\\\\RouteActionHandler), Array)\\n#9 \\\/var\\\/www\\\/html\\\/lib\\\/base.php(1000): OC\\\\Route\\\\Router->match('\\\/apps\\\/files\\\/')\\n#10 \\\/var\\\/www\\\/html\\\/index.php(40): OC::handleRequest()\\n#11 {main}\",\"File\":\"\\\/var\\\/www\\\/html\\\/lib\\\/private\\\/legacy\\\/helper.php\",\"Line\":543}","userAgent":"Mozilla\/5.0 (Macintosh; Intel Mac OS X 10_12_5) AppleWebKit\/537.36 (KHTML, like Gecko) Chrome\/59.0.3071.104 Safari\/537.36","version":"12.0.0.29"}
{"reqId":"UBSD5omBgvlmOKydpQau","level":3,"time":"2017-06-22T20:16:10+00:00","remoteAddr":"172.16.38.4","user":"c6920486-eaff-1036-9f81-bb71a5790f66","app":"core","method":"GET","url":"\/index.php\/apps\/files\/","message":"Exception: {\"Exception\":\"OCP\\\\Files\\\\NotFoundException\",\"Message\":\"\",\"Code\":0,\"Trace\":\"#0 \\\/var\\\/www\\\/html\\\/lib\\\/private\\\/Files\\\/Node\\\/Node.php(217): OC\\\\Files\\\\Node\\\\Node->getFileInfo()\\n#1 \\\/var\\\/www\\\/html\\\/lib\\\/private\\\/Files\\\/Node\\\/Node.php(117): OC\\\\Files\\\\Node\\\\Node->getPermissions()\\n#2 \\\/var\\\/www\\\/html\\\/lib\\\/private\\\/Files\\\/Node\\\/File.php(62): OC\\\\Files\\\\Node\\\\Node->checkPermissions(2)\\n#3 \\\/var\\\/www\\\/html\\\/lib\\\/private\\\/Files\\\/SimpleFS\\\/SimpleFile.php(95): OC\\\\Files\\\\Node\\\\File->putContent('(function($) {\\\\n...')\\n#4 \\\/var\\\/www\\\/html\\\/lib\\\/private\\\/Template\\\/JSCombiner.php(163): OC\\\\Files\\\\SimpleFS\\\\SimpleFile->putContent('(function($) {\\\\n...')\\n#5 \\\/var\\\/www\\\/html\\\/lib\\\/private\\\/Template\\\/JSCombiner.php(89): OC\\\\Template\\\\JSCombiner->cache('\\\/var\\\/www\\\/html\\\/c...', 'merged-template...', Object(OC\\\\Files\\\\SimpleFS\\\\SimpleFolder))\\n#6 \\\/var\\\/www\\\/html\\\/lib\\\/private\\\/Template\\\/JSResourceLocator.php(105): OC\\\\Template\\\\JSCombiner->process('\\\/var\\\/www\\\/html', 'core\\\/js\\\/merged-...', 'core')\\n#7 \\\/var\\\/www\\\/html\\\/lib\\\/private\\\/Template\\\/JSResourceLocator.php(68): OC\\\\Template\\\\JSResourceLocator->cacheAndAppendCombineJsonIfExist('\\\/var\\\/www\\\/html', 'core\\\/js\\\/merged-...')\\n#8 \\\/var\\\/www\\\/html\\\/lib\\\/private\\\/Template\\\/ResourceLocator.php(75): OC\\\\Template\\\\JSResourceLocator->doFind('js\\\/merged-templ...')\\n#9 \\\/var\\\/www\\\/html\\\/lib\\\/private\\\/TemplateLayout.php(249): OC\\\\Template\\\\ResourceLocator->find(Array)\\n#10 \\\/var\\\/www\\\/html\\\/lib\\\/private\\\/TemplateLayout.php(132): OC\\\\TemplateLayout::findJavascriptFiles(Array)\\n#11 \\\/var\\\/www\\\/html\\\/lib\\\/private\\\/legacy\\\/template.php(205): OC\\\\TemplateLayout->__construct('error', '')\\n#12 \\\/var\\\/www\\\/html\\\/lib\\\/private\\\/Template\\\/Base.php(132): OC_Template->fetchPage()\\n#13 \\\/var\\\/www\\\/html\\\/lib\\\/private\\\/legacy\\\/template.php(342): OC\\\\Template\\\\Base->printPage()\\n#14 \\\/var\\\/www\\\/html\\\/index.php(59): OC_Template::printExceptionErrorPage(Object(OCP\\\\Files\\\\NotFoundException))\\n#15 {main}\",\"File\":\"\\\/var\\\/www\\\/html\\\/lib\\\/private\\\/Files\\\/Node\\\/Node.php\",\"Line\":97}","userAgent":"Mozilla\/5.0 (Macintosh; Intel Mac OS X 10_12_5) AppleWebKit\/537.36 (KHTML, like Gecko) Chrome\/59.0.3071.104 Safari\/537.36","version":"12.0.0.29"}
@tompinzler
Copy link

@tompinzler tompinzler commented Jul 5, 2017

I have the same problem with Swift storage configured as primary storage. I'm not using docker but Nextcloud 12 in /var/www/html on Ubuntu 16.04 with Apache2.

Swift storage is working fine for me with Nextcloud version 11.0.3.

@johansmitsnl
Copy link
Author

@johansmitsnl johansmitsnl commented Jul 5, 2017

Found the "solution" this week.
You have to clear first all files from the account. Then make the config change and it will work. There is not migration or detection that you have switched the primary storage backend.

This might be more clearly documented thats why I leave this ticket open. Hope @tompinzler this info is helpful.

@tompinzler
Copy link

@tompinzler tompinzler commented Jul 5, 2017

Hm, I'm trying to activate Swift storage on a brand new installation without existing files. At this point there is only one existing user which was generated during installation and that user has never logged in.
When I try to login I get "Internal Server Error" all the time.

@johansmitsnl
Copy link
Author

@johansmitsnl johansmitsnl commented Jul 5, 2017

But there are always some generated files to "welcome" the user. Delete all these files and empty the trash. And then give it a try.

@tompinzler
Copy link

@tompinzler tompinzler commented Jul 5, 2017

Tried that. No luck. I deleted the files on the file system but the "Internal Server Error" persists.

@johansmitsnl Thanks for your effort!

@MorrisJobke MorrisJobke self-assigned this Jul 18, 2017
@beniwtv
Copy link

@beniwtv beniwtv commented Aug 17, 2017

I have the same problem trying to set up S3 on Nextcloud 12, I get an error 500. The logs reveal:

{"reqId":"XE5C9YB46DGFuZuqhfyP","level":3,"time":"2017-08-17T15:30:32+00:00","remoteAddr":"192.168.0.1","user":"--","app":"objectstore","method":"GET","url":"\/login","message":"Could not create object urn:oid:6633 for appdata_ocqp2z83n7ro\/js\/core\/merged-share-backend.js.deps: {\"Exception\":\"Aws\\\\S3\\\\Exception\\\\MissingContentLengthException\",\"Message\":\"You must provide the Content-Length HTTP header.\",\"Code\":0,\"Trace\":\"#0 \\\/var\\\/www\\\/nextcloud\\\/apps\\\/files_external\\\/3rdparty\\\/aws-sdk-php\\\/Aws\\\/Common\\\/Exception\\\/NamespaceExceptionFactory.php(76): Aws\\\\Common\\\\Exception\\\\NamespaceExceptionFactory->createException('Aws\\\\\\\\S3...

Searching for the "You must provide the Content-Length HTTP header" error reveals a bug in the AWS SDK due to an old version of Guzzle:

aws/aws-sdk-php#497

So I guess unless the files_external app is updated to a new AWS SDK version, it won't work :(

@noplanman
Copy link

@noplanman noplanman commented Dec 10, 2017

Any progress on this? Running into this too...

@610th
Copy link

@610th 610th commented Apr 22, 2019

Seems like this is still an issue. The proposed solution worked for me, but I figure this certainly isn't desired behavior.

@nextcloud-stale nextcloud-stale bot removed the stale label Jun 12, 2019
@virtualdxs
Copy link

@virtualdxs virtualdxs commented Nov 2, 2019

I was able to get my instance (mostly) working by using this solution, but it really shouldn't be necessary. I'd really appreciate more documentation in general on using object storage as primary in Nextcloud.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
10 participants