9.1.0: realpath error messages in OC log although file uploads succeed #26033

Closed
timreeves opened this Issue Sep 6, 2016 · 12 comments

Projects

None yet

2 participants

@timreeves

Steps to reproduce

  1. Install OC 9.1.0 on Server
  2. Install latest Mirall on Windows-PC
  3. Sync some files by adding/editing on your PC so Mirall uploads them to server

Expected behaviour

Should succeed with no OC log messages

Actual behaviour

Succeeds, but for each file uploaded a message like this appears in the OC log:

{"reqId":"...","remoteAddr":"{my-home-ip}","app":"PHP","message":
"realpath(): open_basedir restriction in effect. File(/) is not within the allowed path(s):
(/var/www/vhosts/{domain}/:/tmp/:/dev/urandom:/var/run/redis:/home/mybackup)
at /var/www/vhosts/{domain}/{subdomain}/lib/private/Files/Storage/Local.php#56",
"level":3,"time":"2016-09-06T07:40:45+00:00","method":"PUT",
"url":"/remote.php/webdav/Zed/Server/ownCloud/OC-910-Manual-Fixes.txt","user":"{me}"}

The code line at Local.php#56 is this:

$this->realDataDir = rtrim(realpath($this->datadir), '/') . '/';

If anyone can suggest a debug statement (how to log what is actually in $this->datadir, and even better to output a call stack, then I'll gladly whack it in and post the test results.

I would really appreciate quick support, as I have clients wanting to upgrade and I go on holiday soon!

Server configuration

Operating system
Ubuntu 14.04.5 LTS

Web server:
nginx 1.11.1 (as installed by Plesk)

Database:
mysql

PHP version:
7.0.10 (self compiled)

ownCloud version: 9.1.0

Updated from an older ownCloud or fresh install:
Updated

Where did you install ownCloud from:
https://owncloud.org/install/ From Source-Zip, via CLI

Signing status (ownCloud 9.0 and above):
No errors have been found.

List of activated apps:
Enabled:

  • activity: 2.2.1
  • calendar: 1.3.3
  • contacts: 1.3.1.0
  • dav: 0.1.6
  • federatedfilesharing: 0.1.0
  • files: 1.4.4
  • files_external: 0.5.2
  • files_trashbin: 0.8.0
  • notifications: 0.2.3
  • updatenotification: 0.1.0

Disabled:

  • comments
  • encryption
  • external
  • federation
  • files_pdfviewer
  • files_sharing
  • files_texteditor
  • files_versions
  • files_videoplayer
  • firstrunwizard
  • gallery
  • provisioning_api
  • systemtags
  • templateeditor
  • user_external
  • user_ldap

The content of config/config.php:
{
"system": {
"instanceid": "oc452bf87cea",
"passwordsalt": "_REMOVED SENSITIVE VALUE",
"trusted_domains": [
"owncloud.biblische-reisen.de"
],
"datadirectory": "/var/www/vhosts/biblische-reisen.de/oc-data",
"dbtype": "mysql",
"version": "9.0.4.1",
"dbname": "owncloudbde",
"dbhost": "localhost",
"dbtableprefix": "oc
",
"dbuser": "_REMOVED SENSITIVE VALUE",
"dbpassword": "_REMOVED SENSITIVE VALUE
",
"installed": true,
"loglevel": 4,
"forcessl": true,
"theme": "",
"maintenance": false,
"mail_from_address": "owncloud",
"mail_domain": "biblische-reisen.de",
"mail_smtpmode": "php",
"mail_smtpname": "_REMOVED SENSITIVE VALUE",
"mail_smtppassword": "_REMOVED SENSITIVE VALUE
",
"default_language": "en",
"secret": "_REMOVED SENSITIVE VALUE",
"log_rotate_size": 10485760,
"check_for_working_wellknown_setup": false,
"check_for_working_htaccess": false,
"memcache.local": "\OC\Memcache\APCu",
"filelocking.enabled": "true",
"memcache.locking": "\OC\Memcache\Redis",
"redis": {
"host": "/var/run/redis/redis.sock",
"port": 0,
"timeout": 0,
"dbindex": 1
},
"trashbin_retention_obligation": "auto",
"updater.secret": "_REMOVED SENSITIVE VALUE
"
}
}

Are you using external storage
No

Are you using encryption:
No

Are you using an external user-backend, if yes which one:
No

Client configuration

Browser:
Firefox

Operating system:
Windows 10

Logs

Web server error log

Log is empty.

ownCloud log (data/owncloud.log)

{"reqId":"d4q4S7a4Og+Foxu7gAKe","remoteAddr":"188.192.17.159","app":"PHP","message":
"realpath(): open_basedir restriction in effect. File(/) is not within the allowed path(s):
 (/var/www/vhosts/reeves.one/:/tmp/:/dev/urandom:/var/run/redis:/home/timbackup)
 at /var/www/vhosts/reeves.one/owncloud.reeves.one/lib/private/Files/Storage/Local.php#56",
 "level":3,"time":"2016-09-06T07:40:45+00:00","method":"PUT",
 "url":"/remote.php/webdav/Zed/Server/ownCloud/OC-910-Manual-Fixes.txt","user":"TimReeves"}

Browser log

Irrelevant, problem is at server. And not using browser but Mirall.
@timreeves

HA - the "Local" Storage Constructor IS occassionally being called with "/" as argument!

I have added a debug_trace() as follows to /lib/private/Files/Storage/Local.php:

protected function getPhpStackTraceString()
{
    $arrTrace = debug_backtrace();
    $strTrace = ''; $strSep = '';

    foreach ($arrTrace as $arrCall) {
        if ($arrCall['function'] == 'getPhpStackTraceString') continue;
        $strFile = '...' . substr($arrCall['file'], 20);
        $strCall = $strFile . '#' . $arrCall['line'] . ' ';
        // if ($arrCall['class'] != '')
        //  $strCall .= "{$arrCall['class']}{$arrCall['type']}{$arrCall['function']}(";
        // else
            $strCall .= $arrCall['function'] . '(';
        $strSep2 = '';
        foreach ($arrCall['args'] as $mixArg) {
            $strCall .= $strSep2;
            $strArgType = gettype($mixArg);
            switch ($strArgType) {
                case "boolean":
                    $strCall .= $mixArg ? 'true' : 'false';
                    break;
                case "integer":
                case "double":
                case "string":
                    $strCall .= $mixArg;
                    break;
                default:    // "array", "object", "resource", "NULL", "unknown type"
                    $strCall .= '{' . $strArgType . '}';
                    break;
            }
            $strSep2 = ', ';
        }
        $strCall .= ')';
        $strTrace = $strCall . $strSep . $strTrace;
        $strSep = ' | ';
    }

    return $strTrace;

}   // getPhpStackTraceString()


    public function __construct($arguments) {
        if (!isset($arguments['datadir']) || !is_string($arguments['datadir'])) {
            throw new \InvalidArgumentException('No data directory set for local storage');
        }
        if ($arguments['datadir'] == '/') {
            file_put_contents('/tmp/ocdebug', $arguments['datadir'] . "\n" . $this->getPhpStackTraceString() . "\n\n", FILE_APPEND);
        }

and got this output - once for each file upload from ownCloud Client Version 2.2.3 (build 6307):

/
.../apps/dav/appinfo/v1/webdav.php#56 exec()
.../3rdparty/sabre/dav/lib/DAV/Server.php#248 invokeMethod({object}, {object})
.../3rdparty/sabre/dav/lib/DAV/Server.php#459 emit(method:PUT, {array})
.../3rdparty/sabre/event/lib/EventEmitterTrait.php#105 call_user_func_array({array}, {array})
...# httpPut({object}, {object})
.../3rdparty/sabre/dav/lib/DAV/CorePlugin.php#511 updateFile(Zed/Server/ownCloud/OC-910-Manual-Fixes.txt, {resource},{NULL})
.../3rdparty/sabre/dav/lib/DAV/Server.php#1070 put({resource})
.../apps/dav/lib/Connector/Sabre/File.php#163 emitPreHooks(true)
.../apps/dav/lib/Connector/Sabre/File.php#267 emit(OC_Filesystem, write, {array})
.../lib/private/legacy/hook.php#105 write_hook({array})
.../apps/files_versions/lib/Hooks.php#60 store(/Zed/Server/ownCloud/OC-910-Manual-Fixes.txt)
.../apps/files_versions/lib/Storage.php#192 copy(/files/Zed/Server/ownCloud/OC-910-Manual-Fixes.txt, /files_versions/Zed/Server/ownCloud/OC-910-Manual-Fixes.txt.v1473151129)
.../lib/private/Files/View.php#887 copy(files/Zed/Server/ownCloud/OC-910-Manual-Fixes.txt, files_versions/Zed/Server/ownCloud/OC-910-Manual-Fixes.txt.v1473151129)
.../lib/private/Files/Storage/Wrapper/Wrapper.php#282 copy(files/Zed/Server/ownCloud/OC-910-Manual-Fixes.txt, files_versions/Zed/Server/ownCloud/OC-910-Manual-Fixes.txt.v1473151129)
.../lib/private/Files/Storage/Wrapper/Encryption.php#351 copyFromStorage({object}, files/Zed/Server/ownCloud/OC-910-Manual-Fixes.txt, files_versions/Zed/Server/ownCloud/OC-910-Manual-Fixes.txt.v1473151129)
.../lib/private/Files/Storage/Wrapper/Encryption.php#641 copyBetweenStorage({object}, files/Zed/Server/ownCloud/OC-910-Manual-Fixes.txt, files_versions/Zed/Server/ownCloud/OC-910-Manual-Fixes.txt.v1473151129, false, false)
.../lib/private/Files/Storage/Wrapper/Encryption.php#702 copyFromStorage({object}, files/Zed/Server/ownCloud/OC-910-Manual-Fixes.txt, files_versions/Zed/Server/ownCloud/OC-910-Manual-Fixes.txt.v1473151129)
.../lib/private/Files/Storage/Local.php#457 __construct({array}

I'm not familiar with the SW architecture of OC - can someone who is take a shot at what the cause may be? I'll gladly do more debugging if you tell me what to do.

Tim

@timreeves

Hey guys - this is obviously a real bug - no takers?

I am not the only one to observe this problem; a guy reported it on central a few weeks ago.

So I have done more debugging, here my observations:

  • It does NOT happen with the initial upload of files added to a sync area.
  • But when I EDIT a file which has already been synchronised THEN it happens. This is independent of whether the file has recently been added or has been in the sync process for a long time (i.e. before the recent upgrade of OC).

Hope this helps!

@PVince81
Collaborator
PVince81 commented Sep 7, 2016

this is obviously a real bug - no takers

was busy with other critical bugs, sorry...

Thanks for debugging. Usually I use this kind of statement:

        \OC::$server->getLogger()->debug('some text blah var=' . $somevar, array('app' => 'DEBUG'));

I'll see if I can reproduce your issue. The Local storage should never be initialized with the root, must be a bug.

@PVince81 PVince81 added the bug label Sep 7, 2016
@PVince81
Collaborator
PVince81 commented Sep 7, 2016

I would really appreciate quick support, as I have clients

also note that Github is an issue tracker, not a support platform. See http://owncloud.org/support

@timreeves

@PVince81 Thanks Vincent for taking this up - and of course you are right, this is an issue tracker - it's just sometimes easy to not think straight when one is tired, things are not going right and time is tight. Still you are right - sorry and Grüße nach Stuttgart!

That my debug method was crude - yup, sure was, but it worked without me having to know details of what OC offers in that area.

@PVince81
Collaborator
PVince81 commented Sep 7, 2016

it's just sometimes easy to not think straight when one is tired

That, I fully understand 😉

Okay I got something locally and also I finally learnt what "open_basedir" is:

  1. Set "open_basedir" to "/srv/www" in php.ini
  2. Set "sys_temp_dir" to "/srv/www/tmp" in php.ini to avoid additional complains, make sure to create the dir
  3. Setup OC
  4. Using cadaver (Webdav): put a new file "test.txt"
  5. Reupload the same file to overwrite "test.txt"

Getting this:

{"reqId":"UTAA2JtzPIcYbBpaV026","remoteAddr":"127.0.0.1","app":"PHP","message":"realpath(): open_basedir restriction in effect. File(\/) is not within the allowed path(s): (\/srv\/www\/) at \/srv\/www\/htdocs\/owncloud\/lib\/private\/Files\/Storage\/Local.php#56","level":3,"time":"2016-09-07T10:18:52+00:00","method":"PUT","url":"\/owncloud\/remote.php\/files\/test.txt","user":"admin"}

This is indeed suspicious, will debug in detail.

@PVince81
Collaborator
PVince81 commented Sep 7, 2016

Stack:

0  OC\Files\Storage\Local->__construct() /srv/www/htdocs/owncloud/lib/private/Files/Storage/Local.php:56
1  OC\Files\Storage\Local->copyFromStorage() /srv/www/htdocs/owncloud/lib/private/Files/Storage/Local.php:412
2  OC\Files\Storage\Wrapper\Encryption->copyBetweenStorage() /srv/www/htdocs/owncloud/lib/private/Files/Storage/Wrapper/Encryption.php:702
3  OC\Files\Storage\Wrapper\Encryption->copyFromStorage() /srv/www/htdocs/owncloud/lib/private/Files/Storage/Wrapper/Encryption.php:641
4  OC\Files\Storage\Wrapper\Encryption->copy() /srv/www/htdocs/owncloud/lib/private/Files/Storage/Wrapper/Encryption.php:351
5  OC\Files\Storage\Wrapper\Wrapper->copy() /srv/www/htdocs/owncloud/lib/private/Files/Storage/Wrapper/Wrapper.php:282
6  OC\Files\View->copy() /srv/www/htdocs/owncloud/lib/private/Files/View.php:887
7  OCA\Files_Versions\Storage::store() /srv/www/htdocs/owncloud/apps/files_versions/lib/Storage.php:192
8  OCA\Files_Versions\Hooks::write_hook() /srv/www/htdocs/owncloud/apps/files_versions/lib/Hooks.php:60
9  call_user_func:{/srv/www/htdocs/owncloud/lib/private/legacy/hook.php:105}() /srv/www/htdocs/owncloud/lib/private/legacy/hook.php:105
10 OC_Hook::emit() /srv/www/htdocs/owncloud/lib/private/legacy/hook.php:105
11 OCA\DAV\Connector\Sabre\File->emitPreHooks() /srv/www/htdocs/owncloud/apps/dav/lib/Connector/Sabre/File.php:268
12 OCA\DAV\Connector\Sabre\File->put() /srv/www/htdocs/owncloud/apps/dav/lib/Connector/Sabre/File.php:163
13 Sabre\DAV\Server->updateFile() /srv/www/htdocs/owncloud/3rdparty/sabre/dav/lib/DAV/Server.php:1070
14 Sabre\DAV\CorePlugin->httpPut() /srv/www/htdocs/owncloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php:511
15 call_user_func_array:{/srv/www/htdocs/owncloud/3rdparty/sabre/event/lib/EventEmitterTrait.php:105}() /srv/www/htdocs/owncloud/3rdparty/sabre/event/lib/EventEmitterTrait.php:105
16 Sabre\Event\EventEmitter->emit() /srv/www/htdocs/owncloud/3rdparty/sabre/event/lib/EventEmitterTrait.php:105
17 Sabre\DAV\Server->invokeMethod() /srv/www/htdocs/owncloud/3rdparty/sabre/dav/lib/DAV/Server.php:459
18 Sabre\DAV\Server->exec() /srv/www/htdocs/owncloud/3rdparty/sabre/dav/lib/DAV/Server.php:248
19 require_once()  /srv/www/htdocs/owncloud/apps/dav/appinfo/v1/webdav.php:56
20 {main}          /srv/www/htdocs/owncloud/remote.php:164

Args given to "Local":

  $arguments['datadir']          = (string[1]) '/';
@PVince81
Collaborator
PVince81 commented Sep 7, 2016

Wha... ?!

https://github.com/owncloud/core/blob/v9.1.0/lib/private/Files/Storage/Local.php#L412

    public function copyFromStorage(\OCP\Files\Storage $sourceStorage, $sourceInternalPath, $targetInternalPath) {
        if ($sourceStorage->instanceOfStorage('\OC\Files\Storage\Local')) {
            /**
             * @var \OC\Files\Storage\Local $sourceStorage
             */
            $rootStorage = new Local(['datadir' => '/']);
            return $rootStorage->copy($sourceStorage->getSourcePath($sourceInternalPath), $this->getSourcePath($targetInternalPath));
        } else {
            return parent::copyFromStorage($sourceStorage, $sourceInternalPath, $targetInternalPath);
        }
    }
@PVince81
Collaborator
PVince81 commented Sep 7, 2016

Wow, this is old: 31e9470 from #13360

So even OC 8.1 might be affected.

@PVince81
Collaborator
PVince81 commented Sep 7, 2016

My first thought was: changing the path to be the data dir. Doesn't work directly. There might be ways to make it work.
However, I think the reason this code looks like this is to accommodate for the "local" external storage which can also be outside the data directory, say in "/somewhere"

@PVince81 PVince81 self-assigned this Sep 7, 2016
@PVince81 PVince81 referenced this issue Sep 7, 2016
Merged

Only use realpath for real directories #26058

5 of 10 tasks complete
@PVince81
Collaborator
PVince81 commented Sep 7, 2016

Ok, I got a fix to simply get rid of the useless warning: #26058

Ideally we should find a way to get rid of this root storage but it might be more complicated.

@PVince81 PVince81 added this to the 9.1.1 milestone Sep 7, 2016
@PVince81 PVince81 added the sev2-high label Sep 7, 2016
@PVince81 PVince81 referenced this issue in owncloud/QA Sep 7, 2016
Open

Tests with open_basedir #302

@timreeves

Thanks Vincent - alles klar!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment