SFTP External storage doesn't work

[Turgon37]

Expected behaviour

I want to browse my SFTP External Storage on owncloud server

Actual behaviour

Android app says 'Nothing here, send something'
But all other ownCloud client (Linux Desktop and WebUI) work fine

Steps to reproduce

1. Configure ldap_backend on owncloud server
2. Configure an external remote share with SFTP protocol
3. Browse to the external share folder with Android App

Can this problem be reproduced with the official owncloud server?
no because it need an external mount storage with SFTP and LDAP user backend

Environment data

Android version:
6.0.1

Device model: Samsung Galaxy S6 SM G920F

Stock or customized system: Stock

ownCloud app version: 2.0.0

ownCloud server version: ownCloud 9.0.2 (stable)

Logs

Web server error log

nginx_owncloud: 192.168.57.81 - pgindraud [15/May/2016:11:27:17 +0200] "GET /ocs/v1.php/cloud/capabilities?format=json HTTP/1.1" 200 607 "-" "Mozilla/5.0 (Android) ownCloud-android/2.0.0"
nginx_owncloud: 192.168.57.81 - pgindraud [15/May/2016:11:27:17 +0200] "PROPFIND /remote.php/webdav/LINUX_HOME/ HTTP/1.1" 503 234 "-" "Mozilla/5.0 (Android) ownCloud-android/2.0.0"

ownCloud log (data/owncloud.log)

ownCloud[9730]: {webdav} Exception: {
"Message":"HTTP/1.1 503 Storage not available",
"Exception":"Sabre\DAV\Exception\ServiceUnavailable",
"Code":0,
"Trace":"
#0 /var/www/owncloud/3rdparty/sabre/dav/lib/DAV/Server.php(903): OCA\DAV\Connector\Sabre\ObjectTree->getNodeForPath('LINUX_HOME')
#1 /var/www/owncloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php(334): Sabre\DAV\Server->getPropertiesForPath('LINUX_HOME', Array, 0)
#2 [internal function]: Sabre\DAV\CorePlugin->httpPropFind(Object(Sabre\HTTP\Request), Object(Sabre\HTTP\Response))
#3 /var/www/owncloud/3rdparty/sabre/event/lib/EventEmitterTrait.php(105): call_user_func_array(Array, Array)
#4 /var/www/owncloud/3rdparty/sabre/dav/lib/DAV/Server.php(459): Sabre\Event\EventEmitter->emit('method:PROPFIND', Array)
#5 /var/www/owncloud/3rdparty/sabre/dav/lib/DAV/Server.php(248): Sabre\DAV\Server->invokeMethod(Object(Sabre\HTTP\Request), Object(Sabre\HTTP\Response))
#6 /var/www/owncloud/apps/dav/appinfo/v1/webdav.php(55): Sabre\DAV\Server->exec()
#7 /var/www/owncloud/remote.php(138): require_once('/var/www/ownclo...')
#8 {main}","File":"/var/www/owncloud/apps/dav/lib/connector/sabre/objecttree.php","Line":159,"User":"ae877d96-3ba9-1035-95bb-6fbf9d048cee"}

[Turgon37]

It appear that WebUi and Desktop client make PROPFIND query with no ending slash after the directory name. Like PROPFIND /remote.php/webdav/LINUX_HOME
But the Android App make the same query with an ending slash like PROPFIND /remote.php/webdav/LINUX_HOME/ and this, make a 503 error.

[QuentinC]

Hello,

I have the same problem with a SMB share.
When I connect the WebDAV on a Windows client it works great. But not with the Android app.

I've tried to remove the trailing / using a reverse proxy, but it still make 503 errors...

Here is the error:
{"reqId":"0ScgJgdboyJsp67fi3Cn","remoteAddr":"192.168.1.20","app":"webdav","message":"Exception: {\"Message\":\"HTTP\\\/1.1 503 Storage not available\",\"Exception\":\"Sabre\\\\DAV\\\\Exception\\\\ServiceUnavailable\",\"Code\":0,\"Trace\":\"#0 \\\/var\\\/www\\\/cloud\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(903): OCA\\\\DAV\\\\Connector\\\\Sabre\\\\ObjectTree->getNodeForPath('Partages')\\n#1 \\\/var\\\/www\\\/cloud\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/CorePlugin.php(334): Sabre\\\\DAV\\\\Server->getPropertiesForPath('Partages', Array, 0)\\n#2 [internal function]: Sabre\\\\DAV\\\\CorePlugin->httpPropFind(Object(Sabre\\\\HTTP\\\\Request), Object(Sabre\\\\HTTP\\\\Response))\\n#3 \\\/var\\\/www\\\/cloud\\\/3rdparty\\\/sabre\\\/event\\\/lib\\\/EventEmitterTrait.php(105): call_user_func_array(Array, Array)\\n#4 \\\/var\\\/www\\\/cloud\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(459): Sabre\\\\Event\\\\EventEmitter->emit('method:PROPFIND', Array)\\n#5 \\\/var\\\/www\\\/cloud\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(248): Sabre\\\\DAV\\\\Server->invokeMethod(Object(Sabre\\\\HTTP\\\\Request), Object(Sabre\\\\HTTP\\\\Response))\\n#6 \\\/var\\\/www\\\/cloud\\\/apps\\\/dav\\\/appinfo\\\/v1\\\/webdav.php(55): Sabre\\\\DAV\\\\Server->exec()\\n#7 \\\/var\\\/www\\\/cloud\\\/remote.php(138): require_once('\\\/var\\\/www\\\/cloud\\\/...')\\n#8 {main}\",\"File\":\"\\\/var\\\/www\\\/cloud\\\/apps\\\/dav\\\/lib\\\/connector\\\/sabre\\\/objecttree.php\",\"Line\":159,\"User\":\"quentin\"}","level":4,"time":"2016-05-16T20:57:55+00:00","method":"PROPFIND","url":"\/remote.php\/webdav\/Partages","user":"quentin"}

And the corresponding apache log:
192.168.1.20 - quentin [16/May/2016:23:20:30 +0200] "PROPFIND /remote.php/webdav/Partages HTTP/1.1" 503 1277 "-" "Mozilla/5.0 (Android) ownCloud-android/20160427"

I'm trying here to access a "Partages" share on a Windows server (using LDAP authentication against an Active Directory Server). It works perfectly on the web UI (even in a mobile browser) and on the Windows WebDAV client.

Thanks !

[QuentinC]

Hello,

I have some news on this !
I was wondering why this were working on Windows via the DAV client but not on Android app...

So I used a man-in-the-middle proxy, and I have found that the Android app never sends cookies where Windows does.

I can see two cookies: Cookie:

• ocp8t021zp1q
• oc_sessionPassphrase

If I add these two cookies to the request, the answer is correct !

Thanks !

[infratactix]

I can confirm the same.

Why does cookies not pass in mobile app? It also seems to be an issue with 3rd party OC apps as well for android.

Does it happen on IOS?

[davivel]

Android app doesn't use session cookies. It sends user and password in every request.

Is there any reason why accessing SFTP external storage with user and password shouldn't work?

[davivel]

@PVince81 , I vaguely remember there were some discussions "recently" about bugs related with sending user and password instead of session in OCS paths. I'm not sure if something similar could affect external storage. Do you recall any of this?

[PVince81]

@davivel that shouldn't make any difference.

My steps:

1. Setup OC v9.0.2
2. Mount SFTP storage with username/password and specified root, mount as "/sftp"
3. curl -u admin -X PROPFIND http://localhost/owncloud/remote.php/webdav/sftp => works
4. curl -u admin -X PROPFIND http://localhost/owncloud/remote.php/webdav/sftp/ => works
5. With Android client => works

And the log shows me that the requests did have a trailing slash and worked fine:

192.168.1.35 - admin [15/Jul/2016:10:28:29 +0200] "PROPFIND /owncloud/remote.php/webdav/sftp/ HTTP/1.1" 207 810 "-" "Mozilla/5.0 (Android) ownCloud-android/2.0.1"
192.168.1.35 - admin [15/Jul/2016:10:28:30 +0200] "PROPFIND /owncloud/remote.php/webdav/sftp/ HTTP/1.1" 207 5480 "-" "Mozilla/5.0 (Android) ownCloud-android/2.0.1"

Please double check your setup, maybe something else (reverse proxy?) is messing with the requests ?

Could you guys try with the similar curl commands ?

[QuentinC]

Hello,

Are you trying this using "use session credentials" ?

Here is what I get when trying:

curl -u quentin -X PROPFIND https://cloud.o2r.fr/remote.php/webdav/Home

<d:error xmlns:d="DAV:" xmlns:s="http://sabredav.org/ns">
<s:exception>Sabre\DAV\Exception\ServiceUnavailable/s:exception
<s:message>Storage not available/s:message
/d:error

(Exactly the same with the trailing / )

Apache2 logs:
192.168.1.20 - quentin [16/Jul/2016:21:38:38 +0200] "PROPFIND /remote.php/webdav/Home HTTP/1.1" 503 1213 "-" "curl/7.38.0"
192.168.1.20 - quentin [16/Jul/2016:21:40:43 +0200] "PROPFIND /remote.php/webdav/Home/ HTTP/1.1" 503 1209 "-" "curl/7.38.0"

[PVince81]

@QuentinC try the following SQL query: update oc_storages set available=1;

Then retry with curl and check your "owncloud.log" (not apache log)

[QuentinC]

@PVince81 it's exactly the same (en fact, all storages where already with "available=1" in the SQL DB...)

Here is the owncloud log:

{"reqId":"inYdEssDOb/XFi7MjDKj",
"remoteAddr":"192.168.1.20",
"app":"webdav",
"message":"Exception:

{"Message":"HTTP/1.1 503 Storage not available",
"Exception":"Sabre DAV Exception Service Unavailable",
"Code":0,"Trace":
#0 /var/www/cloud/3rdparty/sabre/dav/lib/DAV/Server.php(903): OCA\DAV\Connector\Sabre\ObjectTree->getNodeForPath('Home')
#1 /var/www/cloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php(334): Sabre\DAV\Server->getPropertiesForPath('Home', Array, 1)
#2 [internal function]: Sabre\DAV\CorePlugin->httpPropFind(Object(Sabre\HTTP\Request), Object(Sabre\HTTP\Response))
#3 /var/www/cloud/3rdparty/sabre/event/lib/EventEmitterTrait.php(105): call_user_func_array(Array, Array)
#4 /var/www/cloud/3rdparty/sabre/dav/lib/DAV/Server.php(459): Sabre\Event\EventEmitter->emit('method:PROPFIND', Array)
#5 /var/www/cloud/3rdparty/sabre/dav/lib/DAV/Server.php(248): Sabre\DAV\Server->invokeMethod(Object(Sabre\HTTP\Request), Object(Sabre\HTTP\Response))
#6 /var/www/cloud/apps/dav/appinfo/v1/webdav.php(56): Sabre\DAV\Server->exec()
#7 /var/www/cloud/remote.php(164): require_once('/var/www/cloud/...')
#8 {main}

"File":
/var/www/cloud/apps/dav/lib/Connector/Sabre/ObjectTree.php
,"Line":159,
"User":"quentin" }

"level":4,
"time":"2016-07-18T08:19:42+00:00",
"method":"PROPFIND",
"url":"/remote.php/webdav/Home",
"user":"quentin"}`

[PVince81]

@QuentinC are you able to access it over the web UI ? Originally this bug was only about Android/curl, but I don't see any reason why it would work in the web UI. It's either it works everywhere or not at all.

[QuentinC]

@PVince81 yes, it's working great on the web UI, and also using the Windows WebDAV client !
Using an HTTP proxy, the only difference I've found on the request is the cookies sent. Windows client send cookies, but not Android (and neither does curl)...

[PVince81]

Can you give more details about your SFTP connection ?
Maybe a screenshot of the settings page would help.

I suspect that you might be using "use session credentials" mode which might explain the different results.

[QuentinC]

@PVince81
Yes, I'm using "use session credentials" !
Here are the storage's definition:

[QuentinC]

@PVince81 If I set an external storage with fixed user and password (ie instead of session credentials), it works everywhere !

[PVince81]

Ok, thanks. Then this is a server error.

Even if there are no cookies, I expect a single request that does authenticate to also have the credentials stored in the temporary session.

[PVince81]

Closing this and raised another ticket in core to address this: owncloud/core#25511

[PVince81]

@QuentinC I still cannot reproduce this with 9.0.3 and session mode. Curl works fine.

[QuentinC]

I have the problem mainly on a SMB share. But, to be sure, I've set up a sftp share, using session credentials, and curl won't work...

I'm using the version ownCloud 9.1.0 RC1 (beta) - latest daily build.

[PVince81]

I just tried on the stable9.1 branch which is more recent, also works.

@QuentinC can you try with an empty sftp storage folder ? Just in case there are files in yours that would cause this as side effect.

[QuentinC]

Exactly the same...
Are you using LDAP authentication ?

[PVince81]

I haven't, will try that soon... having trouble with the LDAP testing docker

[QuentinC]

Well, could be the problem...
I've setup a local account on ownCloud and on my sftp box, and it works (even when using session credentials...)

[PVince81]

Am now back on v9.0.3 and it looks even more broken. Will continue investigating in owncloud/core#25511