Nextcloud app doesn't show all files and directories on an external storage #468

j-ed · 2016-12-22T14:51:40Z

Actual behaviour

The Nextcloud app doesn't show all available files and directories of a user, if these have been mounted in via a WebDAV share (the problem may exist for other external storages too). In the Nextcloud app some of the external storage links are shown as empty and a request to upload files is displayed.

Root of nextcloud user:

Sub-directory doesn't show any content:

All files and directories can be seen and accessed using the Nextcloud Web GUI.

I did a cross-check with the iOS ownCloud app v3.5.2 and all files and directories are instantly shown in the app so that it seems to be a problem of the Nextcloud app itself.

Expected behaviour

The Nextcloud app should show all available files and directories of a user, including files which are mounted in via a WebDAV share.

Log files

No related errors are logged in the nextcloud log file.

Environment data

Android version: 5.1.1
Stock or customized system: stock

Nextcloud app version: 1.4.0
Nextcloud: 10.0.2

PR: nextcloud/android-library#70

tobiasKaminsky · 2017-06-26T12:27:30Z

@icewind1991 have you heard about this bug? I always thought that all external storages are transparent for the android client and all is handled by the server...

tobiasKaminsky · 2017-06-27T08:36:46Z

I tried this with a local mount, e.g. mount /tmp in nextcloud and it is shown in webUI and in android app.

exmoorone · 2017-06-27T09:25:21Z

I did see this same problem on my first day using nextcloud and started watching this bug.
A day or 2 later it was working for me and has been since. Not sure if there is any way the external shares could not be correctly populated into the list of directories on a brand new installation for some reason, or could not refresh properly after the shares are added or something. Is this done by any background processing that might take some time? Based on the above information this would require there to be some difference in the mechanism between the android and iOS apps getting the directory list.

Either that or there was an update to the android app at just the time I started using it which fixed this.

j-ed · 2017-06-27T10:00:08Z

I also tried to find the root cause of the problem, and e.g. re-scanned all files using the occ-command etc.
But due to the fact that all files and directories can been seen and accessed using the Nextcloud web GUI, using the ownCloud iPhone app v3.6.2 (extremely fast), using the Nextcloud desktop app 2.3.1.8 or using the ES File Explorer Android app (WebDAV, relatively slow) it seems to be a Nextcloud Android app specific issue.
I don't know what other information I can provide to narrow down the problem. If you want you can send me a specific debug version of the Nextcloud app to collect more data. Unfortunately the current app seems not to have any kind of debug trace function included to do the job.

tobiasKaminsky · 2017-06-30T08:14:30Z

@j-ed I tried it now with mounting another nextcloud via https://nc.server.com/remote.php/webdav/ to the root folder.
I then could successfully view, browse and download the content of this folder, even with 3 subfolders.

You can try our dev version https://github.com/nextcloud/android/blob/master/CONTRIBUTING.md#dev-release

tobiasKaminsky · 2017-07-31T14:26:53Z

No response within the last month, therefore closing.
Please feel free to reopen with additional infos, if the problem still exists.

bpcurse · 2017-08-07T14:27:00Z

The problem still persists. I can confirm this behaviour on Android app 1.4.3 and NC Server 12.0.0.
Server ist connected to an external WebDAV Storage. Everything works fine in the web interface and the Linux client syncs all data.
In the android app the directory is visible and can be accessed, but it displays no content. Upload via Android app is possible and the uploaded data stays visible until you close and open the app again. This uploaded data (by the android app) remains on the server and can be accessed via web and linux sync client, but not in the android app.
EDIT: Same problem with the dev version 20170807
Android ver: 6.0.1 (Blackberry DTEK50)

tobiasKaminsky · 2017-08-08T07:24:01Z

@MorrisJobke do you have an idea here?
Especially "upload works, but listing files fails" is quite mysterious for me :/

MorrisJobke · 2017-08-08T09:10:55Z

Especially "upload works, but listing files fails" is quite mysterious for me :/

Could it be, that the request runs into a timeout? I have seen this recently (especially after playing with the brute force detection) and the request returned fine, but the Android app didn't take care of the request.

@bpcurse Could you run this with the folder that doesn't work in your scenario:

time curl -X PROPFIND https://YOURSERVER.COM/remote.php/webdav/PATH/TO/NONLOADING/FOLDer -u USERNAME:PASSWORD

bpcurse · 2017-08-08T10:46:58Z

Another info by the way, the android app shows and updates the size of the folder correctly, without listing the files. And i'm on a slow vpn connection, will try it again while inside our university network.
Working on the time curl...

bpcurse · 2017-08-08T12:00:44Z

More debugging info (still on the vpn connection):
On the same Android device the connection is possible via Total Commander WebDAV Plugin, but it takes quite a while.
NC Server is now running ver 12.0.1 (stable) (shared webhosting), using app v1.4.3

On Manjaro Linux:
time curl -X PROPFIND https://xxxxxxxx/remote.php/dav/files/xxxxxxx/FS_E-Mail/ -u xxxxx:xxxxx
responds as follows:

<d:error xmlns:d="DAV:" xmlns:s="http://sabredav.org/ns">
<s:exception>Sabre\DAV\Exception\NotAuthenticated</s:exception>
<s:message>No public access to this resource., Username or password was incorrect, No 'Authorization: Bearer' header found. Either the client didn't send one, or the server is mis-configured, Username or password was incorrect</s:message>
</d:error>

real 0m30,672s
user 0m0,050s
sys 0m0,000s

The DAV path and the supplied credentials are correct. Both tested on Linux Thunar with "davs://" and on the Android Total Commander WebDAV Plugin -> working.
The same happens with another folder that is accessible from within the app.

However the WebDAV Resource itself is accessible (independent server, not the NC server):
time curl -X PROPFIND -u xxxxxxx:xxxxxxx https://xxxxxxxxx/mail/zu_besprechen

<title>301 Moved Permanently</title>

Moved Permanently

The document has moved here.

real 0m1,985s
user 0m0,043s
sys 0m0,003s

Sorry, that I couldn't supply the information via time curl and thank you very much for looking after this!

MorrisJobke · 2017-08-08T12:20:42Z

real 0m30,672s

This indicates, that the brute force protection hit in. Have you changed recently your password? And also the error message says: "your password is wrong". Could you check the access log for the user agent "android" to see what responses are returned there.

To clear the brute force problem: go to the database and drop all entries with your IP in the table "oc_bruteforce_attempts".

bpcurse · 2017-08-08T14:13:59Z

Thank you for the brute-force hint.
Looks like i could identify the issue by "try and error" or at least narrow it down:

As soon as our WebDAV server creates a hidden file (.something) in the directory, the android client doesn't show any newly added files, the Linux sync client and the web interface still seem to work fine.
Deleting the hidden file from the directory makes the Android client show all files again.

Is this a bug in the Android client?

time curl not working was my own fault. When trying the curl command I forgot to escape special characters in my password e.g. "\$". That caused the wrong password / brute-force issue.

Here's my time curl response (without the hidden file):

<d:multistatus xmlns:d="DAV:" xmlns:s="http://sabredav.org/ns" xmlns:cal="urn:ietf:params:xml:ns:caldav" xmlns:cs="http://calendarserver.org/ns/" xmlns:card="urn:ietf:params:xml:ns:carddav" xmlns:oc="http://owncloud.org/ns" xmlns:nc="http://nextcloud.org/ns"><d:response><d:href>/remote.php/dav/files/xxxxxxx/FS_E-Mail/</d:href><d:propstat><d:prop><d:getlastmodified>Tue, 08 Aug 2017 14:07:16 GMT</d:getlastmodified><d:resourcetype><d:collection/></d:resourcetype><d:quota-used-bytes>25</d:quota-used-bytes><d:quota-available-bytes>-3</d:quota-available-bytes><d:getetag>"5989c5941f3d5"</d:getetag></d:prop><d:status>HTTP/1.1 200 OK</d:status></d:propstat></d:response><d:response><d:href>/remote.php/dav/files/xxxxxxx/FS_E-Mail/test_web.txt</d:href><d:propstat><d:prop><d:getlastmodified>Tue, 08 Aug 2017 13:46:53 GMT</d:getlastmodified><d:getcontentlength>25</d:getcontentlength><d:resourcetype/><d:getetag>"19-5563e322a5b7a"</d:getetag><d:getcontenttype>text/plain</d:getcontenttype></d:prop><d:status>HTTP/1.1 200 OK</d:status></d:propstat><d:propstat><d:prop><d:quota-used-bytes/><d:quota-available-bytes/></d:prop><d:status>HTTP/1.1 404 Not Found</d:status></d:propstat></d:response></d:multistatus>
real 0m0,879s
user 0m0,043s
sys 0m0,007s

And here's my time curl response (with the hidden file):

<d:multistatus xmlns:d="DAV:" xmlns:s="http://sabredav.org/ns" xmlns:cal="urn:ietf:params:xml:ns:caldav" xmlns:cs="http://calendarserver.org/ns/" xmlns:card="urn:ietf:params:xml:ns:carddav" xmlns:oc="http://owncloud.org/ns" xmlns:nc="http://nextcloud.org/ns"><d:response><d:href>/remote.php/dav/files/xxxxxxx/FS_E-Mail/</d:href><d:propstat><d:prop><d:getlastmodified>Tue, 08 Aug 2017 14:07:44 GMT</d:getlastmodified><d:resourcetype><d:collection/></d:resourcetype><d:quota-used-bytes>25</d:quota-used-bytes><d:quota-available-bytes>-3</d:quota-available-bytes><d:getetag>"5989c5b00ed0a"</d:getetag></d:prop><d:status>HTTP/1.1 200 OK</d:status></d:propstat></d:response><d:response><d:href>/remote.php/dav/files/xxxxxxx/FS_E-Mail/.hidden</d:href><d:propstat><d:prop><d:getlastmodified>Tue, 08 Aug 2017 14:07:37 GMT</d:getlastmodified><d:getcontentlength>0</d:getcontentlength><d:resourcetype/><d:getetag>"0-5563e7c5c4291"</d:getetag><d:getcontenttype></d:getcontenttype></d:prop><d:status>HTTP/1.1 200 OK</d:status></d:propstat><d:propstat><d:prop><d:quota-used-bytes/><d:quota-available-bytes/></d:prop><d:status>HTTP/1.1 404 Not Found</d:status></d:propstat></d:response><d:response><d:href>/remote.php/dav/files/xxxxxxx/FS_E-Mail/test_web.txt</d:href><d:propstat><d:prop><d:getlastmodified>Tue, 08 Aug 2017 13:46:53 GMT</d:getlastmodified><d:getcontentlength>25</d:getcontentlength><d:resourcetype/><d:getetag>"19-5563e322a5b7a"</d:getetag><d:getcontenttype>text/plain</d:getcontenttype></d:prop><d:status>HTTP/1.1 200 OK</d:status></d:propstat><d:propstat><d:prop><d:quota-used-bytes/><d:quota-available-bytes/></d:prop><d:status>HTTP/1.1 404 Not Found</d:status></d:propstat></d:response></d:multistatus>
real 0m1,020s
user 0m0,047s
sys 0m0,007s

MorrisJobke · 2017-08-08T16:15:00Z

Is this a bug in the Android client?

I would say so. If the web UI and desktop client works, there seems to be an issue in the android app. @tobiasKaminsky Does that help you to reproduce the issue?

tobiasKaminsky · 2017-08-09T09:39:42Z

I tried to reproduce it

add external storage via webdav on Server1, which links to another NC12
test if external storage works on android app -> 👍
add ".something" to external storage: shows correctly on webUI
refresh externalstorage on android app -> 👍
uploaded a file to externalstorage on android app -> 👍
Even the hidden file is shown correctly.

bpcurse · 2017-08-09T12:41:01Z

Sorry, I know this is in contrary to my first post, but the way you did it works here, too. Don't know why it didn't then.

You need to follow these steps to reproduce it:

Create a new text file via WebUI (first_created_file.txt) -> OK, shown on WebUI and App
Create a new hidden text file via WebUI but delete the .txt extension in the create/naming window (.second_created_file)
-> FAIL, shown on the WebUI but not on the App (with the "show hidden files" option set)
Create another text file via Web UI (third_created_file.txt) -> FAIL, shown on WebUI but not on the App
Remove the hidden file via Web UI (.second_created_file) -> OK, The files 1 & 3 are visible again

Hope this will bring up the same issue in your NC instance/app. I Tried this again on a completely new NC 12.0.1 / App 1.4.3 (not newly installed) and enabled the "external storage" app.

tobiasKaminsky · 2017-08-09T14:12:50Z

Working file:

<d:response>
		<d:href>/remote.php/webdav/WebDAV/first_created_file.txt</d:href>
		<d:propstat>
			<d:prop>
				<d:getlastmodified>Wed, 09 Aug 2017 12:42:32 GMT</d:getlastmodified>
				<d:getcontentlength>28</d:getcontentlength>
				<d:resourcetype/>
				<d:getetag>&quot;1c-5565169dd3e00&quot;</d:getetag>
				<d:getcontenttype>text/plain</d:getcontenttype>
			</d:prop>

Non working file:

<d:response>
		<d:href>/remote.php/webdav/WebDAV/.second_created_file</d:href>
		<d:propstat>
			<d:prop>
				<d:getlastmodified>Wed, 09 Aug 2017 12:43:11 GMT</d:getlastmodified>
				<d:getcontentlength>29</d:getcontentlength>
				<d:resourcetype/>
				<d:getetag>&quot;1d-556516c3055c0&quot;</d:getetag>
				<d:getcontenttype></d:getcontenttype>
			</d:prop>

We fail if the contenttype is null.
Will dig into it...

tobiasKaminsky · 2017-08-09T14:18:46Z

@MorrisJobke the problem seems to be:

unknown file type is normally "application/octet-stream"
on webdav remote it is however: empty/null

It seems that the file type is only guessed by the extension and therefore returns "" if not possible.

MorrisJobke · 2017-08-09T18:28:34Z

on webdav remote it is however: empty/null

😢 Could you report this as a bug report in the server repo with reproduction steps?

tobiasKaminsky · 2017-08-10T05:18:05Z

Done in nextcloud/server#6059

tobiasKaminsky · 2017-08-10T06:05:56Z

So I did this also on our side and fixed it:
If there is no contenttype it falls back to "application/octet-stream".
Will be in next version and next dev version, if you want to try it:
https://github.com/nextcloud/android/blob/master/CONTRIBUTING.md#dev-release

tobiasKaminsky · 2017-08-10T10:51:45Z

This in now in android lib 24 and will be in upcoming version.

j-ed · 2017-08-17T12:43:54Z

@tobiasKaminsky I've just installed the Nextcloud-dev (20170817) app and can confirm that the missing content is now displayed instantly.

j-ed mentioned this issue May 30, 2017

Nextcloud Dev: 20170528: Some folders do not appear in the list. #1047

Closed

tobiasKaminsky added the bug label Jun 26, 2017

tobiasKaminsky added the needs info Waiting for info from user(s). Issues with this label will auto-stale. label Jun 27, 2017

tobiasKaminsky mentioned this issue Jun 30, 2017

Opening a folder on an external share shows "no data available" #994

Closed

tobiasKaminsky closed this as completed Jul 31, 2017

tobiasKaminsky reopened this Aug 8, 2017

tobiasKaminsky mentioned this issue Aug 10, 2017

Content-Type null on external webdav nextcloud/server#6059

Closed

tobiasKaminsky added approved and removed needs info Waiting for info from user(s). Issues with this label will auto-stale. labels Aug 10, 2017

tobiasKaminsky mentioned this issue Aug 10, 2017

Prevent npe if missing contenttype via webdav nextcloud/android-library#70

Merged

tobiasKaminsky added this to the Nextcloud App 2.0.0 milestone Aug 10, 2017

tobiasKaminsky added the pr exists label Aug 10, 2017

tobiasKaminsky closed this as completed Aug 10, 2017

j-ed mentioned this issue Sep 18, 2017

NC 12.0.0 - storage with mount id XX is not available nextcloud/server#5153

Closed

nextcloud-android-bot mentioned this issue Oct 12, 2018

Show only size of local files when browsing On device directories #3136

Closed

nextcloud-android-bot mentioned this issue Jan 25, 2019

Warn if app was not installed in 'internal storage' #3495

Closed

meichthys mentioned this issue Jan 29, 2022

Auto upload to external storages not working with IOS App v4.1.0.17 and later nextcloud/ios#1788

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Nextcloud app doesn't show all files and directories on an external storage #468

Nextcloud app doesn't show all files and directories on an external storage #468

j-ed commented Dec 22, 2016 •

edited by tobiasKaminsky

Loading

tobiasKaminsky commented Jun 26, 2017

tobiasKaminsky commented Jun 27, 2017

exmoorone commented Jun 27, 2017 •

edited

Loading

j-ed commented Jun 27, 2017

tobiasKaminsky commented Jun 30, 2017

tobiasKaminsky commented Jul 31, 2017

bpcurse commented Aug 7, 2017 •

edited

Loading

tobiasKaminsky commented Aug 8, 2017

MorrisJobke commented Aug 8, 2017

bpcurse commented Aug 8, 2017 •

edited

Loading

bpcurse commented Aug 8, 2017 •

edited

Loading

MorrisJobke commented Aug 8, 2017

bpcurse commented Aug 8, 2017

MorrisJobke commented Aug 8, 2017

tobiasKaminsky commented Aug 9, 2017

bpcurse commented Aug 9, 2017

tobiasKaminsky commented Aug 9, 2017

tobiasKaminsky commented Aug 9, 2017

MorrisJobke commented Aug 9, 2017

tobiasKaminsky commented Aug 10, 2017

tobiasKaminsky commented Aug 10, 2017 •

edited

Loading

tobiasKaminsky commented Aug 10, 2017

j-ed commented Aug 17, 2017

Nextcloud app doesn't show all files and directories on an external storage #468

Nextcloud app doesn't show all files and directories on an external storage #468

Comments

j-ed commented Dec 22, 2016 • edited by tobiasKaminsky Loading

Actual behaviour

Expected behaviour

Log files

Environment data

tobiasKaminsky commented Jun 26, 2017

tobiasKaminsky commented Jun 27, 2017

exmoorone commented Jun 27, 2017 • edited Loading

j-ed commented Jun 27, 2017

tobiasKaminsky commented Jun 30, 2017

tobiasKaminsky commented Jul 31, 2017

bpcurse commented Aug 7, 2017 • edited Loading

tobiasKaminsky commented Aug 8, 2017

MorrisJobke commented Aug 8, 2017

bpcurse commented Aug 8, 2017 • edited Loading

bpcurse commented Aug 8, 2017 • edited Loading

Moved Permanently

MorrisJobke commented Aug 8, 2017

bpcurse commented Aug 8, 2017

MorrisJobke commented Aug 8, 2017

tobiasKaminsky commented Aug 9, 2017

bpcurse commented Aug 9, 2017

tobiasKaminsky commented Aug 9, 2017

tobiasKaminsky commented Aug 9, 2017

MorrisJobke commented Aug 9, 2017

tobiasKaminsky commented Aug 10, 2017

tobiasKaminsky commented Aug 10, 2017 • edited Loading

tobiasKaminsky commented Aug 10, 2017

j-ed commented Aug 17, 2017

j-ed commented Dec 22, 2016 •

edited by tobiasKaminsky

Loading

exmoorone commented Jun 27, 2017 •

edited

Loading

bpcurse commented Aug 7, 2017 •

edited

Loading

bpcurse commented Aug 8, 2017 •

edited

Loading

bpcurse commented Aug 8, 2017 •

edited

Loading

tobiasKaminsky commented Aug 10, 2017 •

edited

Loading