mail.ru cloud Unknown resource type "camera-upload"

[kuhytitusdlz]

Hi,

I get blank list:

C:\rclone>rclone --log-file=rclone.log --log-level DEBUG lsd test-mail:/

C:\\rclone>

log-file:

2020/03/02 12:35:37 DEBUG : rclone: Version "v1.51.0" starting with parameters ["rclone" "--log-file=rclone.log" "--log-level" "DEBUG" "lsd" "test-mail:/"]
2020/03/02 12:35:37 DEBUG : Using config file from "C:\\rclone\\rclone.conf"
2020/03/02 12:35:38 DEBUG : []: Excluding path "/Camera Uploads" from list: Unknown resource type "camera-upload"
2020/03/02 12:35:38 DEBUG : 5 go routines active
2020/03/02 12:35:38 DEBUG : rclone: Version "v1.51.0" finishing with parameters ["rclone" "--log-file=rclone.log" "--log-level" "DEBUG" "lsd" "test-mail:/"]

[ncw]

2020/03/02 12:35:38 DEBUG : []: Excluding path "/Camera Uploads" from list: Unknown resource type "camera-upload"

That is because the "Camera Uploads" object has reported itself as being a "camera-upload" instead of being a "file" or a "folder".

Maybe we could treat it as a "folder" I don't know - what do you think @ivandeex ?

[shvchk]

Temporary workaround is to make a copy of this folder via web interface (make sure you have enough free space), use rclone to copy/sync that copy, then you can delete it (to free used space).

[Kabachek]

Hi.
This also affects folders that's shared with other accounts:
2020/05/03 16:11:29 DEBUG : []: Excluding path "/CUBE" from list: Unknown resource type "shared"

[ivandeex]

@kuhytitusdlz @shvchk @Kabachek
Please run beta rclone build from https://github.com/ivandeex/rclone/releases/tag/v1.51.cameraupload and report whether it solves your problem.

[Kabachek]

Running beta on my NB, not on target system (mipsle needed for that)

2020/05/18 13:18:01 DEBUG : rclone: Version "v1.51.0-DEV" starting with parameters ["rclone-windows-4.0-amd64.exe" "-vv" "--log-file=rclone-log.txt" "--use-mmap" "size" "mailru:/"]
2020/05/18 13:18:01 DEBUG : Using config file from "C:\\Users\\User\\.config\\rclone\\rclone.conf"
2020/05/18 13:18:01 DEBUG : fs cache: renaming cache item "mailru:/" to be canonical "mailru:"
2020/05/18 13:18:05 ERROR : Camera Uploads: error listing: is a file not a directory
2020/05/18 13:18:17 DEBUG : 21 go routines active
2020/05/18 13:18:17 Failed to size with 2 errors: last error was: is a file not a directory

[ivandeex]

@Kabachek
Please check out another beta from https://github.com/ivandeex/rclone/releases/tag/v1.51.cameraupload2
Now it treats share/camera-upload items as files. You can try mipsle build now.
I'm still wondering how to reproduce camera uploads. Can I install some Android app and tap something?

[Kabachek]

I'm still wondering how to reproduce camera uploads. Can I install some Android app and tap something?

Yep, Mail.ru have it's own app in GPlay https://play.google.com/store/apps/details?id=ru.mail.cloud
In the android application, you should select the "Auto load" option or something similar. But i'm not sure, because i have iOS.

Well, when i check the root dir size, it's still exclude:

C:\Users\User\Downloads\rclone>rclone-windows-4.0-amd64.exe -vv --use-mmap size mailru:

2020/05/18 19:46:54 DEBUG : rclone: Version "v1.51.0-DEV" starting with parameters ["rclone-windows-4.0-amd64.exe" "-vv" "--use-mmap" "size" "mailru:"]
2020/05/18 19:46:54 DEBUG : Using config file from "C:\\Users\\User\\.config\\rclone\\rclone.conf"
2020/05/18 19:46:54 INFO  : []: Valid token not found, authorizing.
2020/05/18 19:46:54 DEBUG : Keeping previous permissions for config file: -rw-rw-rw-
2020/05/18 19:46:54 DEBUG : mailru: Saved new token in config file
2020/05/18 19:46:55 INFO  : []: Handling item "Camera Uploads" of type "camera-upload" as a file
2020/05/18 19:46:55 DEBUG : []: Excluding path "/Camera Uploads" from list: invalid hash
Total objects: 45889
Total size: 215.021 GBytes (230876750310 Bytes)
2020/05/18 19:47:11 DEBUG : 22 go routines active 

But, if i check "mailru:Camera uploads" instead, rclone "size" command works properly:

C:\Users\User\Downloads\rclone>rclone-windows-4.0-amd64.exe -vv --use-mmap size "mailru:/Camera uploads"

2020/05/18 19:47:55 DEBUG : rclone: Version "v1.51.0-DEV" starting with parameters ["rclone-windows-4.0-amd64.exe" "-vv" "--use-mmap" "size" "mailru:/Camera uploads"]
2020/05/18 19:47:55 DEBUG : Using config file from "C:\\Users\\User\\.config\\rclone\\rclone.conf"
2020/05/18 19:47:55 DEBUG : fs cache: renaming cache item "mailru:/Camera uploads" to be canonical "mailru:Camera uploads"
Total objects: 11148
Total size: 63.750 GBytes (68450664349 Bytes)
2020/05/18 19:48:00 DEBUG : 5 go routines active

[ivandeex]

Thanks for the hint. I installed the android app and reproduced your issue. Investigating it now.

[PikaYellow35]

Same problem on folders, that shared to you.

rclone lsd mail-drive: -vv
2020/06/04 11:41:12 DEBUG : rclone: Version "v1.52.0" starting with parameters ["rclone" "lsd" "mail-drive:" "-vv"]
2020/06/04 11:41:12 DEBUG : Using config file from "/home/yellow/.config/rclone/rclone.conf"
2020/06/04 11:41:12 DEBUG : []: Excluding path "/Aspia" from list: Unknown resource type "mounted"
2020/06/04 11:41:12 DEBUG : []: Excluding path "/KeeP" from list: Unknown resource type "mounted"
2020/06/04 11:41:12 DEBUG : 4 go routines active

[Cinerar]

Same problem for shared folders via mailru backend.
If i use webdav backend to access mail.ru everything works ok.

[PikaYellow35]

@Cinerar webdav access to mail.ru cloud enabled only for business account. I'm simple user and for me webdav return 403 code.

[ivandeex]

I'm going to extend the list of special folders as follows:

"folder", "camera-upload", "mounted", "shared"

(taken from WebDavMailRuCloud).
Rclone will treat all of them as a directory.
Also rclone will print a debugging (-vv) message for non-"folder" kinds.
I think it's also ok to add a flag --mailru-quirks=unknowndirs to make rclone accept unknown/future kinds of folders but print a warning.
I'm going to cut a beta with the fix soon, sorry and thanks all for patience.

[ivandeex]

@kuhytitusdlz @shvchk @Kabachek @PikaYellow35 @Cinerar

I released a beta that fixes this issue (linux/windows only atm). Please give it a try and report:
https://github.com/ivandeex/rclone/releases/tag/v1.53-mailru-camera-4

The test procedure is described in the pull request #4690 (comment)

If the fix works for you, we will merge #4690 on master, publish a new beta with more architectures, and include it in the next rclone release.

Thanks.

[Kabachek]

Seems no problems there

2020/10/18 17:55:44 DEBUG : rclone: Version "v1.53-mailru-camera-4" starting with parameters ["rclone-windows-4.0-amd64.exe" "-vv" "--use-mmap" "sync" "mailru:"
"Z:\\" "--log-file" "mailru-camera-3.txt"]
2020/10/18 17:55:44 DEBUG : Using config file from "C:\\Users\\User\\.config\\rclone\\rclone.conf"
2020/10/18 17:55:44 DEBUG : Creating backend with remote "mailru:"
2020/10/18 17:55:45 DEBUG : Creating backend with remote "Z:\\"
2020/10/18 17:55:45 DEBUG : fs cache: renaming cache item "Z:\\" to be canonical "//?/Z:/"
2020/10/18 17:55:45 DEBUG : []: [Camera Uploads]: folder has type "camera-upload"

2020/10/18 18:09:03 DEBUG : Camera Uploads/2020-10-17 19.18.26.MOV: Starting multi-thread copy with 2 parts of size 127.312M
2020/10/18 18:09:03 DEBUG : Camera Uploads/2020-10-17 19.18.26.MOV: multi-thread copy: stream 2/2 (133496832-266974740) size 127.294M starting
2020/10/18 18:09:03 DEBUG : Camera Uploads/2020-10-17 19.18.26.MOV: multi-thread copy: stream 1/2 (0-133496832) size 127.312M starting
2020/10/18 18:09:04 DEBUG : []: Switch file server: locks 1, url https://cloclo22.datacloudmail.ru/oauth-get/, expiry "2020-10-18T18:12:03.5884129+03:00"
2020/10/18 18:09:04 DEBUG : []: Switch file server: locks 1, url https://cloclo4.datacloudmail.ru/oauth-get/, expiry "2020-10-18T18:12:03.5684118+03:00"
2020/10/18 18:09:31 DEBUG : Camera Uploads/2020-10-17 19.05.54.MOV: MailruHash = 64d431268db95bae63cb3c6229166ce1ce3a0a60 OK
2020/10/18 18:09:31 INFO  : Camera Uploads/2020-10-17 19.05.54.MOV: Copied (new)
2020/10/18 18:09:31 DEBUG : []: Lock file server:   locks 2, url https://cloclo4.datacloudmail.ru/oauth-get/
....

2020/10/18 18:33:27 INFO  :
Transferred:      914.676M / 914.676 MBytes, 100%, 420.983 kBytes/s, ETA 0s
Checks:             58086 / 58086, 100%
Transferred:           39 / 39, 100%
Elapsed time:     37m42.7s

2020/10/18 18:33:27 DEBUG : 3 go routines active