drive: Rclone cannot see all files/folder in public shared folder

[xyou365]

What is the problem you are having with rclone?

I want to copy files to my Google Drive by using the folder id of one publicly shared folder. But find that only part of files/folder can be seen by rclone. The way I use folder id is by creating one item in rclone.conf like this,

[src001]
type = drive
scope = drive
service_account_file = C:/AutoRclone/accounts/1.json
root_folder_id = 1OqnIlrj4BRgPcbFYRJTxIBSi7quzjax8

I have run rclone --config rclone.conf lsd src001:. It seems okay because all folders are listed.
But when run 3 times rclone --config rclone.conf size src001:

Total objects: 669
Total size: 19.793 GBytes (21252837599 Bytes)

Total objects: 659
Total size: 21.840 GBytes (23450518799 Bytes)

Total objects: 354
Total size: 22.185 GBytes (23820911775 Bytes)

it prints different size in different run.

After some discussion, find that means that --fast-list on Google Drive backend has a bug.

Details of this wacky situation and some more discussion are on forum.

[ncw]

This is easy to replicate

$ rclone size --drive-root-folder-id 1OqnIlrj4BRgPcbFYRJTxIBSi7quzjax8 drive:
Total objects: 304
Total size: 22.019 GBytes (23642721355 Bytes)
$ rclone size --drive-root-folder-id 1OqnIlrj4BRgPcbFYRJTxIBSi7quzjax8 drive:
Total objects: 0
Total size: 0 Bytes (0 Bytes)
$ rclone size --drive-root-folder-id 1OqnIlrj4BRgPcbFYRJTxIBSi7quzjax8 drive:
Total objects: 0
Total size: 0 Bytes (0 Bytes)
$ rclone size --drive-root-folder-id 1OqnIlrj4BRgPcbFYRJTxIBSi7quzjax8 drive:
Total objects: 398
Total size: 22.204 GBytes (23841092675 Bytes)

But works like this

$ rclone size --disable ListR  --drive-root-folder-id 1OqnIlrj4BRgPcbFYRJTxIBSi7quzjax8 drive:
Total objects: 6493
Total size: 245.790 GBytes (263915257230 Bytes)

If I set the grouping to 1 in ListR then it works but if I set it to 2 it doesn't, indicating there is some problem with more than one parent in the search term sometimes returning empty. For example

2019/10/22 12:21:12 DEBUG : HTTP REQUEST (req 0xc00097c500)
2019/10/22 12:21:12 DEBUG : GET /drive/v3/files?alt=json&fields=files%28id%2Cname%2Csize%2Cmd5Checksum%2Ctrashed%2CmodifiedTime%2CcreatedTime%2CmimeType%2Cparents%2CwebViewLink%29%2CnextPageToken&includeItemsFromAllDrives=true&pageSize=1000&prettyPrint=false&q=trashed%3Dfalse+and+%28%271F433xyzz9Zbjo6U5GNKIVS4AOwQNLOhR%27+in+parents+or+%271mVQsW5ZEQZItqwDJUKw6-apk9fR1V_rY%27+in+parents%29&supportsAllDrives=true HTTP/1.1
Host: www.googleapis.com
User-Agent: rclone/v1.49.5-DEV
Authorization: XXXX
X-Goog-Api-Client: gl-go/1.11.0 gdcl/20190802
Accept-Encoding: gzip

returns

2019/10/22 12:21:12 DEBUG : HTTP RESPONSE (req 0xc00097c500)
2019/10/22 12:21:12 DEBUG : HTTP/1.1 200 OK
Transfer-Encoding: chunked
Alt-Svc: quic=":443"; ma=2592000; v="46,43",h3-Q048=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000
Cache-Control: private, max-age=0, must-revalidate, no-transform
Content-Type: application/json; charset=UTF-8
Date: Tue, 22 Oct 2019 11:21:12 GMT
Expires: Tue, 22 Oct 2019 11:21:12 GMT
Server: GSE
Vary: Origin
Vary: X-Origin
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Xss-Protection: 1; mode=block

c
{"files":[]}
0

The query decodes to

"trashed=false and ('1F433xyzz9Zbjo6U5GNKIVS4AOwQNLOhR' in parents or '1mVQsW5ZEQZItqwDJUKw6-apk9fR1V_rY' in parents)"

which doesn't look out of the ordinary. There is definitely stuff in those folders

$ rclone lsf --drive-root-folder-id 1F433xyzz9Zbjo6U5GNKIVS4AOwQNLOhR drive:
intro.mp4
第一章 微信小程序介绍与开发准备/
第七章 微信小程序开发项目实战案例/
第三章 微信小程序开发框架/
第二章 上手第一个微信小程序/
第五章 微信小程序之原生API/
第八章 微信小程序开发实战注意事项与进阶指导/
第六章 微信小程序开发进阶之核心技能/
第四章 微信小程序开发之原生组件/
$ rclone lsf --drive-root-folder-id 1mVQsW5ZEQZItqwDJUKw6-apk9fR1V_rY drive:
mp3/
pdfs/
更多资源加Q群：170701297.txt

So I think this must be a bug in google drive, or some protection at work in google drive.

Use --disable ListR to work around until we figure out a solution!

[xyou365]

Looking forwarding your solution.
Some told that it is okay for Rclone Version 1.44.
So think that --disable ListR is default flag in that Rclone version.

[ncw]

Some told that it is okay for Rclone Version 1.44.
So think that --disable ListR is default flag in that Rclone version.

That is correct.