Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

New kind of GDrive Ban? (error googleapi: Error 429: Rate Limit Exceeded, rateLimitExceeded) #3200

Closed
snickers2k opened this issue May 15, 2019 · 5 comments
Closed

New kind of GDrive Ban? (error googleapi: Error 429: Rate Limit Exceeded, rateLimitExceeded) #3200

snickers2k opened this issue May 15, 2019 · 5 comments

Comments

@snickers2k
Copy link

snickers2k commented May 15, 2019 •
edited
Loading

For months i had no Google Ban. So i decided to let Plex crawl "on event", not on schedule. Worked for the last 2 or 3 weeks. Until today. I'll go back to schedule and see what happens. But i thought the Bans are "impossible" nowdays?

Same on Mac OS and Linux on different Locations.

2019/05/15 12:21:38 DEBUG : rclone: Version "v1.47.0" starting with parameters ["rclone" "mount" "SecretDrive:" "/Users/snickers/Documents/SecretDrive" "--allow-other" "--dir-cache-time" "48h" "--vfs-read-chunk-size" "32M" "--vfs-read-chunk-size-limit" "2G" "--buffer-size" "512M" "--log-file=/Users/snickers/Documents/RCLONE-LOG.log" "--log-level=DEBUG"]
2019/05/15 12:21:38 DEBUG : Using config file from "/Users/snickers/.config/rclone/rclone.conf"
2019/05/15 12:21:39 DEBUG : pacer: low level retry 1/10 (error googleapi: Error 429: Rate Limit Exceeded, rateLimitExceeded)
2019/05/15 12:21:39 DEBUG : pacer: Rate limited, increasing sleep to 1.80151924s
2019/05/15 12:21:39 DEBUG : pacer: Reducing sleep to 0s
2019/05/15 12:21:41 DEBUG : pacer: low level retry 1/10 (error googleapi: Error 429: Rate Limit Exceeded, rateLimitExceeded)
2019/05/15 12:21:41 DEBUG : pacer: Rate limited, increasing sleep to 1.80407828s
2019/05/15 12:21:41 DEBUG : pacer: low level retry 2/10 (error googleapi: Error 429: Rate Limit Exceeded, rateLimitExceeded)
2019/05/15 12:21:41 DEBUG : pacer: Rate limited, increasing sleep to 2.778318749s
2019/05/15 12:21:43 DEBUG : pacer: low level retry 3/10 (error googleapi: Error 429: Rate Limit Exceeded, rateLimitExceeded)
2019/05/15 12:21:43 DEBUG : pacer: Rate limited, increasing sleep to 4.418965336s
2019/05/15 12:21:45 DEBUG : pacer: Reducing sleep to 0s
2019/05/15 12:21:50 DEBUG : Encrypted drive 'SecretDrive:Misc/': Mounting on "/Users/snickers/Documents/PrivateDrive"
2019/05/15 12:21:51 DEBUG : pacer: low level retry 1/10 (error googleapi: Error 429: Rate Limit Exceeded, rateLimitExceeded)
2019/05/15 12:21:51 DEBUG : pacer: Rate limited, increasing sleep to 1.482635255s
2019/05/15 12:21:51 DEBUG : pacer: low level retry 2/10 (error googleapi: Error 429: Rate Limit Exceeded, rateLimitExceeded)
2019/05/15 12:21:51 DEBUG : pacer: Rate limited, increasing sleep to 2.751767441s
2019/05/15 12:21:52 DEBUG : pacer: Reducing sleep to 0s
2019/05/15 12:21:52 DEBUG : Adding path "vfs/forget" to remote control registry
2019/05/15 12:21:52 DEBUG : Adding path "vfs/refresh" to remote control registry
2019/05/15 12:21:52 DEBUG : Adding path "vfs/poll-interval" to remote control registry
2019/05/15 12:21:52 DEBUG : : Root: 
2019/05/15 12:21:52 DEBUG : : >Root: node=/, err=<nil>
2019/05/15 12:21:52 DEBUG : : Statfs: 
2019/05/15 12:21:52 DEBUG : /: Attr: 
2019/05/15 12:21:52 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=<nil>
2019/05/15 12:21:52 DEBUG : : Statfs: 
2019/05/15 12:21:52 DEBUG : : Statfs: 
2019/05/15 12:21:55 DEBUG : : >Statfs: stat={Blocks:4294967295 Bfree:4294967295 Bavail:4294967295 Files:1000000000 Ffree:1000000000 Bsize:4096 Namelen:255 Frsize:4096}, err=<nil>
2019/05/15 12:21:55 DEBUG : : >Statfs: stat={Blocks:4294967295 Bfree:4294967295 Bavail:4294967295 Files:1000000000 Ffree:1000000000 Bsize:4096 Namelen:255 Frsize:4096}, err=<nil>
2019/05/15 12:21:55 DEBUG : : >Statfs: stat={Blocks:4294967295 Bfree:4294967295 Bavail:4294967295 Files:1000000000 Ffree:1000000000 Bsize:4096 Namelen:255 Frsize:4096}, err=<nil>
2019/05/15 12:21:55 DEBUG : /: Attr: 
2019/05/15 12:21:55 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=<nil>
2019/05/15 12:21:55 DEBUG : : Statfs: 
2019/05/15 12:21:55 DEBUG : : >Statfs: stat={Blocks:4294967295 Bfree:4294967295 Bavail:4294967295 Files:1000000000 Ffree:1000000000 Bsize:4096 Namelen:255 Frsize:4096}, err=<nil>
2019/05/15 12:21:55 DEBUG : : Statfs: 
2019/05/15 12:21:55 DEBUG : : >Statfs: stat={Blocks:4294967295 Bfree:4294967295 Bavail:4294967295 Files:1000000000 Ffree:1000000000 Bsize:4096 Namelen:255 Frsize:4096}, err=<nil>
2019/05/15 12:21:55 DEBUG : : Statfs: 
2019/05/15 12:21:55 DEBUG : : >Statfs: stat={Blocks:4294967295 Bfree:4294967295 Bavail:4294967295 Files:1000000000 Ffree:1000000000 Bsize:4096 Namelen:255 Frsize:4096}, err=<nil>
2019/05/15 12:21:55 DEBUG : : Statfs: 
2019/05/15 12:21:55 DEBUG : : >Statfs: stat={Blocks:4294967295 Bfree:4294967295 Bavail:4294967295 Files:1000000000 Ffree:1000000000 Bsize:4096 Namelen:255 Frsize:4096}, err=<nil>
2019/05/15 12:21:55 DEBUG : : Statfs: 
2019/05/15 12:21:55 DEBUG : : >Statfs: stat={Blocks:4294967295 Bfree:4294967295 Bavail:4294967295 Files:1000000000 Ffree:1000000000 Bsize:4096 Namelen:255 Frsize:4096}, err=<nil>
2019/05/15 12:21:55 DEBUG : : Statfs: 
2019/05/15 12:21:55 DEBUG : : >Statfs: stat={Blocks:4294967295 Bfree:4294967295 Bavail:4294967295 Files:1000000000 Ffree:1000000000 Bsize:4096 Namelen:255 Frsize:4096}, err=<nil>
2019/05/15 12:21:55 DEBUG : /: Attr: 
2019/05/15 12:21:55 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=<nil>
2019/05/15 12:21:55 DEBUG : : Statfs: 
2019/05/15 12:21:55 DEBUG : : >Statfs: stat={Blocks:4294967295 Bfree:4294967295 Bavail:4294967295 Files:1000000000 Ffree:1000000000 Bsize:4096 Namelen:255 Frsize:4096}, err=<nil>
2019/05/15 12:21:55 DEBUG : : Statfs: 
2019/05/15 12:21:55 DEBUG : : >Statfs: stat={Blocks:4294967295 Bfree:4294967295 Bavail:4294967295 Files:1000000000 Ffree:1000000000 Bsize:4096 Namelen:255 Frsize:4096}, err=<nil>
2019/05/15 12:21:55 DEBUG : : Statfs: 
2019/05/15 12:21:55 DEBUG : : >Statfs: stat={Blocks:4294967295 Bfree:4294967295 Bavail:4294967295 Files:1000000000 Ffree:1000000000 Bsize:4096 Namelen:255 Frsize:4096}, err=<nil>
2019/05/15 12:21:55 DEBUG : : Statfs: 
2019/05/15 12:21:55 DEBUG : : Statfs: 
2019/05/15 12:21:55 DEBUG : : >Statfs: stat={Blocks:4294967295 Bfree:4294967295 Bavail:4294967295 Files:1000000000 Ffree:1000000000 Bsize:4096 Namelen:255 Frsize:4096}, err=<nil>
2019/05/15 12:21:55 DEBUG : : >Statfs: stat={Blocks:4294967295 Bfree:4294967295 Bavail:4294967295 Files:1000000000 Ffree:1000000000 Bsize:4096 Namelen:255 Frsize:4096}, err=<nil>
2019/05/15 12:21:55 DEBUG : : Statfs: 
2019/05/15 12:21:55 DEBUG : : >Statfs: stat={Blocks:4294967295 Bfree:4294967295 Bavail:4294967295 Files:1000000000 Ffree:1000000000 Bsize:4096 Namelen:255 Frsize:4096}, err=<nil>
2019/05/15 12:21:55 DEBUG : /: Lookup: name="DCIM"
2019/05/15 12:21:55 DEBUG : pacer: low level retry 1/10 (error googleapi: Error 429: Rate Limit Exceeded, rateLimitExceeded)
2019/05/15 12:21:55 DEBUG : pacer: Rate limited, increasing sleep to 1.792921945s
2019/05/15 12:21:55 DEBUG : pacer: low level retry 2/10 (error googleapi: Error 429: Rate Limit Exceeded, rateLimitExceeded)
2019/05/15 12:21:55 DEBUG : pacer: Rate limited, increasing sleep to 2.068730842s
2019/05/15 12:21:58 DEBUG : pacer: Reducing sleep to 0s
2019/05/15 12:21:59 DEBUG : pacer: low level retry 1/10 (error googleapi: Error 429: Rate Limit Exceeded, rateLimitExceeded)
2019/05/15 12:21:59 DEBUG : pacer: Rate limited, increasing sleep to 1.055973034s
2019/05/15 12:21:59 DEBUG : pacer: Reducing sleep to 0s
2019/05/15 12:21:59 DEBUG : /: >Lookup: node=<nil>, err=no such file or directory
2019/05/15 12:21:59 DEBUG : /: Attr: 
2019/05/15 12:21:59 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=<nil>
2019/05/15 12:21:59 DEBUG : /: Lookup: name=".Spotlight-V100"
2019/05/15 12:21:59 DEBUG : /: >Lookup: node=<nil>, err=no such file or directory
2019/05/15 12:21:59 DEBUG : : Statfs: 
2019/05/15 12:21:59 DEBUG : /: Lookup: name=".metadata_never_index"
2019/05/15 12:21:59 DEBUG : : >Statfs: stat={Blocks:4294967295 Bfree:4294967295 Bavail:4294967295 Files:1000000000 Ffree:1000000000 Bsize:4096 Namelen:255 Frsize:4096}, err=<nil>
2019/05/15 12:21:59 DEBUG : /: >Lookup: node=<nil>, err=no such file or directory
2019/05/15 12:21:59 DEBUG : /: Lookup: name=".metadata_never_index_unless_rootfs"
2019/05/15 12:21:59 DEBUG : /: >Lookup: node=<nil>, err=no such file or directory
2019/05/15 12:21:59 DEBUG : : Statfs: 
2019/05/15 12:21:59 DEBUG : : >Statfs: stat={Blocks:4294967295 Bfree:4294967295 Bavail:4294967295 Files:1000000000 Ffree:1000000000 Bsize:4096 Namelen:255 Frsize:4096}, err=<nil>
2019/05/15 12:21:59 DEBUG : : Statfs: 
2019/05/15 12:21:59 DEBUG : : >Statfs: stat={Blocks:4294967295 Bfree:4294967295 Bavail:4294967295 Files:1000000000 Ffree:1000000000 Bsize:4096 Namelen:255 Frsize:4096}, err=<nil>
2019/05/15 12:21:59 DEBUG : : Statfs: 
2019/05/15 12:21:59 DEBUG : : >Statfs: stat={Blocks:4294967295 Bfree:4294967295 Bavail:4294967295 Files:1000000000 Ffree:1000000000 Bsize:4096 Namelen:255 Frsize:4096}, err=<nil>
2019/05/15 12:21:59 DEBUG : : Statfs: 
2019/05/15 12:21:59 DEBUG : : >Statfs: stat={Blocks:4294967295 Bfree:4294967295 Bavail:4294967295 Files:1000000000 Ffree:1000000000 Bsize:4096 Namelen:255 Frsize:4096}, err=<nil>
2019/05/15 12:22:00 DEBUG : /: Lookup: name=".localized"
2019/05/15 12:22:00 DEBUG : /: >Lookup: node=<nil>, err=no such file or directory
2019/05/15 12:22:00 DEBUG : /: Lookup: name=".metadata_never_index"
2019/05/15 12:22:00 DEBUG : /: >Lookup: node=<nil>, err=no such file or directory
2019/05/15 12:22:00 DEBUG : /: Lookup: name=".metadata_never_index_unless_rootfs"
2019/05/15 12:22:00 DEBUG : /: >Lookup: node=<nil>, err=no such file or directory
2019/05/15 12:22:00 DEBUG : : Statfs: 
2019/05/15 12:22:00 DEBUG : : >Statfs: stat={Blocks:4294967295 Bfree:4294967295 Bavail:4294967295 Files:1000000000 Ffree:1000000000 Bsize:4096 Namelen:255 Frsize:4096}, err=<nil>
2019/05/15 12:22:00 DEBUG : : Statfs: 
2019/05/15 12:22:00 DEBUG : : >Statfs: stat={Blocks:4294967295 Bfree:4294967295 Bavail:4294967295 Files:1000000000 Ffree:1000000000 Bsize:4096 Namelen:255 Frsize:4096}, err=<nil>
2019/05/15 12:22:00 DEBUG : : Statfs: 
2019/05/15 12:22:00 DEBUG : : Statfs: 
2019/05/15 12:22:00 DEBUG : : >Statfs: stat={Blocks:4294967295 Bfree:4294967295 Bavail:4294967295 Files:1000000000 Ffree:1000000000 Bsize:4096 Namelen:255 Frsize:4096}, err=<nil>
2019/05/15 12:22:00 DEBUG : : >Statfs: stat={Blocks:4294967295 Bfree:4294967295 Bavail:4294967295 Files:1000000000 Ffree:1000000000 Bsize:4096 Namelen:255 Frsize:4096}, err=<nil>
2019/05/15 12:22:00 DEBUG : : Statfs: 
2019/05/15 12:22:00 DEBUG : : >Statfs: stat={Blocks:4294967295 Bfree:4294967295 Bavail:4294967295 Files:1000000000 Ffree:1000000000 Bsize:4096 Namelen:255 Frsize:4096}, err=<nil>
2019/05/15 12:22:00 DEBUG : : Statfs: 
2019/05/15 12:22:00 DEBUG : : >Statfs: stat={Blocks:4294967295 Bfree:4294967295 Bavail:4294967295 Files:1000000000 Ffree:1000000000 Bsize:4096 Namelen:255 Frsize:4096}, err=<nil>
2019/05/15 12:22:00 DEBUG : : Statfs: 
2019/05/15 12:22:00 DEBUG : : Statfs: 
2019/05/15 12:22:00 DEBUG : : >Statfs: stat={Blocks:4294967295 Bfree:4294967295 Bavail:4294967295 Files:1000000000 Ffree:1000000000 Bsize:4096 Namelen:255 Frsize:4096}, err=<nil>
2019/05/15 12:22:00 DEBUG : : >Statfs: stat={Blocks:4294967295 Bfree:4294967295 Bavail:4294967295 Files:1000000000 Ffree:1000000000 Bsize:4096 Namelen:255 Frsize:4096}, err=<nil>
2019/05/15 12:22:00 DEBUG : : Statfs: 
2019/05/15 12:22:00 DEBUG : : >Statfs: stat={Blocks:4294967295 Bfree:4294967295 Bavail:4294967295 Files:1000000000 Ffree:1000000000 Bsize:4096 Namelen:255 Frsize:4096}, err=<nil>
2019/05/15 12:22:00 DEBUG : /: Lookup: name=".localized"
2019/05/15 12:22:00 DEBUG : /: >Lookup: node=<nil>, err=no such file or directory
2019/05/15 12:22:00 DEBUG : : Statfs: 
2019/05/15 12:22:00 DEBUG : : >Statfs: stat={Blocks:4294967295 Bfree:4294967295 Bavail:4294967295 Files:1000000000 Ffree:1000000000 Bsize:4096 Namelen:255 Frsize:4096}, err=<nil>
2019/05/15 12:22:00 DEBUG : : Statfs: 
2019/05/15 12:22:00 DEBUG : : >Statfs: stat={Blocks:4294967295 Bfree:4294967295 Bavail:4294967295 Files:1000000000 Ffree:1000000000 Bsize:4096 Namelen:255 Frsize:4096}, err=<nil>
2019/05/15 12:22:00 DEBUG : : Statfs: 
2019/05/15 12:22:00 DEBUG : : >Statfs: stat={Blocks:4294967295 Bfree:4294967295 Bavail:4294967295 Files:1000000000 Ffree:1000000000 Bsize:4096 Namelen:255 Frsize:4096}, err=<nil>
2019/05/15 12:22:00 DEBUG : : Statfs: 
2019/05/15 12:22:00 DEBUG : : >Statfs: stat={Blocks:4294967295 Bfree:4294967295 Bavail:4294967295 Files:1000000000 Ffree:1000000000 Bsize:4096 Namelen:255 Frsize:4096}, err=<nil>
2019/05/15 12:22:00 DEBUG : : Statfs: 
2019/05/15 12:22:00 DEBUG : : >Statfs: stat={Blocks:4294967295 Bfree:4294967295 Bavail:4294967295 Files:1000000000 Ffree:1000000000 Bsize:4096 Namelen:255 Frsize:4096}, err=<nil>
2019/05/15 12:22:00 DEBUG : /: Lookup: name=".localized"
2019/05/15 12:22:00 DEBUG : /: >Lookup: node=<nil>, err=no such file or directory
2019/05/15 12:22:14 DEBUG : /: Fsync: 
2019/05/15 12:22:14 DEBUG : /: >Fsync: err=<nil>
2019/05/15 12:22:14 DEBUG : rclone: Version "v1.47.0" finishing with parameters ["rclone" "mount" "SecretDrive:Misc/" "/Users/snickers/Documents/PrivateDrive" "--allow-other" "--dir-cache-time" "48h" "--vfs-read-chunk-size" "32M" "--vfs-read-chunk-size-limit" "2G" "--buffer-size" "512M" "--log-file=/Users/snickers/Documents/RCLONE-LOG.log" "--log-level=DEBUG"]
@abclution
Copy link

abclution commented May 15, 2019 •
edited
Loading

Same here, getting those errors as well and I am not dealing with any large files or anything.

Literally using the .link.html format on a rclone mount to organize my Gdocs, no data transfers just browsing directories.

Edit: Also checked all my Google API quotas and am WELL below any limits. I'm only on a DSL connection so can't really saturate nuffin.

2019/05/15 13:31:24 DEBUG : zzz## TODelete/Scans: list: cold listing: 2019-05-15 00:19:38.508322007 +0300 EEST
2019/05/15 13:31:26 DEBUG : pacer: low level retry 1/10 (error googleapi: Error 429: Rate Limit Exceeded, rateLimitExceeded)
2019/05/15 13:31:26 DEBUG : pacer: Rate limited, increasing sleep to 1.157514066s
2019/05/15 13:31:26 DEBUG : pacer: low level retry 2/10 (error googleapi: Error 429: Rate Limit Exceeded, rateLimitExceeded)
2019/05/15 13:31:26 DEBUG : pacer: Rate limited, increasing sleep to 2.338131065s
2019/05/15 13:31:28 DEBUG : pacer: low level retry 3/10 (error googleapi: Error 429: Rate Limit Exceeded, rateLimitExceeded)
2019/05/15 13:31:28 DEBUG : pacer: Rate limited, increasing sleep to 4.22586565s
2019/05/15 13:31:30 DEBUG : pacer: low level retry 4/10 (error googleapi: Error 429: Rate Limit Exceeded, rateLimitExceeded)
2019/05/15 13:31:30 DEBUG : pacer: Rate limited, increasing sleep to 8.328344799s
2019/05/15 13:31:34 DEBUG : pacer: low level retry 5/10 (error googleapi: Error 429: Rate Limit Exceeded, rateLimitExceeded)
2019/05/15 13:31:34 DEBUG : pacer: Rate limited, increasing sleep to 16.914832246s
2019/05/15 13:31:43 DEBUG : pacer: low level retry 6/10 (error googleapi: Error 429: Rate Limit Exceeded, rateLimitExceeded)
2019/05/15 13:31:43 DEBUG : pacer: Rate limited, increasing sleep to 16.933096064s
2019/05/15 13:31:59 DEBUG : pacer: low level retry 7/10 (error googleapi: Error 429: Rate Limit Exceeded, rateLimitExceeded)

@dionvl
Copy link

dionvl commented May 15, 2019 •
edited
Loading

Adding --user-agent "random string" to your command will fix it. It seems like since last night Google has started to rate limit every user agent with rclone (which is the default) in it.

@abclution
Copy link

abclution commented May 15, 2019 •
edited
Loading

Hilarious, it works.

I did this to match current Chrome release string.

--user-agent "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.77 Safari/537.36"

Works great, 10x faster than it was before. If they are playing games with limits, it makes sense that they would prioritize Chrome.

Thanks Oba... Google!

ismail added a commit to ismail/hacks that referenced this issue May 15, 2019
@ismail
Ok google. Ref: rclone/rclone#3200 (comment)
1a5a995
@ncw
Copy link
Member

ncw commented May 15, 2019 •
edited
Loading

There is a forum thread about this here: https://forum.rclone.org/t/all-of-a-sudden-googleapi-error-429/9977

Please take discussion there - thanks!

This was referenced May 15, 2019
Open
Open
@NoLooseEnds NoLooseEnds mentioned this issue May 16, 2019
Closed
@ncw
Copy link
Member

ncw commented May 17, 2019

I think the issue is over now, so I'm going to close this.

@ncw ncw closed this as completed May 17, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@ncw @snickers2k @abclution @dionvl