New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CRITICAL: Amazon Drive does not work anymore with rclone 429: "429 Too Many Requests" / Rate exceeded #1417

Closed
ajkis opened this Issue May 18, 2017 · 93 comments

Comments

Projects
None yet
@ajkis
Copy link

ajkis commented May 18, 2017

It seems Amazon Drive blocked rclone, I tested it on 4 different servers, tried reauth the app but no success.

Any rclone command will deliver the following errors:

2017/05/18 11:19:14 DEBUG : pacer: Rate limited, sleeping for 666.145821ms (1 consecutive low level retries)
2017/05/18 11:19:14 DEBUG : pacer: low level retry 1/10 (error HTTP code 429: "429 Too Many Requests": response body: "{\"message\":\"Rate exceeded\"}")
@Technifocal

This comment has been minimized.

Copy link

Technifocal commented May 18, 2017

@ncw Already seems to be aware, #1415 (comment)

@PhilipFraDIKU

This comment has been minimized.

Copy link

PhilipFraDIKU commented May 18, 2017

Yes, getting the exact same problems. When typing rclone lsd ACD:ACD_Encrypted in terminal I get:

2017/05/18 11:24:30 Failed to create file system for "ACD:ACD_Encrypted": failed to get endpoints: HTTP code 429: "429 Too Many Requests": response body: "{\"message\":\"Rate exceeded\"}"

Can replicate the results in rcloneBrowser where it just hangs when you try to list directories or gain access to a remote.

@l3uddz

This comment has been minimized.

Copy link

l3uddz commented May 18, 2017

If rclone is indeed banned like acd_cli, they can kiss my subscription goodbye (not that they care). This is ridiculous. Perhaps they dont like that rclone security profile keys can be accessed by anyone, as evidenced by those guys trying to impersonate rclone with acd_cli. We must wait to hear what @ncw says as anything else is hearsay.

@mdubash

This comment has been minimized.

Copy link

mdubash commented May 18, 2017

I had a transfer in operation at 01:25 this morning (UK BST) when the logs show the transfer just stopped - it went to 0 bytes/s and stayed there. rclone lsd produces the exact same message as the OP.

@ncw

This comment has been minimized.

Copy link
Owner

ncw commented May 18, 2017

I have some emails out to contacts at Amazon asking what has happened. They are on the west coast of the US I think, so they won't be in to work for a few hours...

@mdubash interesting info - thanks

@Joshua1337

This comment has been minimized.

Copy link

Joshua1337 commented May 18, 2017

You can still get authcodes btw...

@geekcroft

This comment has been minimized.

Copy link

geekcroft commented May 18, 2017

@ncw have you seen the forum post from the guy who had a reply from Amazon DE?

@ncw

This comment has been minimized.

Copy link
Owner

ncw commented May 18, 2017

@geekcroft yes I have. I'd like confirmation though.

@geekcroft

This comment has been minimized.

Copy link

geekcroft commented May 18, 2017

@ncw Yeah am hoping its just some 2-bit support guy not knowing his stuff :(

@ajkis

This comment has been minimized.

Copy link
Author

ajkis commented May 18, 2017

I also wanted to check for my self and called customer support just now, they told me they got notification today that rclone wont be supported anymore and that I should use their Desktop App.

This is really beyond retarded, @ncw I hope you guys can deliver local cache soon so we can use gdrive instead.

I just switched all my servers to plexdrive until rclone gets local cache.

p.s. Its really shame that multipurpose tools like rclone gets removed from Amazon eg all apps with big traffic will get removed eventually.

@Technifocal

This comment has been minimized.

Copy link

Technifocal commented May 18, 2017

@ajkis If this is true, I will have to move over to GDrive. :(

@scriptzteam

This comment has been minimized.

Copy link

scriptzteam commented May 18, 2017

F()CK OFF AMAZON, I KNOW YOU READ, F()CK OFF !

@thenoahcomputer

This comment has been minimized.

Copy link

thenoahcomputer commented May 18, 2017

Wow this is seriously bad timing for me. I was in the process of restoring a backup when acd_cli got disabled, switched to rclone and now it's disabled too. I've emailed Amazon about it, but don't expect much from them given their recent blatant disregard for their paying customers need to access their own data.

Anyone know of another way to mount or pull files out of ACD with linux or shuffle it over in bulk to gdrive?

@Aldigator

This comment has been minimized.

Copy link

Aldigator commented May 18, 2017

An employee of Amazon Germany just told me on the phone that they are making changes to their services and infrastructure and that rclone probably needs to be updated accordingly.

EDIT:
Scratch that, just got on their support chat to clarify after reading some more comments and the bastards actually terminated support for rclone! Apparently some office-sitter at Amazon is under the impression that informing your own support staff (let alone PAYING CUSTOMERS) about such minor changes as dropping support for an entire goddamn platform is somehow optional.

@ajkis

This comment has been minimized.

Copy link
Author

ajkis commented May 18, 2017

An employee of Amazon Germany just told me on the phone that they are making changes to their services and infrastructure and that rclone probably needs to be updated accordingly.

I highly doubt that they would instruct their Tehnical staff to tell customer that rclone access is revoke and it would not be enabled again. Thats what they told me and i was on phone for more then 20 minutes asking if its possible it would work in the future since my usage is strictly rclone depended.

They put me on hold for 5 minutes ( after i was first told that rclone access is revoked ) then he told me that rclone access wont be renabled for sure and that i need to find another way to use Amazon Drive ( since i told them i need linux client that they dont support )

@geekcroft

This comment has been minimized.

Copy link

geekcroft commented May 18, 2017

@ncw anything on the Email front yet mate?

@nbyloff

This comment has been minimized.

Copy link

nbyloff commented May 18, 2017

@thenoahcomputer I am currently using these services to move data from ACD to Google
https://www.cloudhq.net
https://www.multcloud.com

@thenoahcomputer

This comment has been minimized.

Copy link

thenoahcomputer commented May 18, 2017

I called Amazon. They were very friendly and understanding, but they said it's going to take them 2 days to get back to me with a solution.

@nbyloff
Thanks for the info. Looks like a much faster solution than waiting on Amazon.

@ginsu0

This comment has been minimized.

Copy link

ginsu0 commented May 18, 2017

@nbyloff I started a transfer a few hours ago using both of those, but neither is exceeding 0.5MB/s. At that rate, my transfer will take 6 months. Looking for another solution, but haven't found anything good so far.

@mdubash

This comment has been minimized.

Copy link

mdubash commented May 18, 2017

@thenoahcomputer
You got a better response than I did. Which was: we don't support anything other than our own apps as everything is always changing at the server end. Fingers crossed.

@ginsu0

This comment has been minimized.

Copy link

ginsu0 commented May 18, 2017

@M0V3 Are you sure you linked the right page?

@M0V3

This comment has been minimized.

Copy link

M0V3 commented May 18, 2017

@ginsu0 Yea, edited my post.
Don't know how I managed to copy the wrong link.

Here it is one more time https://www.reddit.com/r/DataHoarder/comments/6bwbst/how_would_you_transfer_8tb_of_linux_isos_from_acd/dhpzcnn/

@thenoahcomputer

This comment has been minimized.

Copy link

thenoahcomputer commented May 18, 2017

I probably got better response because I didn't ask for support for rclone. I told them I just want my data back and can't get at it, because they disabled the only apps that would let me do that on Ubuntu. They suggested the web interface, which I told them would just spin it's wheel and then not download anything.

It'd be ideal if they'd offer a read-only S3 bucket without data charges to get my stuff back so I can just rclone it to Gdrive. The multcloud.com site seems to be working, but significantly slower than rclone.

@ginsu0

This comment has been minimized.

Copy link

ginsu0 commented May 18, 2017

@M0V3 Thanks. If that's what it's come down to, I'll spin up a Windows VM with the native client and move a few directories at a time. It's a shame there isn't a better way.

@kiding

This comment has been minimized.

Copy link

kiding commented May 18, 2017

#1409 #1415 have basically converged into the same issue after all. Leaving a comment for reference's sake.

@uncuervo

This comment has been minimized.

Copy link

uncuervo commented May 18, 2017

Hi guys,

I'm a newbie on this community so please be kind to me if I'm not compliance 100% with any terms or conditions that apply here :P

The point is;

I've been using rclone for a while (great app btw) just to connect my linux to ACD and encrypt on the fly, but such a nasty surprise I've got after seeing my ACD access with rclone was not working anymore...! maybe my fault? I was hoping so.. but NO!! it gets worse after seing all of this happened due all of the monkey business amazon is carrying out...? stating to "be open" and at the same time pushing us to do things exactly how they want (no blame, is their service but at least be honest!).

Anyway, I don't now for how far... but looks like amazon drive is not accepting more developers?

https://developer.amazon.com/amazon-drive

I'm saying that because despite of how complicated it can be, after checking the current situation I was considering (also as a suggesiton to everyone) to create our own apps so they can't ban any specific one due many users using it, or things like that... but, no luck going that way after finding that they've locked new registrations :/

Which drives me to; @ncw !!!, I have faith on you :) (sure many do)... and I'm sure after seeing such great job all of you have done so far you will be able to figure out how to address this issue. I can read on some comments that amazon support is saying they are doing changes and developers will need to adapt their apps? is that true?

I hope to hear from you guys.

All the best,

@gryphonmyers

This comment has been minimized.

Copy link

gryphonmyers commented May 19, 2017

The funny thing about their "use our sync client" response is that I tried, I really did, and it is utter garbage. It made my CPU Max out constantly and constantly forgot what it was doing, forcing me to reauth. POS client for an increasingly POS service

@barnabyc

This comment has been minimized.

Copy link

barnabyc commented May 19, 2017

For reference I'm seeing a slightly different message right now stating 401 Unauthorized and "invalid_client":

2017/05/18 22:35:07 Failed to create file system for "remote:foobar": failed to get endpoints: Get https://drive.amazonaws.com/drive/v1/account/endpoint: oauth2: cannot fetch token: 401 Unauthorized
Response: {"error_description":"Client authentication failed","error":"invalid_client"}

Edit: Ah, I see this was reported 35min ago in issue #1415

@calisro

This comment has been minimized.

Copy link

calisro commented May 19, 2017

Yes. If look in the manage apps section in cloud drive, rclone is no longer listed just like acdcli was removed. This looks like permanent or at least long term ban now....

@ajkis

This comment has been minimized.

Copy link
Author

ajkis commented May 20, 2017

@poom google does not enforce the limit so even with 1 user you will have unlimited space.

@vampywiz17

This comment has been minimized.

Copy link

vampywiz17 commented May 20, 2017

https://forum.rclone.org/t/rclone-has-been-banned-from-amazon-drive/2314

Update 2017-05-20

I have now heard back from Amazon. Rclone has been banned for having the encrypted secrets in the source code.

I've asked for new credentials so I can build an auth server and get rclone going again - I haven't heard back on that reply yet, but it is plausible we could get rclone running again with Amazon Drive in the not too distant future.

Thank you all for your patience

Nick

@gordan-bobic

This comment has been minimized.

Copy link

gordan-bobic commented May 20, 2017

Any chance of shipping a pre-compiled binary module containing the secrets, and thus avoiding needing an external authentication server?

@drvdijk

This comment has been minimized.

Copy link

drvdijk commented May 20, 2017

@gordan-bobic what would prevent other apps not to use such a binary module?

@gordan-bobic

This comment has been minimized.

Copy link

gordan-bobic commented May 20, 2017

@drvdijk only the fact that it'd be compiled into the binary and not human readable like it is in the source code.

There are other ACD apps already out there (not naming them for obvious reasons) that haven't been banned that ship with secrets in binaries (they are closed source) and don't use an external authentication server. They may get banned at some point, but having one fewer dependency seems like a good thing right up to the point where it becomes an absolute necessity to introduce it.

@Technifocal

This comment has been minimized.

Copy link

Technifocal commented May 20, 2017

@gordan-bobic what would prevent people from reverse engineering the compiled binary to acquire the secrets? Or simply intercepting the connection (As it's running on my PC, even using HTTPs wouldn't be that hard to read, considering I can read RClone's memory as it decrypts the traffic)? I mean, it's not like it's not done on a daily basis in the video game industry to create external memory modification programs to cheat, or simply bypass DRM and pirate content.

I understand Amazon's point of view on this one, they banned ACD_CLI, then people using that software started using another software's authentication credentials to continue using it. While ACD_CLI was not banned for its usage patterns (But instead a concurrency issue that leaked credentials), an app that's banned should be banned, and if the developer wishes to fix the issues and request their credentials back, that's up to the developer.

I'm obviously not saying that I agree with people doing this, I'm simply saying that if there's a will, there's a way, and there obviously is a will.

EDIT: Question, what stops someone authenticating with RClone (if they get their credentials back using a server-sided OAUTH system) then using the access token in ~/.rclone.conf to communicate with other software?

@gordan-bobic

This comment has been minimized.

Copy link

gordan-bobic commented May 20, 2017

@Technifocal the fact that very few people know how to do it. Yes, what you describe is indeed entirely viable (not getting into the exact attack vectors, but yes, it does work). But the fact remains there are quite a few other apps still out there that haven't been banned that avoid using authentication servers.

The entire "banning by app" concept is fundamentally flawed, though, because it is entirely ineffective. The sensible thing to do might be to simply alter the rclone back-end and make it fall back on web-scraping - and unless Amazon are going to kill the web interface they might as well give up at that point.

@JanHellwig

This comment has been minimized.

Copy link

JanHellwig commented May 20, 2017

EDIT: Question, what stops someone authenticating with RClone (if they get their credentials back using a server-sided OAUTH system) then using the access token in ~/.rclone.conf to communicate with other software?

I used the token from another software yesterday in order to download my data from ACD with Rclone. Worked perfectly fine except for the fact that refreshing the token didn't work automatically as the other software uses a different format for the expiry date.

@animosity22

This comment has been minimized.

Copy link
Contributor

animosity22 commented May 20, 2017

@poom - few months now. All the business plans have this and you just have to move a domain over. It took me a few minutes to get it setup.

@SirJMD

This comment has been minimized.

Copy link

SirJMD commented May 20, 2017

Just went to try out Google Drive.. horribly slow :( Getting around 800 kb/s on a 200 MBit/s fiber. What a fail. ACD managed 3-5 MB/s before they shut down access.

@gordan-bobic

This comment has been minimized.

Copy link

gordan-bobic commented May 20, 2017

Performance varies depending on time of day, where you are and luck. I had ACD struggle to achieve 3Mbit/s at times, and GD is managing to saturate my 200Mbit connection at the moment.

@SirJMD

This comment has been minimized.

Copy link

SirJMD commented May 20, 2017

@gordan-bobic Do you do anything to achieve that? GD is utter garbage for me.

Transferred: 2.274 GBytes (707.319 kBytes/s)
Errors: 0
Checks: 0
Transferred: 4
Elapsed time: 56m10.9s

@calisro

This comment has been minimized.

Copy link

calisro commented May 20, 2017

@SirJMD that is terrible. I've always had really good performance from gd. Must be the end point your connected to.

@calisro

This comment has been minimized.

Copy link

calisro commented May 20, 2017

@SirJMD I just maxed my 150 megabit connection with a single transfer to gd. Perhaps there is a way to change your end point.

@gordan-bobic

This comment has been minimized.

Copy link

gordan-bobic commented May 20, 2017

@SirJMD no special configuration, just straight rclone sync.

@SirJMD

This comment has been minimized.

Copy link

SirJMD commented May 20, 2017

Downloading is no problem.. So peering isn't the issue here I guess?

Transferred: 2.274 GBytes (13.883 MBytes/s)
Errors: 0
Checks: 0
Transferred: 4
Elapsed time: 2m47.7s

@mc40638

This comment has been minimized.

Copy link

mc40638 commented May 20, 2017

I am been monitoring what has been going on. As far as the 'cloud' is concerned (I prefer to say google and amazon servers) I, like the rest of you, have gotten some contradictory information. I am not a power user (have a mere 5 MB/s up from my ISP [Charter, :( ].

I was using ACD to archive files and to store image backups. Since my compressed backups are about 350 GB, a transfer to the cloud (at best) will take 6 days.

I started evaluating GoodSync (GS) (on Windows, though they have a free Linux CLI) and for the most part find the product competent. The overhead is not insignificant apparently, since when I installed rclone on a VM and used that to transfer files, I got better throughput. I suspect it is the multiple threads it uses by default for bulk copies/syncs.

Rclone stopped working (the 429 errors and then authentication errors) which have occupied this thread. At that point GS was still working but about 45% of the transfers failed (error 400 Bad Request). GS support gave me some BS and guaranteed they are using the official API. Sidebar: GS with GoogleDrive (GD) was working very well, just as it had been with ACD days earlier.

After numerous calls to Amazon Drive support (support is a loose word; they are not technically trained) I spoke to a gentlemen who knew about rclone being banned and about the acd_cli issues. He put me on hold and after about 10 minues came back with some startling news.

Apparently ACD will stop supporting ANY third party client application. I asked him to double check. In fact he looked up a closed ticket about the '400' errors in GS and mentioned that the amazon development team reviewed the ticket and said the errors were expected as they are phasing out third part clients.

He could not tell me if AWS was affected (but it uses the same framework as ACD) . He speculated that Amazon will eventually roll out a real client - he had no idea. I mentioned that AWS users (especially enterprise) might not tolerate this, and he did not have an answer.

I cannot verify any of this, though it is consistent with my observations.

I may exceed the 1TB limit of the GD $10/month offering. I was intrigued by the post of having a single user G Suite account with the observation that Google isnt enforcing the 1TB limit.

My current Google relationship:

I have a FREE google apps account with 10 licenses and 8 users. I have an upgraded GDRIVE account associated with an account outside of the google apps account domain. (Like G Suite, you must have a domain name). If I were to convert the goodle apps account to get unlimited storage (8 users x $5/month x 12 months/yr) the annual cost would be $480. WOW! (If they count licenses even more).

I could simply purchase a new domain for about $12 a year, and sign up for G Suite Business (1 user). Maybe the best way to go. Unfortunately I will have weeks of transferring data from one Google drive to another. I need to investigate that (GS can be setup to do this, but if buffers the file on my laptop - so I suffer the same horrendous upload speeds).

There seems to be other solutions out there, but this may be the best price point. There is a Canadian outfit SYNC.COM which has attractive pricing for 2TB, however are no clients I have found to access their servers.

Thanks all.

@left1000

This comment has been minimized.

Copy link

left1000 commented May 21, 2017

would it be possible for anyone to donate their official developer api to nick to use to get rclone back open? or is that pointless? because they'll just reban it unless they specifically ask nick to have a new token of his own?

@niranjanshr13

This comment has been minimized.

Copy link

niranjanshr13 commented May 21, 2017

@left1000 try reading this https://forum.rclone.org/t/guide-rclone-with-acd-using-drivesink-token/2421 you can get api to use in amazon drive on rclone. Look at tutorial, you have to get token every 1 hour, because of expiration.
Side Noe: In rclone format the token, with one space (remove all the <enter/return>).

@inquam

This comment has been minimized.

Copy link

inquam commented May 25, 2017

Amazon has indeed not just changed their API requiring changes in rclone but blocked rclone access.
I sent an inquery asking and if this was the case demanded a full refund since my entire use case was mounting with rclone to upload backups taken of my machines to be stored in the cloud.

This was their response:

Hello,

I understand your concern about using third party clients or apps to access Amazon drive.

Rclone app access has been revoked and it no longer supports Amazon Drive access. I'm sorry for this disappointment.

However, As requested, I've canceled the Unlimited storage subscription and issued full refund in the amount of $59.99.

Refunds are issued to the payment method used to make the original purchase and usually complete within a few business days. For more information, go to: www.amazon.com/help/aboutrefunds.

In the meanwhile, try using Amazon Drive Desktop application for Windows / Mac or use Amazon Drive website for other OS devices.

It is always important for us to hear how customers react to all aspects. Your valuable feedback will help us to improve the selection and service we provide and we're glad you took time to write to us. We do value the feedback of our customers and make every attempt to integrate their comments or suggestions into our stores and procedures.

We appreciate your patience and understanding as we work to resolve this issue.

For any further assistance you can reach us 24 hours a day, seven days a week using the following link:

https://www.amazon.com/gp/help/contact-us/cloud-drive.html

We look forward to your next visit.

We'd appreciate your feedback. Please use the buttons below to vote about your experience today.

Best regards,
Sadiq
Amazon.com

So if everyone who was using Amazon cloud drive along with rclone demands their money back they might start to listen.

@AggelosM

This comment has been minimized.

Copy link

AggelosM commented May 25, 2017

@inquam Post it on /r/DataHoarder

@Fmstrat

This comment has been minimized.

Copy link

Fmstrat commented May 26, 2017

@ncw Any chance you could lock this issue down for comments from anyone except contributors? It's gotten out of hand from a discussion standpoint (GitHub should be about the fix). That way users can monitor for new comments to get updates (from only you). Thanks for all the hard work!

@etomm

This comment has been minimized.

Copy link

etomm commented May 31, 2017

What @niranjanshr13 is really interesting and seems to replace the access token well? Why not following the way of drivesink to solve this case?

@duke-87

This comment has been minimized.

Copy link

duke-87 commented May 31, 2017

I've done the same as @inquam.
They returned my money no questions asked. In less than 25 minutes.

@newtonseple

This comment has been minimized.

Copy link

newtonseple commented Jun 3, 2017

@ncw Have you considered authenticating through/using go-acd? That project hosts an open auth server at go-acd.appspot.com.

@ncw

This comment has been minimized.

Copy link
Owner

ncw commented Jun 5, 2017

I'm going to close this issue now - for the resolution see https://forum.rclone.org/t/rclone-has-been-banned-from-amazon-drive/

@ncw ncw closed this Jun 5, 2017

@carlyuanliu

This comment has been minimized.

Copy link

carlyuanliu commented Jun 7, 2017

It looks like I will not be able to use rclone to upload/download files to/from acd at least for a while. What a shame.

I have been using rclone to upload large files and folders to acd, as the acd client is not reliable and puts a limit on the file size. Also, rclone allows me to obtain directory listings for search purposes, as acd does not provide directory listings nor does it do search properly with double byte characters (you would think these functions to be basic for cloud drives).

I thought about ditching acd, but I can't beat the price. For example, Google's price for 1TB exceeds acd's Unlimited Storage. Now that I have accumulated over 2TB on acd, I decide to keep it. With all its limitations, acd is still good for archiving purposes.

As for my large files (mainly videos), I'll just split them to within limits before uploading.

As to directory listing, I will obtain it before uploading. I have yet to find a solution to get existing files on acd. sigh...

@ajthemacboy

This comment has been minimized.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment