Are we safe? Amazon Cloud Drive

[scriptzteam]

I mean could be there the same scenarion that they disable rclone app from amazon? or does the rclone handle it other way than acd_cli did?

ACD_CLI weird:

yadayada/acd_cli#562 - "I created this pull request only to ask what happend to acd_cli's issues page?! It just vanished! "

[ncw]

It is always a possibility. However I've had various email conversations with the amazon developers in the past and I don't think they would do that without a dialogue first.

[ncw]

I was distressed to see on that thread a discussion of how to use rclone's keys to restore service. Please don't do this - it will likely lead to rclone being banned in the same way.

[mizzuri]

So, apparently there's a workaround for acd_cli using rclone's application id and secret. Won't this cause a negative effect on rclone?

Edit: Link here.
Edit 2: Now that stuff on ACD is encrypted using rclone, I'm worried that we won't be able to retrieve and decrypt the backup. :(

[ncw]

Edit 2: Now that stuff on ACD is encrypted using rclone, I'm worried that we won't be able to retrieve and decrypt the backup. :(

Provided you can retrieve the files somehow, you can decrypt them with rclone locally very easily.

[mizzuri]

Provided you can retrieve the files somehow, you can decrypt them with rclone locally very easily.

You are right. But it would still require downloading the whole directory just to get one file, right?
Because we will no longer be able to rclone lsl remote-crypted:directory

Nevertheless, it's still a peace of mind that there's still a way.
Thank you.

[fuyar]

Do you have a clue of why acd_cli key has been forbidden ?

API hammer or ToS issues or that thing about being able to access other people drive ?

[ncw]

You are right. But it would still require downloading the whole directory just to get one file, right?
Because we will no longer be able to rclone lsl remote-crypted:directory

Assuming that there existed another tool that could mount acd, then you could set it up so the above would work just fine.

[scriptzteam]

Just an update about acd_cli and amazon reply:
yadayada/acd_cli#562 (comment)

Thank you for providing more information.

We investigated the security issue you reported regarding Amazon Drive (reference HGXXXXXXX).

Our investigation did not reveal any incorrect handling of session tokens in our systems. However, we also reviewed the source code of a third-party application used to authenticate to Amazon Drive (https://tensile-runway-92512.appspot.com/src) and think that there is a concurrency issue present in their code which could result in a customer receiving incorrect authentication tokens.

To ensure our customers safety, we have disabled access to Amazon Drive from the third-party application "acd_cli_oa”.

We appreciate you reporting this security issue.

[mizzuri]

@scriptzteam .. So.. Does it mean that another (abusing?) app was using acd_cli's security tokens, and thus acd_cli was banned because of it? I'm a bit lost here.

[Rufflewind]

@unnfav No, https://tensile-runway-92512.appspot.com/src is acd_cli’s default authentication server. Amazon’s investigation indicates that it’s buggy and can sometimes leak tokens to the wrong user.

[mizzuri]

@Rufflewind .. Thanks for the clarification.

[ncw]

@Rufflewind wrote

No, https://tensile-runway-92512.appspot.com/src is acd_cli’s default authentication server. Amazon’s investigation indicates that it’s buggy and can sometimes leak tokens to the wrong user.

Just so everyone is clear, rclone doesn't use an external authentication server - it is all done on your computer. That is why the signup for Amazon Drive (and all the oauth providers like Google etc) is a little awkward.

I didn't want to have the responsibility of other people's credentials going through my server so I deliberately crafted rclone so that wasn't necessary. Your credentials never leave your computer!

[scriptzteam]

when i firstly installed acd_cli and there was that 3-rd party auth needed i know it was unsafe, thus i never used, i know you can trust, BUT WH THE FUCK I SHOULD TRUST SOMEONE SH*T CODE thats hosted on apppot.com lol, he could cache everything ;) thats why i preffered rclone, as @ncw said and THIS MATTER TO ME -> "Just so everyone is clear, rclone doesn't use an external authentication server - it is all done on your computer. That is why the signup for Amazon Drive (and all the oauth providers like Google etc) is a little awkward.

I didn't want to have the responsibility of other people's credentials going through my server so I deliberately crafted rclone so that wasn't necessary. Your credentials never leave your computer!"

[boblatino]

I think they also removed the rclone key:

2017/05/18 06:43:17 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2017/05/18 06:43:17 DEBUG : HTTP REQUEST (req 0x10fea400)
2017/05/18 06:43:17 DEBUG : GET /drive/v1/account/endpoint HTTP/1.1
Host: drive.amazonaws.com
User-Agent: rclone/v1.36
Authorization: XXXX
Accept-Encoding: gzip

2017/05/18 06:43:17 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2017/05/18 06:43:17 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2017/05/18 06:43:17 DEBUG : HTTP RESPONSE (req 0x10fea400)
2017/05/18 06:43:17 DEBUG : HTTP/1.1 429 Too Many Requests
Cache-Control: no-store
Connection: keep-alive
Content-Type: application/json
Date: Thu, 18 May 2017 06:43:17 GMT
Pragma: no-cache
Server: Server
Vary: User-Agent
X-Amzn-Errortype: TooManyRequests:http://internal.amazon.com/coral/com.amazon.clouddrive.service/
X-Amzn-Requestid: xxxxx

I have contacted amazon and they told me that rclone was disabled today :(

[mizzuri]

Sadly, I can also confirm that rclone is no longer working now. :((

[l3uddz]

If it was disabled, why is it still in Third Party Apps in Amazon Drive account settings? acd_cli was removed from there for me without me doing anything, rclone is still there. Seems to suggest some kind of api limit was reached perhaps?

[jdrydn]

Same here. 429s, 429 everywhere. It's sad because after acd_cli's downfall (and subsequent unusual silence) I'd decided to move to B2, except now that's going to be extremely difficult given rclone was my choice of tool to move between the two services! 😢

[ncw]

I think maybe rclone's quota has been reduced. I'm contacting Amazon to find out what is happening.

[geekcroft]

Good luck ncw 👍

[DIKU-DK-3000]

Having the "429 Too Many Requests" posts as well.

When typing # rclone lsd ACD:ACD_Encrypted in terminal I get:

2017/05/18 11:24:30 Failed to create file system for "ACD:ACD_Encrypted": failed to get endpoints: HTTP code 429: "429 Too Many Requests": response body: "{\"message\":\"Rate exceeded\"}"

[Julz2k]

Problems started for me two days ago, files could not be uploaded correctly (error 400) even after 10 tries and uploading over 800GB for the bin.

Since this morning I also face the 429 too many requests message. Sad 👎

[geekcroft]

User on the Rclone forum reporting that Amazon Support have told them that its a ban on rclone and its permanent :/

Shall wait to see what @ncw says after talking with Amazon :(

[Em31Et]

Is it because rclone don't have cache and spam amazon api for processing?

[DIKU-DK-3000]

"banned"?!?

I sure hope not, but let's see what @ncw can report back with.

[jdrydn]

@Em31Et It'll likely be because all the acd_cli users moved over to rclone when the former was banned from the ACD API, and sounds like rclone is going in a similar direction (although thank you @ncw for not being radio-silent!)

[JackDash]

Just wait for ncw's answer. It wouldn't be the first time that Amazon support gave wrong info and certainty not the first time someone would make something up.

[Em31Et]

@jdrydn
acd_cli users is the reason, but I think the main reasons is because of rclone's implementation.
rclone do not have cache and keep on retrying the api calls.

[gdomod]

its possible to make a own rclone app and register with acd ?
i thought there was too much hits from one app.

[Em31Et]

@gdomod
No, amazon do not accept new apply anymore.

[henryford]

@zawlin I'd advise against that. At this current point I'd wager that they disabled access for rclone precisely because acd_cli clients used its credentials to connect. I think it still says something that rclone is still showing up on the access page on acd.

[Krandor1]

Right The fact they haven't flat out told ncw straigth no and it is still on the access page are good signs. No need to do anything to antagonize them.

[ajkis]

I'd advise against that. At this current point I'd wager that they disabled access for rclone precisely because acd_cli clients used its credentials to connect

Yea I agree but we all know acd_cli guys will rip other app keys until amazon bans them all :/

[bgemmill]

@ajkis I'm one of the "acd_cli guys" and we care about the outage just as much as you do. The difference in time between our outage and yours was short enough that blaming things on malice seems hasty.

On our side of the fence we have a few users who kindly donated white-listed security profiles, but we're waiting to see what the official response is in case they get banned straight away.

We're waiting for word from @yadayada and you should wait for @ncw, and when one gets word we can all benefit.

In the mean time the only real protest is to set your acd subscription to not renew and hope they notice.

[itsrainingben]

chiming in with a 429 error code when listing directories via rclone.

i just got off with ACD help desk, she told me Amazon banned rclone. in asking why Amazon wouldn't notify users or developers, her response from 'leadership' was that they actually weren't clear who (ACD or rclone) killed it.

just gonna sit on my hands and monitor this thread for an update from @ncw

[techknowlogick]

@itsrainingben: @ncw has given an update via twitter https://twitter.com/njcw/status/865319897580097537

[itsrainingben]

@techknowlogick appreciate that

[gordan-bobic]

Several people have reported getting refunds on their ACD subscriptions, and I can confirm I just got mine. That seems to imply that ACD is burned for rclone. It may be an idea to start looking at pruning the bug list of ACD specific issues/features, and looking at removing the ACD back-end from rclone.

[rpgdev]

@gordan-bobic Amazon almost never denies a refund so this is hardly indicative of anything. @ncw hasn't received an answer from ACD team yet so removing anything ACD related from rclone is premature. His most recent tweet related to the issue: https://twitter.com/njcw/status/865583044870189056

[nextime]

I don't know if it has been already mentioned, and i'm not in any relation with it, i'm just a customer, but for me the alternative service that i'm using to sobstitute my ACD storage i was using with rclone until few days ago is Hubic, the OVH cloud storage: 50 eur/year for 10 terabyte of storage and it works great with rclone.

[Thinkscape]

Hubic is speed capped to just 10mbps. It's in the fine print.

On Mon, 22 May 2017 at 9:51 am, Franco (nextime) Lanza < ***@***.***> wrote: I don't know if it has been already mentioned, and i'm not in any relation with it, i'm just a customer, but for me the alternative service that i'm using to sobstitute my ACD storage i was using with rclone until few days ago is Hubic, the OVH cloud storage: 50 eur/year for 10 terabyte of storage and it works great with rclone. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#1409 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAQgwAx2hMcY8kVA6BtHI4Su9muMP_Aeks5r8M3jgaJpZM4Nbiqz> .

-- Artur Bodera abodera@gmail.com http://thinkscape.pro

[jdrydn]

@Thinkscape is correct, I've just checked the Hubic "General conditions of use":

The bandwidth is limited to 10 Mbit/s upstream and downstream. The connection speed also depends on the quality of the Customer's internet connection.

And

OVH cannot guarantee that files exceeding 5 GB will be deposited. In the event of needing to put a file of this size online, OVH asks the Customer to contact the technical support team in order to find out the most effective method of carrying out this operation.

🤦‍♂️

[AiMAnsarie]

So is there a cli on Ubuntu other than acd_cli and rclone?

[Tvax]

What should we use now?

[gordan-bobic]

Google Drive and HubiC seems to be what most people I know are migrating to.
Last I checked there were no fuse-mountable alternatives to acd_cli and rclone.

[AiMAnsarie]

Any change in situation?

[vampywiz17]

Google Drive and HubiC seems to be what most people I know are migrating to.
Last I checked there were no fuse-mountable alternatives to acd_cli and rclone.

I thinking HubiC... It is fast?

[michael-k]

@AiMAnsarie https://forum.rclone.org/t/rclone-has-been-banned-from-amazon-drive/2314

Update 2017-05-30

I've finally heard back from Amazon about my request for new developer credentials (after 10 days!) to build an auth server

We have ended the Amazon Drive API and SDK invitation period to focus on enabling new customer experiences with current developers. Should this change, we will provide an update on our developer website: https://developer.amazon.com/amazon-drive140

So I guess that is a no 😦

[toddfries]

HubiC is sluggish and sometimes has auth issues randomly for no reason. I am currently paying < $5/mo for its 10T service, and treat it as a 'once I get data there, it is good' service not a 'I will wet my pants if I cannot get data uploaded 3s after I want it there' service. I'd really prefer Amazon Drive (which is the same storage mechanism as Amazon Photo Storage, free unlimited photos with amazon prime), but of course the current fiasco of rclone being banned will have to be sorted. If I can help I hope to figure out how. In the mean time, it would be nice to consolidate all the duplicate issues .. #1429, #1420, #1417, #1415 ...

[gordan-bobic]

@toddfries from what I can tell, Amazon revoked app keys from many apps, not just rclone and acd_cli, and from what I hear, they are not going to be issuing any new app keys. So I think ACD is gone for good as a useful cloud storage service.

I suspect that they underestimated both the popularity of the service and the amount of space abusers of the "unlimited" storage capacity were using.

[toddfries]

Should be noted that AWS still works (for now???)

[gordan-bobic]

@toddfries What exactly do you mean? AWS != ACD.

[mc40638]

AWS-Amazon Web Services which spans the gamut from individual users to very large enterprise clients. ACD-Amazon Cloud Drive (much like Google Drive).

ACD is built on the AWS framework but is a separate entity. AWS access is different and will depend on each organization using a published and accepted API. I would think there are commercial clients as well.

ACD has a limited number of acceptable clients, and the last time I talked to Amazon, eventually only clients publish by amazon will be available. This may not be true; just what they told me.

[mizzuri]

@gordan-bobic

What exactly do you mean? AWS != ACD.

I think @toddfries meant Amazon S3.

[ncw]

I'm going to close this issue now - for the resolution see https://forum.rclone.org/t/rclone-has-been-banned-from-amazon-drive/

[marvinpinto]

@ncw Hi! Is the resolution essentially "rclone can no-longer be used with Amazon Cloud Drive?"

(That thread was quite long and Discourse does not make it very easy to search or navigate)

[rpgdev]

@marvinpinto there is an update in the first post, you don't have to scroll the entire thread.