Appspot users may get throttled starting September 23rd

[yadayada]

This is a public service announcement.

Please adjust your Amazon Drive API usage

Thank you for integrating your app with Amazon Drive.

We noticed abnormal call patterns from your app into Amazon Drive's APIs. Your customers are calling the getChanges method up to 2,900 times per hour; higher than what we saw when we approved your app. We recommend that you call getChanges a maximum of 15 times per hour per customer. In order to maintain our service performance at optimal levels for all of our customers, we will throttle your app at developer levels starting on September 23rd, 2016.

If you would like the throttle removed, please re-submit your app for approval.

Best regards,
Amazon Cloud Drive 3P Support

I do not know what it means exactly for the app to be on developer limits. [This is not for a lack of asking; Amazon would not tell me what limitations there are.]

So, if any acd_cli customers experience issues starting on the 23rd, I suggest creating your own security profile.

[msh100]

Your customers are calling the getChanges method up to 2,900 times per hour

Does this mean there's a single user calling getChanges 2,900 times an hour? or all users combined?

It sounds like the former. They recommend limiting it to 15, maybe time to add some throttle/burst logic into acd_cli itself?

[EldonMcGuinness]

What makes the getChanges call, is it just the sync command? Is there anything users can do to lower the number of calls to the service?

[yadayada]

Does this mean there's a single user calling getChanges 2,900 times an hour? or all users combined?

I'm not sure anymore, but judging from the number of roughly 17,500 requests per day to the App Engine instance there should be 730 users per hour. So, that's 4 requests per hour per user on average and already well below 15 rph.

What makes the getChanges call, is it just the sync command?

Yeah, it's the sync action and the autosync function that's disabled by default.

They recommend limiting it to 15, maybe time to add some throttle/burst logic into acd_cli itself?

Limiting the (auto)sync command to run only once per 4 minute would be fairly easy, but getting every user to update is another thing.

[Axcy]

How hard would it be to change your app key and disable the old one? This could be a way to force people to upgrade perhaps?

[a5m0]

@yadayada or anyone else would you be able to provide updated steps for how to get our own security profile working? I am able to create one but I don't see how to get anything about ACD working with it... like there seems to be no ability to apply for ACD api access to the new security profile?

[breunigs]

@a5m0 request an invitation: https://developer.amazon.com/public/apis/experience/cloud-drive

[a5m0]

I did this a couple weeks ago, has anyone actually received an email back?

On Sun, Sep 25, 2016, 2:14 AM Stefan notifications@github.com wrote:

@a5m0 https://github.com/a5m0 request an invitation:
https://developer.amazon.com/public/apis/experience/cloud-drive

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#416 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AExEejZ-Ar-Dx5f_A6Vv0A6g4ZfUgn8lks5qthFbgaJpZM4KBQjW
.

[bgemmill]

@yadayada looking at bug #400 what is the correct way of dealing with the end not reached for change requests now that incomplete syncs need to be spaced by multiple minutes to make Amazon happy?

I've switched over to old syncing for my uses since I can't seem to get a complete incremental sync at all.

[Craeckie]

@a5m0 I received this response some time last month:

Thank you for your interest in using the Amazon Drive developer tools. We need more information in order to consider your request. Please let us know more about your application and/or future plans for using the Amazon Drive developer tools.

• What is the use case you plan to support with your app?
• What type of customers, small business, or enterprise does your app support?
• How frequently will your app need to make requests to our service?
• Please provide a link to your website and link to download your app.
• Please provide your phone number in case we have additional questions.

Best Regards,

The Amazon Drive Team

I answered them several weeks ago without any response..

[vx28643]

I just received this same email and answered as truthfully as possible, that I want to whitelist my security profile for personal file backup, giving the link here to acd_cli

Will see what happens.

[2aee1c40199c7754da766e61452612cc]

@vx28643 Any update at the Amazon side? I replied weeks ago. No answer to the moment.

[vx28643]

@maiaftv no, I will post here when I get any reply. They took a few weeks to reply to my initial request, so I will be patient.

[vx28643]

Just received a response as follows:

Thank you for your interest in using the Amazon Drive developer tools. We consider a variety of factors as we evaluate each invite request, including the use case your application will support. While our goal is to offer a broad and compelling assortment of applications to Amazon Drive customers, we are currently unable to approve your request for access.

I suppose they don't want to grant api access for developers wishing to use acd for personal file backup. Will report back if I am able to change their mind...

[bgemmill]

@yadayada should we be defaulting to old-sync if an incremental one fails? There are examples in the wild of people incremental syncing in tight loops to make it work when #400 strikes them:
jlippuner/acd_cli@cc05b38

[a5m0]

Received the same negative response email as @vx28643

[2aee1c40199c7754da766e61452612cc]

@vx28643 and @a5m0, same thing here. Denied.

[vx28643]

I suggest those who are receiving negative responses should all be replying to Amazon, asking them to reconsider and pointing them to this page. By them visiting this page they may see that they are turning away some potential paying customers.

I mean, we are all wanting to be paying customers of Amazon Drive and have a legitimate reason for wanting to use the API for our personal file storage and backups.

As I hope to be a long time customer of Amazon Drive I would much prefer to whitelist my own security profile than simply rely on the shared Appspot profile that acd_cli provides.

By helping Amazon to understand our use case perhaps they will see a way to change their mind.

I will reply to them and post here if anything changes.

[2aee1c40199c7754da766e61452612cc]

I guess they do not want people like us using their services, @vx28643. Being able to mount and upload huge amounts of encrypted data could not be what they had on mind for the business plan of Amazon Cloud Drive.

If you get any response, please update this issue.

[vx28643]

@maiaftv I realize this and while amazon promote Amazon Drive as "Unlimited Storage" for $59.99 per year I assume they must have some kind of throttling in place for unreasonable users.

That seems to be what triggered the problems with the Appspot profile that acd_cli provides, and is one of the reasons I feel the need for my own security profile if I were going to use Amazon Drive as part of my long term backup plan.

You are probably right - but I thought it worth asking Amazon to have another look before I completely give up on Amazon Drive as a backup option. I suppose we will see what happens. I can only speak for myself but my uses would be ~1TB total storage, not huge amounts.

Will update if I hear something. Cheers.

[vx28643]

Tired of waiting for Amazon so have opted for one of the affordable backup/storage hosting plans to be found on www.lowendtalk.com I was able to find one at less than $50/year for 1TB. Will update if I do hear anything from Amazon

[gdragotto]

@yadayada Actually, after the --RequestError 429 "Rate Exceeded--, i can't access to Amazon api both with acd_cli init and custom security profile.
What i get is

We're sorry!

An error occurred when we tried to process your request. Rest assured, we're already working on the problem and expect to
resolve it shortly.
Show Details

Error Summary

400 Bad Request
Unknown client_id

[Keeperjoey]

Seems like acd_cli has been banned or maybe a rate limit

[bgemmill]

acdcli is up on my end, so probably not an outright ban. If you were getting 429s, I've observed amazon shutting me out for a length of time after a high number of operations. You could be in a state like that; I'd wait a day and see if it's still down.