Error: Too Many Requests

[Heziode]

Hi

I get the folowings errors when I use the app: Too Many Requests

Logs:

[2018-11-22 09:28:01.053] [error] [WEBVIEW:protonmail] [index] [XMLHttpRequest.load handler] {"status":429,"statusText":"Too Many Requests","responseURL":"https://mail.protonmail.com/api/conversations/<token>"}
[2018-11-22 09:28:01.054] [error] [WEBVIEW:protonmail] [index] [XMLHttpRequest.load handler] {"status":429,"statusText":"Too Many Requests","responseURL":"https://mail.protonmail.com/api/conversations/<token>"}
[2018-11-22 09:28:01.055] [error] [WEBVIEW:protonmail] [index] [XMLHttpRequest.load handler] {"status":429,"statusText":"Too Many Requests","responseURL":"https://mail.protonmail.com/api/conversations/<token>"}
[2018-11-22 09:28:01.060] [error] [WEBVIEW:protonmail] [index] [XMLHttpRequest.load handler] {"status":429,"statusText":"Too Many Requests","responseURL":"https://mail.protonmail.com/api/conversations/<token>"}
[2018-11-22 09:28:01.060] [error] [WEBVIEW:protonmail] [index] [XMLHttpRequest.load handler] {"status":429,"statusText":"Too Many Requests","responseURL":"https://mail.protonmail.com/api/conversations/<token>"}
[2018-11-22 09:28:01.060] [error] [WEBVIEW:protonmail] [index] [XMLHttpRequest.load handler] {"status":429,"statusText":"Too Many Requests","responseURL":"https://mail.protonmail.com/api/conversations/<token>"}
[2018-11-22 09:28:01.089] [error] [WEBVIEW:protonmail] [index] [XMLHttpRequest.load handler] {"status":429,"statusText":"Too Many Requests","responseURL":"https://mail.protonmail.com/api/conversations/<token>"}
[2018-11-22 09:28:01.090] [error] [WEBVIEW:protonmail] [index] [XMLHttpRequest.load handler] {"status":429,"statusText":"Too Many Requests","responseURL":"https://mail.protonmail.com/api/conversations/<token>"}
[2018-11-22 09:28:01.090] [error] [WEBVIEW:protonmail] [index] [XMLHttpRequest.load handler] {"status":429,"statusText":"Too Many Requests","responseURL":"https://mail.protonmail.com/api/conversations/<token>"}
[2018-11-22 09:28:01.135] [error] [WEBVIEW:protonmail] [index] [XMLHttpRequest.load handler] {"status":429,"statusText":"Too Many Requests","responseURL":"https://mail.protonmail.com/api/contacts/<token>"}
[2018-11-22 09:28:01.136] [error] [WEBVIEW:protonmail] [index] [XMLHttpRequest.load handler] {"status":429,"statusText":"Too Many Requests","responseURL":"https://mail.protonmail.com/api/contacts/<token>"}
[2018-11-22 09:28:01.136] [error] [WEBVIEW:protonmail] [index] [XMLHttpRequest.load handler] {"status":429,"statusText":"Too Many Requests","responseURL":"https://mail.protonmail.com/api/contacts/<token>"}

app verison: 2.0.0-beta-6

electron: 3.0.9
chrome: 66.0.3359.181
node: 10.2.0
v8: 6.6.346.32

[vladimiry]

Does it happen when you first time enable the local store feature? The app does an initial data load when the local store is enabled the first time for the account. At the moment app does the fetching without any delays and even more, currently the app runs up to 3 requests of this kind /api/conversations/<token> in parallel. It was in general anticipated that Protonmail has some sort of rate limiting enabled at the server side, but it was not triggered on the about 150 conversations case when I was developing the feature, so I guess you have significantly more conversations in your account?

If the guess is correct, then there will be a need to enable delay between requests, which will slow the data load process down but will let it be successfully completed. The thing is that Protonmail doesn't expose its rate limiting settings information and that's understandable. So I think I better disable a parallel loading and enable the configurable delay in config.json letting you play around the value.

[Heziode]

Yes I have around 500 mails archived.

[vladimiry]

Can you try the build uploaded here?

It enables the 250 requests per 1 minute rate-limiting (config.json file):

    "fetchingRateLimiting": {
        "intervalMs": 60000,
        "maxInInterval": 250
    },

If it still fails with the same error, you can adjust the settings in order to make them more strict by editing config.json (exit the app => edit the file => wait for 1/few minutes, so protonmail's limits get reset => start the app). For example starting from changing "maxInInterval": 250 to "maxInInterval": 200. Unfortunately, we don't know the rate-limiting settings Protonmail has enabled for API requesting, so we have to be creative. Once we found proper settings, I will update the default values.

Besides, keep in mind that this is per account rate-limiting. So if you have many accounts I'd suggest adding them to app (or enabling local store feature) sequentially, one by one, so bootstrap fetch doesn't run in parallel.

The initial/bootstrap fetch will work longer with this rate-limiting thing, but it's one-off case since the consequent syncing occurs in incremental mode, ie app fetches the changes diff only.

[Heziode]

I have tested with you release.

I have 4 protonmail accounts. I have disabled the local storage settings for each one except for the main wich contains more than 500 mails.
With you default configuration I do not have any problem to synchronize the local storage.

[vladimiry]

Thanks for confirming issue resolving, so I keep the default configuration for the final v2 release. The fix is included in just published v2.0.0-beta.7 release.