New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Restrict access of public CORS Anywhere demo #45

Open
Rob--W opened this Issue May 26, 2016 · 3 comments

Comments

Projects
None yet
2 participants
@Rob--W
Owner

Rob--W commented May 26, 2016

The CORS Anywhere demo is hosted at Heroku. I've been informed that the app is causing performance issues on the platform and that hosting an open proxy is against the Acceptable Use Policy:

  1. Use the Service to operate an "open proxy" or any other form of Internet proxy service that is capable of forwarding requests to any End User or third party-supplied Internet host;

I'll be monitoring the traffic in the coming period, and blacklist all origins/destinations that have excessive usage demands, and/or automatically block sites that have more than X requests per Y seconds, unless explicitly whitelisted.

These measures are needed to 1) resolve the performance issues on Heroku platform 2) prevent abuse of the open CORS Anywhere proxy 3) Ensure that the CORS Anywhere demo can continue to operate for (small) sites who want to try out CORS Anywhere before hosting it themselves.

Rob--W added a commit that referenced this issue May 30, 2016

Add rate-limiting functionality #45
- Add checkRateLimit option to the API.
- Extend the default server.js with environment variables
  CORSANYWHERE_WHITELIST (re-using originWhitelist) and
  CORSANYWHERE_RATELIMIT (using the new checkRateLimit option)
  to make it easy to enforce usage limits.
- Document that Heroku doesn't want open proxies.
@Rob--W

This comment has been minimized.

Owner

Rob--W commented May 31, 2016

I have now imposed some rate limits.

With some exceptions (e.g. localhost because it's used for development and blocking localhost altogether doesn't make sense), you can only make 600 requests per origin per hour. I came to this number by capturing Origin headers for a whole hour, 5 times (in the weekend and weekday, at different times on a given day) (resulting in 1741K requests from 2.9k unique origins, excluding previously blacklisted origins). The data shows that:

  • 95% of all sites are not affected by the limit.
  • 88% of the requests are dropped (this means that 5% of the sites were using most of the resources of CORS Anywhere)

600 requests per hour is a royal limit. If you need more, host your own instance of CORS Anywhere (see https://github.com/Rob--W/cors-anywhere#demo-server for instructions). If this limit turns out to be too permissive, the limit may be tightened even more.

Here are some statistics based on the data.

Max requests per hour Origins not affected by the limit (% of all) Requests from sites affected by the limit (% of all)
4000 99% 66%
3600 98% 68%
3000 98% 71%
2000 98% 75%
1000 96% 83%
800 95% 85%
700 95% 86%
600 (current limit) 95% 88%
500 94% 89%
400 92% 92%
300 91% 93%
200 89% 95%
100 86% 96%
90 85% 96.6%
80 84% 96.9%
70 83% 97.2%
60 82% 97.5%
50 80% 97.8%
40 79% 98.1%
30 76% 98.4%
20 71% 98.8%
10 62% 99.3%
9 60% 99.4%
8 59% 99.4%
7 56% 99.5%
6 55% 99.5%
5 50% 99.6%
4 48% 99.7%
3 41% 99.8%
2 37% 99.8%
1 20% 99.9%
0 0% 100.0%

I captured another 1 hour of requests after setting the limit to 600 per hour, and got the following actual results (excluding blacklisted sites):

  • 82 out of the 1348 origins were rate-limited (6%).
  • 94k out of 321k requests were accepted (29%) (so 227k were blocked, 71%).
  • 49k of all accepted requests were from rate-limited origins (15% of all, 52% of accepted requests)
  • 1.5k of all accepted requests were from origins that are allowed to make an unlimited number of requests (0.5% of all requests, 1.6% of accepted requests)
  • 43.5k of all accepted requests came from origins that were not affected by the rate limit (13.6%)

These actual findings are within the expectations from the original investigation, so I'll stick to these limits for now.

And here are some more healthy statistics from the past hour (similar to the analysis from #42 (comment)):

Error codes:

     70 H12
      4 H15
   5528 H27

Status codes (including requests that were immediately blacklisted and not visible in the above stats):

 101253 200
    174 206
   1300 301
     87 304
   3925 400
 662592 403 <-- Most of them are blacklisted
   3607 404
 226238 429 <-- Rate-limited
   5528 499
     24 500
     74 503
(total # of requests in 1 hour: 1004802)

Rob--W added a commit that referenced this issue Dec 1, 2017

Version 0.4.1
- Add ability to rate-limit/block requests by origin (#45).
- Avoid crashing on invalid HTTP status codes (#95).
- Support and test coverage for latest Node.js versions (up to 9).
- Support `Access-Control-Max-Age` via `corsMaxAge` option.
- Listen on `0.0.0.0:8080` by default instead of `127.0.0.1:8080`.
  (use `HOST` and `PORT` environment variables to override this).
- Update gTLD list.
@tstibbs

This comment has been minimized.

tstibbs commented Sep 5, 2018

Are file:/// origins affected by the limits or are they excepted?

@Rob--W

This comment has been minimized.

Owner

Rob--W commented Sep 5, 2018

@tstibbs Excepted, at my discretion.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment