Rough sketch of user quota and bandwidth limiting #1118

kuenishi · 2015-04-06T08:02:05Z

Still no tests
only applies for object and object part uploads / gets
riak_cs_quota is a behaviour where programmers can make thier own module (see callback definition)
riak_cs_simple_quota is a simple quota limiter which takes care of Object puts
riak_cs_bw_limitter is a stupid bandwidth / access rate limitter on objects
they are only controllable from attached console: type riak_cs_simple_quota:help() for detail usage
to use them, write {quota_modules, [riak_cs_simple_quota, riak_cs_bw_limitter]} in advanced.config
to change quota modules, config change and restart are required
All data size units are in Bytes.

For @joecaswell 's review.

Remaining works or notes on usage:

separate owner and accessor case: bandwidth and access is for accessor, whild owner is for usage
retrieving user's latest storage usage from moss.storage won't scale for many CS nodes
add tests
Add some cool http headers
Multibag

kuenishi · 2015-04-06T08:21:12Z

Quota

simple idea of quota

Bahaviour named riak_cs_quota that define callback spec
Plugin-architecture
Live users' usage, access and bandwidth monitoring
Limits of users' usage, access and bandwidth

Use cases

Stop upload as Quota over
Stop upload as too much bandwidth
Stop upload as too much request rate, rate limitation

-> three infos stored in moss.users? no nah

Quota => on bucket owners

** For usage: cluster-wide / soft quota / both user and bucket level / persistent state
** For bandwidth and counts: Node-wide / soft quota / both user and bucket level / non-persistent

check hook point is right after policy chedck
update hook point is right after in riak_cs_access_log_handler

kuenishi · 2015-04-06T08:26:09Z

Other notification style on rate limiting / quota: below are all quotations -

Twitter API

Note that these HTTP headers are contextual. When using app-only auth, they indicate the rate limit for the application context. When using user-based auth, they indicate the rate limit for that user-application context.

X-Rate-Limit-Limit: the rate limit ceiling for that given request
X-Rate-Limit-Remaining: the number of requests left for the 15 minute window
X-Rate-Limit-Reset: the remaining window before the rate limit resets in UTC epoch seconds

When an application exceeds the rate limit for a given API endpoint, the Twitter API will now return an HTTP 429 “Too Many Requests” response code instead of the variety of codes you would find across the v1’s Search and REST APIs.

Foursquare

Please note that rate limits are not per endpoint, but per top-level endpoint group. For example, in an hour, if you make 2,500 userless requests to venues/search and 2,500 userless requests to venues/explore, you would have exhausted your venues/* rate limits for that hour; however, you would still have 500 requests left for tips/*.

Generally rate limits follow the rules above; however, some endpoints may have different limits due to security or other considerations. For the most current and accurate rate limit usage information for a particular request type, inspect the X-RateLimit-Remaining and X-RateLimit-Limit HTTP headers of API responses.

If you are currently over limits, our API will return a 403 error, and the response object returned by our API will be empty. We will also include a X-RateLimit-Reset header in the response, which is a timestamp that corresponds to when your rate limits will reset.

Etsy

Headers: Every API response bears rate limit headers, as described below:

X-RateLimit-Limit: 10000
X-RateLimit-Remaining: 9924

These default rate limits are subject to change at any time, without notice.

Calais

Cse releasenote update

kuenishi · 2015-04-09T02:33:01Z

please dialyzer
Change status code to 429 for bandwidth / rate limit
Maybe change status code to 507 for full quota?

They're done.

kuenishi · 2015-04-09T07:52:35Z

r_t passed for me. I believe it's ready for review.

shino · 2015-04-09T08:23:31Z

src/riak_cs_quota.erl

+%% This is experimental feature. These interfaces and behaviousr
+%% may or may not change until general availability.
+%%
+%% To use 'inherited' your own module, it should be configured in


What does inherited mean?

Behaviour implementation.

kuenishi · 2015-04-10T03:36:28Z

I need more to please dialyzer

kuenishi · 2015-04-10T07:31:15Z

I think this is ready for review again. Will run r_t next week, and if we agree on basic ideas, then I'll squash commits and create a cleanly rebased pull request.

shino · 2015-04-13T08:39:26Z

src/riak_cs_simple_quota.erl

+                Len when is_integer(Len) ->
+                    case Method of
+                        'PUT' -> Len;
+                        'DELETE' -> - Len


DELETE request length is almost zero.

shino · 2015-04-13T09:26:54Z

src/riak_cs_simple_quota.erl

+get_latest_usage(_Pid, _User, _, N, N) -> {error, notfound};
+get_latest_usage(Pid, User, EndSec, N, Max) ->
+    End = calendar:gregorian_seconds_to_datetime(EndSec),
+    ADayAgo = calendar:gregorian_seconds_to_datetime(EndSec - 86400),


If hard coded 86400 is meant for the default value of storage_archive_period, using application environment value would be better.

shino · 2015-04-13T09:32:49Z

Both bw and quota limiter modules work well, great! ☀️
I will 👍 after above comments are addressed and dialyzer failure is fixed.

kuenishi · 2015-04-13T11:19:52Z

Updated ^^;

Detailed storage stats Reviewed-by: kuenishi

shino · 2015-04-14T05:37:02Z

src/riak_cs_simple_quota.erl

+get_latest_usage(Pid, User) ->
+    Now = calendar:now_to_datetime(os:timestamp()),
+    NowASec = calendar:datetime_to_gregorian_seconds(Now),
+    LeewaySeconds = riak_cs_gc:leeway_seconds(),


riak_cs_storage:archive_period() ?

shino · 2015-04-14T05:37:43Z

Added the last comment 😅

kuenishi · 2015-04-14T05:53:11Z

oh, conflicts....

Conflicts: src/riak_cs_config.erl src/riak_cs_storage_d.erl

kuenishi · 2015-04-14T05:56:53Z

Conflicts resolved.

Rough sketch of user quota and bandwidth limiting Reviewed-by: shino

kuenishi · 2015-04-14T06:39:37Z

@borshop merge

Revert #1118, #1120 and #1123 from release notes Reviewed-by: shino

Basho-JIRA · 2015-08-14T05:30:00Z

Release note already written: see an item starting "Module-level hook point for .." in https://github.com/basho/riak_cs/blob/develop/RELEASE-NOTES.md#additions

_[posted via JIRA by Kota Uenishi]_

Drew Pirrone-Brusse added 3 commits April 3, 2015 11:12

Add note on app.config name changes.

ded7542

Improve legacy app.config note

accac9d

Copy edits

7d78868

kuenishi added this to the 2.0.1 milestone Apr 6, 2015

Drew Pirrone-Brusse and others added 2 commits April 6, 2015 12:51

Remove Riak notes from Riak CS notes.

7c29d43

Merge pull request #1119 from basho/cse-releasenote-update

e79fcf6

Cse releasenote update

Handle storage calc error message

2434ab3

shino reviewed Apr 9, 2015
View reviewed changes

kuenishi force-pushed the feature/quota-hook branch from e7f0b6a to f14571c Compare April 10, 2015 07:28

shino added 3 commits April 10, 2015 16:57

Add detailed mode for storage stats calculation

95a774f

Support for detailed storage calc mode of inspector

3d0b34b

Split unit tests to test module

19834b4

shino added 3 commits April 13, 2015 16:38

Fix riak_test case for changes of JSON keys

dbde9da

Use single new/old division timestamp in whole storage calc M/Rs

f07a6d3

Document uuidgen dependency for ceph's s3-tests

f9d981c

shino reviewed Apr 13, 2015
View reviewed changes

Refactoring and cosmetics on comments

715ee28

shino reviewed Apr 13, 2015
View reviewed changes

Merge pull request #1120 from basho/feature/detailed-storage-stats

d6a1dd2

Detailed storage stats Reviewed-by: kuenishi

shino reviewed Apr 14, 2015
View reviewed changes

Merge latest 2.0 branch into 'feature/quota-hook'

bd6f4a2

Conflicts: src/riak_cs_config.erl src/riak_cs_storage_d.erl

borshop added a commit that referenced this pull request Apr 14, 2015

Merge pull request #1118 from basho/feature/quota-hook

bf67549

Rough sketch of user quota and bandwidth limiting Reviewed-by: shino

borshop merged commit bd6f4a2 into 2.0 Apr 14, 2015

kuenishi deleted the feature/quota-hook branch April 14, 2015 07:04

kuenishi restored the feature/quota-hook branch April 28, 2015 01:23

kuenishi deleted the feature/quota-hook branch April 28, 2015 01:24

kuenishi added a commit that referenced this pull request Apr 28, 2015

Revert #1118, #1120 and #1123 from release notes

27b74a1

kuenishi modified the milestones: 2.1.0, 2.0.1 Apr 28, 2015

borshop added a commit that referenced this pull request Apr 28, 2015

Merge pull request #1131 from basho/revert/features

70f8cab

Revert #1118, #1120 and #1123 from release notes Reviewed-by: shino

kuenishi modified the milestones: 2.2.0, 2.1.0 Apr 28, 2015

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Rough sketch of user quota and bandwidth limiting #1118

Rough sketch of user quota and bandwidth limiting #1118

kuenishi commented Apr 6, 2015

kuenishi commented Apr 6, 2015

kuenishi commented Apr 6, 2015

kuenishi commented Apr 9, 2015

kuenishi commented Apr 9, 2015

shino Apr 9, 2015

kuenishi Apr 9, 2015

kuenishi commented Apr 10, 2015

kuenishi commented Apr 10, 2015

shino Apr 13, 2015

kuenishi Apr 13, 2015

shino Apr 13, 2015

shino commented Apr 13, 2015

kuenishi commented Apr 13, 2015

shino Apr 14, 2015

shino commented Apr 14, 2015

kuenishi commented Apr 14, 2015

kuenishi commented Apr 14, 2015

kuenishi commented Apr 14, 2015

Basho-JIRA commented Aug 14, 2015

Rough sketch of user quota and bandwidth limiting #1118

Rough sketch of user quota and bandwidth limiting #1118

Conversation

kuenishi commented Apr 6, 2015

kuenishi commented Apr 6, 2015

Quota

kuenishi commented Apr 6, 2015

kuenishi commented Apr 9, 2015

kuenishi commented Apr 9, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kuenishi commented Apr 10, 2015

kuenishi commented Apr 10, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shino commented Apr 13, 2015

kuenishi commented Apr 13, 2015

Choose a reason for hiding this comment

shino commented Apr 14, 2015

kuenishi commented Apr 14, 2015

kuenishi commented Apr 14, 2015

kuenishi commented Apr 14, 2015

Basho-JIRA commented Aug 14, 2015