Feature request: Default rate limit for projects

[marbon87]

Summary

Add a possibility to globally define a default rate limit for projects

Motivation

In the current version of sentry, we can configure a rate limit for the complete instance or manually for each project.
It would be helpful if we could set a rate limit globally that is relevant for each project.

┆Issue is synchronized with this Jira Improvement by Unito

[github-actions]

This issue has gone three weeks without activity. In another week, I will close it.

But! If you comment or otherwise update it, I will reset the clock, and if you label it Status: Accepted, I will leave it alone ... forever!

---

"A weed is but an unloved flower." ― Ella Wheeler Wilcox 🥀

[marbon87]

@BYK do you think this is worth to be implemented in the future?

[getsentry-release]

Routing to @getsentry/owners-ingest for triage. ⏲️

[untitaker]

makes sense, fyi we added it to our backlog but can't share any updates so far. thanks for raising!

[jairbubbles]

I would be very much interested by this feature too!

[nwgeek]

This feature is crucial to us. We had this ability in OnPrem in the past.

[bqbn]

We would love to have this feature too. We use the SaaS sentry, and our organization has almost 100 projects and we're looking to migrate another 100 or so from our on-premise instance to it. Having a default rate limit or org-wide rate limit will greatly help us prevent run-away projects (that send too many events by accident) from eating up all the quotas.

Also note that because we have many projects and the average org-wide event rate is high, the spike protection doesn't really help us in the case where a newly created project sends too many events accidentally.

[PickledChris]

This would be useful to us, but for the benefit of anyone who finds this post we have a reasonable workaround.
I can't post the code, but we have a simple Python script which:

1. Gets all the projects with organizations/{org}/projects/ (don't forget to paginate)
2. Gets each project key using projects/{org}/{slug}/keys/
3. Updates each key using PUT projects/{org}/{project_slug}/keys/{key}/

The body for 3 is taken from the response in 2, it's:

{
      "rateLimit": {
          "window": {int, number of seconds},
          "count": {number of events within window seconds},
      }
  }

If this ever breaks, it's easy to reverse engineer by editing the rate limit on the key page and watching the request in dev tools.

Obviously this is safer on a self-hosted sentry where you can control the upgrades, rather than cloud where it could break at any time. But it's not the end of the world if it does, and presumably you have good alerting using Sentry!

[ale-cota]

@Dhrumil-Sentry, I think there is synergy between this feature request and the Spike Protection initiative or something additional to consider. Thank you!

[getsantry]

Routing to @getsentry/enterprise for triage, due by Wednesday, April 5th at 5:00 pm (sfo). ⏲️

[Dhrumil-Sentry]

Hi all ! thanks a lot for raising this issue , I had a few questions and suggestions

@bqbn Have you looked at Sentry's improved spike protection #44155 ? We've made improvements to ensure new projects don't accidentally consume too many events

Sentry allows you to configure rate limits at a per-project level. @PickledChris 's APIs seem seem to be leveraging the same feature

How do you folks provision new projects, is there a reason you cannot set up DSN level rate limits as a part of project setup?

[PickledChris]

We let teams manage and provision their projects, we strive to make as much as we can to be self-service rather than gatekeep.
But simultaneously we can't expect everyone to know about this, or remember each time they create a key, so we would like the ability to set sensible defaults.

[ghost]

I agree with @PickledChris, sensible defaults should be in place. If you have a project that is generating lots of event then you should probably need to be explicit about that.

[gosiaASI]

+1 to this request. You are unable to control the spending without a default rate limit.

[fsmaia]

+1 to this request.

[nwgeek]

We still want this as our account has over 200+ projects.

[getsantry]

Routing to @getsentry/product-owners-settings-projects for triage ⏲️

[abotkin-cpi]

This would be a great feature and may help improve uptake of Sentry in larger organizations. The inability to set default rate limits leads to larger orgs having to create processes to limit Sentry project creation to avoid rogue developer's project that consumers too much too quickly. Without the safe defaults, management is left having to put process in that slows down or discourages developers from adopting Sentry to "not rock the boat".

[Dhrumil-Sentry]

@abotkin-cpi Thanks a lot for this feedback. We don't have any plans to do this in the next quarter at least but we will keep this issue updated if we work on this

cc @gauthamcs

[m0nikasingh]

There is an alternate way to do this.

sentry explicitly sets NULL for rate_limit_count and rate_limit_window.

A postgres before-insert trigger and trigger function can be added which would check for NULLs in these columns and change it to default.

Something like:

CREATE FUNCTION trf_set_default_rate_limit()
RETURNS trigger AS '
BEGIN
    IF NEW.rate_limit_count IS NULL THEN
        NEW.rate_limit_count=500;
    END IF;
    IF NEW.rate_limit_window IS NULL THEN
        NEW.rate_limit_window=60;
    END IF;
RETURN NEW;
END' LANGUAGE 'plpgsql';

CREATE TRIGGER tr_set_default_rate_limit
    BEFORE INSERT ON sentry_projectkey
    FOR EACH ROW
    EXECUTE PROCEDURE trf_set_default_rate_limit();

Verify that the trigger is created:

SELECT  event_object_table AS table_name ,trigger_name FROM information_schema.triggers GROUP BY table_name , trigger_name ORDER BY table_name ,trigger_name;

Create a new project and check the ratelimit for the project.