Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cache service usage endpoint #4666

Merged
merged 21 commits into from Oct 2, 2023
Merged

Conversation

LMNTL
Copy link
Contributor

@LMNTL LMNTL commented Sep 26, 2023

Checklist

  1. If you've added code that should be tested, add tests
  2. If you've changed APIs, update (or create!) the documentation
  3. Ensure the tests pass
  4. Make sure that your code lints and that you've followed our coding style
  5. Write a title and, if necessary, a description of your work suitable for publishing in our release notes
  6. Mention any related issues in this repository (as #ISSUE) and in other repositories (as kobotoolbox/other#ISSUE)
  7. Open an issue in the docs if there are UI/UX changes

Description

Caches the api/v2/service_usage endpoint to prevent slowdown on sites with a large number of users/projects. Adds a 'last updated' widget to the Usage page. Allows promotion codes to be used when checking out for a Plan.

Notes

  • Added a new setting/env var, ENDPOINT_CACHE_DURATION.
  • Refactored service_usage endpoint to use a detail view (/api/v2/organizations/{ORGANIZATION_ID}/service_usage/) instead of POSTing the organization ID.
  • The badge on the usage page is based on the cached age of the response from the service usage endpoint (user or organization). To reset it, log out and back in.

@LMNTL LMNTL marked this pull request as ready for review September 28, 2023 17:36
@noliveleger noliveleger added Back end Front end API Changes related to API endpoints labels Sep 28, 2023
@noliveleger noliveleger self-assigned this Sep 28, 2023
Comment on lines 16 to 17
@method_decorator(cache_page(60 * 30), name='retrieve')
@method_decorator(cache_page(60 * 30), name='list')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's use a setting for this (which could even be configurable with env variable, then fallback on your default). We could have a shorter timeout in dev settings than prod.

Maybe 30 minutes is a little bit too long, what about reverting it to 15?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've extracted this out to ENDPOINT_CACHE_DURATION. This setting is used by /service_usage/ and /stripe/products/, but that can be refactored if we later need to set those cache settings independently.

The only thing I'm not sure about is using a shorter timeout in dev settings; I'm worried that could make it easier to miss unwanted results of caching while debugging.

@action(methods=['post'], detail=False)
def post(self, request, *args, **kwargs):
return self.list(request, *args, **kwargs)
def retrieve(self, request, pk=None):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'd rather keep DRF signature, i.e. def retrieve(self, request, pk=None, *args, **kwargs).
(for example, format does not work without those parameters)

🤔 Anyway, this endpoint does not seem RESTFul compliant to me. pk is not a unique identifier of an service_usage model.

I'd rather see something like:

  • /api/v2/organizations/<pk>/service_usage/ for organizations
  • /api/v2/service_usage/ for users

It may involve more front-end (and back-end) changes but I think it's worth it ;-)
What do you think?

getUsageForOrganization().then((data) => {
if (!data) {
return;
}
let lastUpdated: UsageState['lastUpdated'] = null;
if ('headers' in data && data.headers instanceof Headers) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could be it simplified with if (data?.headers instanceof Headers)?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately, TS complains here without in acting as a type guard:

image

image

@@ -1363,6 +1363,9 @@ def dj_stripe_request_callback_method():
'default': env.cache(default='redis://redis_cache:6380/3'),
}

# How long to retain cached responses for kpi endpoints
ENDPOINT_CACHE_DURATION = env.str('ENDPOINT_CACHE_DURATION', 60 * 15) # 15 minutes
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should better use env.int() instead of env.str() here? Even the default value is an integer?

@LMNTL LMNTL requested a review from noliveleger October 2, 2023 18:44
@noliveleger noliveleger merged commit c639e7a into release/2.023.37 Oct 2, 2023
4 checks passed
@noliveleger noliveleger deleted the cache-service-usage branch October 2, 2023 20:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
API Changes related to API endpoints Back end Front end
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants