Cache patron's loans to reduce IA requests

[JaydenTeoh]

Closes #7591

Implement cache logic that caches patron's loan with 5 minutes time-to-live. If this 5 minute window has passed or if patron has borrowed another book (via /borrow/ia/$(ocaid)), then the cache will be invalidated => loans will be fetched again via api.

Stakeholders

@mekarpeles

[mekarpeles]

I think this is close, but I think we want both the ability to fetch loans and also to fetch cached loans independently. The borrow flow needs to have an authoritative way to fetch loans with no chance of them being cached. Other pages (like where LoanStatus macro fetches loans) should use the cache.

[JaydenTeoh]

I think this is close, but I think we want both the ability to fetch loans and also to fetch cached loans independently. The borrow flow needs to have an authoritative way to fetch loans with no chance of them being cached. Other pages (like where LoanStatus macro fetches loans) should use the cache.

@mekarpeles I have implement a use_cache=True argument within get_loans_of_user. Within the borrow.py flow, I have added two definite checks to ensure that the loans data will be fetched from API rather than cached results.
First, if a user is present,
lending.get_loans_of_user.cache_reset(user.key) is called to wipe all the cache for that user.key

Also, for all the parent functions of get_loans_of_user that exist within borrow.py, I have added the use_cache=False argument to ensure that only API results are returned.

Although I am unable to test loans on local, I have tested the flow of this cache implementation with a basic python script and it is behaving as expected. See picture below:

[mekarpeles]

Currently, the places where we're making expensive calls for a patron's loans and waitlists are:

1. In macros.LoanStatus
   which is used for Books Page, Search Results, My Books

• https://github.com/internetarchive/openlibrary/blob/master/openlibrary/macros/LoanStatus.html#L20
• https://github.com/internetarchive/openlibrary/blob/master/openlibrary/macros/LoanStatus.html#L37

2. My Loans Page
3. When a borrow occurs in core/lending.py or plugins/upstream/borrow.py

I believe the only time we want to use cache (for now) is within [1].

I would be nice if we had a new @public function called get_patron_loans, which calls our existing slow functions which returns a dictionary that looks like...

{
 "loans": [...],
 "holds": [...]
}

This would get cached for 5 minutes and could be called by any non-critical path, such as [1] LoanStatus. Within the critical paths (borrow endpoints) we should use the expensive, non-cached endpoints).

We may be able to leverage the use_cache=True logic you added for this.

[JaydenTeoh]

@jimchamp
Latest changes:

1. Refactor caching mechanism to use memcache instead of my own cache decorator because there are vulnerabilities in my original cache implementation such as memory leak and memory overload
2. Implement a 'opt-in' instead of 'opt-out' system for using cache
3. Use a constant variable ttl_for_cached_loans = 5 * cache.MINUTE_SECS rather than a magic number 300
4. Use delegate.fakeload() such that the cache function get_cached_user_loans can interact with the API if this is first time calling the function or if the previous cached result has expired. I am not sure whether this would work so I'll need help with staging and testing this.

One issue with my current implementation is that it only caches the user loans, but there is no caching implemented for user's waiting loans yet. Should I do so for waiting loans as well? Is it an expensive call?

[jimchamp]

There are quite a few changes to the critical path that I think should be reverted. Holds (AKA waitlist items) are also expensive to fetch and should also be cached.

Now that you're using Open Library's cache, you'll have to invalidate the cache using memcache_delete. The exact invocation that I suggested may not work, so you may need to experiment a bit.

Following Mek's suggestion to create a public get_patron_loans function that returns both loans and holds would be helpful when we expand this functionality to the My Book page.

[jimchamp]

Since we're only focusing on the LoanStatus buttons for this PR, let's punt on the new get_patron_lists function.

[mekarpeles]

lgtm, tested on testing, to be patch-deployed on prod (on Monday)

[mekarpeles]

Q: why are we calling both IA loans (which presumably uses s3) and OL web.ctx loans store?

[mekarpeles]

finally lgtm!!

[mekarpeles]

lgtm