Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cache configurability - expiry time, read/write controls #13

Merged
merged 7 commits into from
Sep 4, 2022

Conversation

briantist
Copy link
Owner

Closes #4

  • Allows for configuring that cache expiry in minutes (previously was hardcoded at 1 hour. now defaults to 60 minutes)
  • Internal change to cache expiration calculation:
    • when cache is written, the expiration is also written, using the expiry time of the writer
    • reading cache had the ability to calculate whether cache was expired based on local expiry delta + cache creation time, but defaulted to just using the written time
    • this didn't matter when the time was hardcoded: all clients used the same
    • this behavior has been switched to always calculate (for now, it could be configurable later)
    • this means that now, expiry times are still written like before (based on the writer's configuration), but reads always calculate based on the reader's configured expiry time
    • this allows for a reader to decide to trust cached values for a different amount of time than the instance that wrote it
    • a long-running central instance might be set to use a 60 minute expiry time, while a local instance you run on your workstation with a slow connection, where freshness is not that important, could be set to use a 1440 minute expiration; with other writers updating the caches more frequently, it could continually result in not needing to contact the upstream
  • Whether to (attempt to) read and write to the cache is now independently configurable:
    • Setting --cache-read to true (the default) means that caches will be read and the values used, while setting it to false will always proxy to an upstream without regard for cached values
    • Setting --cache-write to true (the default) means that when a request is proxied upstream, the results will be cached (written to Artifactory) so that this or other instances configured to read from cache can avoid contacting the upstream for that request. Setting it to false will not write upstream responses to the cache.
    • Setting --cache-write to false can be especially useful if proxying to an upstream is desired without setting an Artifactory API key (when reads to Artifactory don't require authentication), as this allows for full upstream support without supplying any authentication/secrets.
    • Notes about "cache" nomenclature and behavior with downloads:
      • previously, all downloads from an upstream (the collection doesn't exist in Artifactory yet) resulted in galactory downloading the collection, uploading it to Artifactory, and then streaming the result from a new download from Artifactory
      • Future requests would always serve that file from Artifactory (since it now exists as a "local" collection basically)
      • Those local collections that came from an upstream are not distinguishable from ones intentionally directly uploaded, and do not count as "cached"; as a result --cache-read=false does not affect that behavior (a download of a collection will be served from Artifactory if that's where it exists)
      • While those written-from-upstream collections do not count as "cached", --cache-write=false will necessarily result in an upstream collection that doesn't exist in Artifactory not being uploaded to Artifactory, but streamed directly from the upstream

@briantist briantist added the enhancement New feature or request label Sep 4, 2022
@briantist briantist self-assigned this Sep 4, 2022
@briantist briantist merged commit f3594cf into main Sep 4, 2022
@briantist briantist deleted the cache/configurability branch September 4, 2022 19:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Allow for disabling the upstream proxy cache, and/or caching locally
1 participant