fix: add caching to koanf config to fix performance regression

[David-Wobrock]

Fix #1033 of a performance regression in v0.40.0.
It seems linked to a perpetual config reloading, leading to a lot of JSON marshalling being CPU intensive.
Adding back in-memory caching, as we used to have for Viper configuration.

Related issue(s)

Fixes #1033

Checklist

• I have read the contributing guidelines.
• I have referenced an issue containing the design document if my change
  introduces a new feature.
• I am following the
  contributing code guidelines.
• I have read the security policy.
• I confirm that this pull request does not address a security
  vulnerability. If this pull request addresses a security vulnerability, I
  confirm that I got the approval (please contact
  security@ory.sh) from the maintainers to push
  the changes.
• I have added tests that prove my fix is effective or that my feature
  works.
• I have added or changed the documentation.

Further Comments

[codecov]

Codecov Report

Merging #1042 (77d5b4e) into master (878089d) will increase coverage by 0.46%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master    #1042      +/-   ##
==========================================
+ Coverage   77.74%   78.20%   +0.46%     
==========================================
  Files          83       83              
  Lines        3841     4006     +165     
==========================================
+ Hits         2986     3133     +147     
- Misses        578      594      +16     
- Partials      277      279       +2     

Impacted Files	Coverage Δ
driver/configuration/provider_koanf.go	87.34% <100.00%> (+1.29%)	⬆️
driver/registry_memory.go	89.82% <0.00%> (-3.93%)	⬇️
internal/cloudstorage/useragent.go	100.00% <0.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

[aeneasr]

Thank you very much! I think there's an improvement to this that addresses the problem in a more targeted way and without sideffects :)

[aeneasr]

The culprit of the high CPU usage is in this method. My recommendation would be to cache the result of the validation (only when successful) and the cache key sha256(prefix + "|" + id + "|" + marshalled). That way we will respect hot reloading (because we use the config values) and still save a lot of CPU cycles!

[David-Wobrock]

Hey @aeneasr
Thanks for the review! Good catch, we can reduce the caching range, and avoid side-effects 👍 I wasn't aware of the hot-reloading part.

I pushed edits to the PR.

[aeneasr]

Awesome, looking much better! just a commend on how to improve the ristretto config

[David-Wobrock]

Thanks for the configuration @aeneasr 👍
I updated the commit with the suggestion :)