Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Possible difference in behaviour between caffeine and guava for LoadingCache.refresh calls with non-present keys #236

Closed
slovdahl opened this issue Apr 27, 2018 · 5 comments

Comments

@slovdahl
Copy link

@slovdahl slovdahl commented Apr 27, 2018

I think that caffeine and guava differs in behaviour when LoadingCache.refresh is invoked for a key that doesn't yet have any cached entry.

Consider the following scenario: an initial invocation of refresh triggers the CacheLoader to start computing the value. While the value is being computed, multiple refreshs are invoked for the same key. Guava notices that there's already a refresh in progress, while caffeine triggers new loads for every refresh invocation. Feel free to correct me if I misunderstood the code. 🙂

We hit this issue in production after migrating from guava. At this point it's a bit unclear to me why we use refresh here instead of get. Might be a misuse of refresh. Mitigating the issue would of course be easy by limiting the executor to one thread as well.

HeavyObject obj = loadingCache.getIfPresent(key);
if (obj != null) {
  return obj;
}
else {
  executor.submit(() -> loadingCache.refresh(key));
  // return object representing computation in progress
}
@ben-manes
Copy link
Owner

@ben-manes ben-manes commented Apr 27, 2018

Yes, it's a flaw that I've meant to fix.

Guava has an advantage by forking the hash table and swaps out the underlying entry during the computation. This allows it to serve the current value during the refresh, and the callback restores it to a lighter-weight key/value entry. It is a cute trick that we cannot emulate easily.

The proper solution may be to have a lazily initialized Map<K, CompletableFuture<V>> in the LoadingCache of the in-flight refreshes. This way if one is on-going, it can no-op. However, it is not clear what linearizability expectations users might have with refresh, since a put will stampede it. That could mean that refresh => put => refresh should cancel the first one, update, and finally contain the last refresh's result. We'd need to emulate that in our map by capturing the current value to decide if a new async operation should be triggered or not.

Which all means it got complicated enough that I didn't get to it. Since it is a rarely used feature (vs. refreshAfterWrite) no one seemed to notice until now. 😀

@slovdahl
Copy link
Author

@slovdahl slovdahl commented Apr 27, 2018

Thanks for the informative answer!

Thinking of it further, it doesn't matter if there is a value or not, does it? Doing multiple refresh invocations while a value is being computed triggers equally many loads nevertheless. Wouldn't of course happen in our case, because we don't trigger any refresh if a (stale) value exists.

@ben-manes
Copy link
Owner

@ben-manes ben-manes commented Apr 27, 2018

yep, absence and present values are treated similarly by their respective implementations.

@ben-manes
Copy link
Owner

@ben-manes ben-manes commented Apr 27, 2018

For now please use get in your executor. If you are returning futures anyway, you might want to switch to AsyncLoadingCache and simplify the logic to its get(key) call. If I'm lucky then I may get to this over the weekend, but may not.

ben-manes added a commit that referenced this issue Jan 2, 2021
A mapping of in-flight refreshes is now maintained and lazily
initialized if not used. This allows the cache to ignore redundant
requests for reloads, like Guava does. It also removes disablement
of expiration during refresh and resolves an ABA problem if the
entry is modified in an undectectable way. The refresh future can
now be obtained from LoadingCache to chain operations against.

TODO: unit tests for these changes

fixes #143
fixes #193
fixes #236
fixes #282
fixes #322
fixed #373
fixes #467
ben-manes added a commit that referenced this issue Jan 2, 2021
A mapping of in-flight refreshes is now maintained and lazily
initialized if not used. This allows the cache to ignore redundant
requests for reloads, like Guava does. It also removes disablement
of expiration during refresh and resolves an ABA problem if the
entry is modified in an undectectable way. The refresh future can
now be obtained from LoadingCache to chain operations against.

TODO: unit tests for these changes

fixes #143
fixes #193
fixes #236
fixes #282
fixes #322
fixed #373
fixes #467
ben-manes added a commit that referenced this issue Jan 2, 2021
A mapping of in-flight refreshes is now maintained and lazily
initialized if not used. This allows the cache to ignore redundant
requests for reloads, like Guava does. It also removes disablement
of expiration during refresh and resolves an ABA problem if the
entry is modified in an undectectable way. The refresh future can
now be obtained from LoadingCache to chain operations against.

TODO: unit tests for these changes

fixes #143
fixes #193
fixes #236
fixes #282
fixes #322
fixed #373
fixes #467
ben-manes added a commit that referenced this issue Jan 2, 2021
A mapping of in-flight refreshes is now maintained and lazily
initialized if not used. This allows the cache to ignore redundant
requests for reloads, like Guava does. It also removes disablement
of expiration during refresh and resolves an ABA problem if the
entry is modified in an undectectable way. The refresh future can
now be obtained from LoadingCache to chain operations against.

TODO: unit tests for these changes

fixes #143
fixes #193
fixes #236
fixes #282
fixes #322
fixed #373
fixes #467
ben-manes added a commit that referenced this issue Jan 3, 2021
A mapping of in-flight refreshes is now maintained and lazily
initialized if not used. This allows the cache to ignore redundant
requests for reloads, like Guava does. It also removes disablement
of expiration during refresh and resolves an ABA problem if the
entry is modified in a previously undectectable way. The refresh
future can now be obtained from LoadingCache to chain operations
against.

fixes #143
fixes #193
fixes #236
fixes #282
fixes #322
fixed #373
fixes #467
ben-manes added a commit that referenced this issue Jan 3, 2021
A mapping of in-flight refreshes is now maintained and lazily
initialized if not used. This allows the cache to ignore redundant
requests for reloads, like Guava does. It also removes disablement
of expiration during refresh and resolves an ABA problem if the
entry is modified in a previously undectectable way. The refresh
future can now be obtained from LoadingCache to chain operations
against.

fixes #143
fixes #193
fixes #236
fixes #282
fixes #322
fixed #373
fixes #467
ben-manes added a commit that referenced this issue Jan 3, 2021
A mapping of in-flight refreshes is now maintained and lazily
initialized if not used. This allows the cache to ignore redundant
requests for reloads, like Guava does. It also removes disablement
of expiration during refresh and resolves an ABA problem if the
entry is modified in a previously undectectable way. The refresh
future can now be obtained from LoadingCache to chain operations
against.

fixes #143
fixes #193
fixes #236
fixes #282
fixes #322
fixed #373
fixes #467
ben-manes added a commit that referenced this issue Jan 4, 2021
A mapping of in-flight refreshes is now maintained and lazily
initialized if not used. This allows the cache to ignore redundant
requests for reloads, like Guava does. It also removes disablement
of expiration during refresh and resolves an ABA problem if the
entry is modified in a previously undectectable way. The refresh
future can now be obtained from LoadingCache to chain operations
against.

fixes #143
fixes #193
fixes #236
fixes #282
fixes #322
fixed #373
fixes #467
ben-manes added a commit that referenced this issue Jan 4, 2021
A mapping of in-flight refreshes is now maintained and lazily
initialized if not used. This allows the cache to ignore redundant
requests for reloads, like Guava does. It also removes disablement
of expiration during refresh and resolves an ABA problem if the
entry is modified in a previously undectectable way. The refresh
future can now be obtained from LoadingCache to chain operations
against.

fixes #143
fixes #193
fixes #236
fixes #282
fixes #322
fixed #373
fixes #467
ben-manes added a commit that referenced this issue Jan 4, 2021
A mapping of in-flight refreshes is now maintained and lazily
initialized if not used. This allows the cache to ignore redundant
requests for reloads, like Guava does. It also removes disablement
of expiration during refresh and resolves an ABA problem if the
entry is modified in a previously undectectable way. The refresh
future can now be obtained from LoadingCache to chain operations
against.

fixes #143
fixes #193
fixes #236
fixes #282
fixes #322
fixed #373
fixes #467
ben-manes added a commit that referenced this issue Jan 17, 2021
A mapping of in-flight refreshes is now maintained and lazily
initialized if not used. This allows the cache to ignore redundant
requests for reloads, like Guava does. It also removes disablement
of expiration during refresh and resolves an ABA problem if the
entry is modified in a previously undectectable way. The refresh
future can now be obtained from LoadingCache to chain operations
against.

fixes #143
fixes #193
fixes #236
fixes #282
fixes #322
fixed #373
fixes #467
ben-manes added a commit that referenced this issue Feb 8, 2021
A mapping of in-flight refreshes is now maintained and lazily
initialized if not used. This allows the cache to ignore redundant
requests for reloads, like Guava does. It also removes disablement
of expiration during refresh and resolves an ABA problem if the
entry is modified in a previously undectectable way. The refresh
future can now be obtained from LoadingCache to chain operations
against.

fixes #143
fixes #193
fixes #236
fixes #282
fixes #322
fixed #373
fixes #467
ben-manes added a commit that referenced this issue Feb 8, 2021
A mapping of in-flight refreshes is now maintained and lazily
initialized if not used. This allows the cache to ignore redundant
requests for reloads, like Guava does. It also removes disablement
of expiration during refresh and resolves an ABA problem if the
entry is modified in a previously undectectable way. The refresh
future can now be obtained from LoadingCache to chain operations
against.

fixes #143
fixes #193
fixes #236
fixes #282
fixes #322
fixed #373
fixes #467
ben-manes added a commit that referenced this issue Feb 8, 2021
A mapping of in-flight refreshes is now maintained and lazily
initialized if not used. This allows the cache to ignore redundant
requests for reloads, like Guava does. It also removes disablement
of expiration during refresh and resolves an ABA problem if the
entry is modified in a previously undectectable way. The refresh
future can now be obtained from LoadingCache to chain operations
against.

fixes #143
fixes #193
fixes #236
fixes #282
fixes #322
fixed #373
fixes #467
ben-manes added a commit that referenced this issue Feb 8, 2021
A mapping of in-flight refreshes is now maintained and lazily
initialized if not used. This allows the cache to ignore redundant
requests for reloads, like Guava does. It also removes disablement
of expiration during refresh and resolves an ABA problem if the
entry is modified in a previously undectectable way. The refresh
future can now be obtained from LoadingCache to chain operations
against.

fixes #143
fixes #193
fixes #236
fixes #282
fixes #322
fixed #373
fixes #467
ben-manes added a commit that referenced this issue Feb 14, 2021
A mapping of in-flight refreshes is now maintained and lazily
initialized if not used. This allows the cache to ignore redundant
requests for reloads, like Guava does. It also removes disablement
of expiration during refresh and resolves an ABA problem if the
entry is modified in a previously undectectable way. The refresh
future can now be obtained from LoadingCache to chain operations
against.

fixes #143
fixes #193
fixes #236
fixes #282
fixes #322
fixed #373
fixes #467
ben-manes added a commit that referenced this issue Feb 14, 2021
A mapping of in-flight refreshes is now maintained and lazily
initialized if not used. This allows the cache to ignore redundant
requests for reloads, like Guava does. It also removes disablement
of expiration during refresh and resolves an ABA problem if the
entry is modified in a previously undectectable way. The refresh
future can now be obtained from LoadingCache to chain operations
against.

fixes #143
fixes #193
fixes #236
fixes #282
fixes #322
fixed #373
fixes #467
ben-manes added a commit that referenced this issue Feb 14, 2021
A mapping of in-flight refreshes is now maintained and lazily
initialized if not used. This allows the cache to ignore redundant
requests for reloads, like Guava does. It also removes disablement
of expiration during refresh and resolves an ABA problem if the
entry is modified in a previously undectectable way. The refresh
future can now be obtained from LoadingCache to chain operations
against.

fixes #143
fixes #193
fixes #236
fixes #282
fixes #322
fixed #373
fixes #467
ben-manes added a commit that referenced this issue Feb 15, 2021
A mapping of in-flight refreshes is now maintained and lazily
initialized if not used. This allows the cache to ignore redundant
requests for reloads, like Guava does. It also removes disablement
of expiration during refresh and resolves an ABA problem if the
entry is modified in a previously undectectable way. The refresh
future can now be obtained from LoadingCache to chain operations
against.

fixes #143
fixes #193
fixes #236
fixes #282
fixes #322
fixed #373
fixes #467
ben-manes added a commit that referenced this issue Feb 15, 2021
A mapping of in-flight refreshes is now maintained and lazily
initialized if not used. This allows the cache to ignore redundant
requests for reloads, like Guava does. It also removes disablement
of expiration during refresh and resolves an ABA problem if the
entry is modified in a previously undectectable way. The refresh
future can now be obtained from LoadingCache to chain operations
against.

fixes #143
fixes #193
fixes #236
fixes #282
fixes #322
fixed #373
fixes #467
ben-manes added a commit that referenced this issue Feb 15, 2021
A mapping of in-flight refreshes is now maintained and lazily
initialized if not used. This allows the cache to ignore redundant
requests for reloads, like Guava does. It also removes disablement
of expiration during refresh and resolves an ABA problem if the
entry is modified in a previously undectectable way. The refresh
future can now be obtained from LoadingCache to chain operations
against.

fixes #143
fixes #193
fixes #236
fixes #282
fixes #322
fixed #373
fixes #467
ben-manes added a commit that referenced this issue Feb 15, 2021
A mapping of in-flight refreshes is now maintained and lazily
initialized if not used. This allows the cache to ignore redundant
requests for reloads, like Guava does. It also removes disablement
of expiration during refresh and resolves an ABA problem if the
entry is modified in a previously undectectable way. The refresh
future can now be obtained from LoadingCache to chain operations
against.

fixes #143
fixes #193
fixes #236
fixes #282
fixes #322
fixed #373
fixes #467
ben-manes added a commit that referenced this issue Feb 15, 2021
A mapping of in-flight refreshes is now maintained and lazily
initialized if not used. This allows the cache to ignore redundant
requests for reloads, like Guava does. It also removes disablement
of expiration during refresh and resolves an ABA problem if the
entry is modified in a previously undectectable way. The refresh
future can now be obtained from LoadingCache to chain operations
against.

fixes #143
fixes #193
fixes #236
fixes #282
fixes #322
fixed #373
fixes #467
ben-manes added a commit that referenced this issue Feb 16, 2021
A mapping of in-flight refreshes is now maintained and lazily
initialized if not used. This allows the cache to ignore redundant
requests for reloads, like Guava does. It also removes disablement
of expiration during refresh and resolves an ABA problem if the
entry is modified in a previously undectectable way. The refresh
future can now be obtained from LoadingCache to chain operations
against.

fixes #143
fixes #193
fixes #236
fixes #282
fixes #322
fixed #373
fixes #467
@ben-manes ben-manes closed this in 551ade7 Feb 21, 2021
@ben-manes
Copy link
Owner

@ben-manes ben-manes commented Feb 21, 2021

Released in 3.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants