Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FeatureRequest: enable a Future<Void> for refresh() #143

Closed
boschb opened this issue Feb 12, 2017 · 23 comments
Closed

FeatureRequest: enable a Future<Void> for refresh() #143

boschb opened this issue Feb 12, 2017 · 23 comments

Comments

@boschb
Copy link

@boschb boschb commented Feb 12, 2017

Hi Ben,

Currently there is no way to know when a value is refreshed after calling refresh() on a LoadingCache. Some synchronization techniques could be utilized with a CacheLoader, but I still don't think that guarantees exact entry depending on when context switching occurs.

Thanks!

@ben-manes
Copy link
Owner

@ben-manes ben-manes commented Feb 12, 2017

Can you provide a little more context on how you would use this?

Unfortunately due to semver, this API change would require a major release. I plan on 3.0 being these types of tweaks and targeting jdk9 (no unsafe) whenever it ships.

@boschb
Copy link
Author

@boschb boschb commented Feb 12, 2017

Sure thing.

My scenario is I have a Multimap<Key,Value> that I use as truth. I want to provide an immutable 'snapshot' of the map using a Cache<Key,Set<Value>> with a CacheLoader that simply synchronizes on the Multimap and builds an ImmutableSet from the current values (for a given key). This might seem like overkill, but the idea is that I will always have a 'snapshot' set that I can iterate over (at high frequency) without blocking the underlying Cache or Multimap. Adds and removes are less frequent and not under the same blocking requirements. Also my service allows for non-parity between the cache and map (for short a short period of time) as best effort, mostly accurate, snapshots provide all we need.

By utilizing the features of refresh() where a new value is loaded in the background while the old value continues to serve, this gives me exactly the behavior I need.

The only catch that I have is that I need to be able to gate (especially during tests) the point at which a new entry added to the Multimap actually is present in the underlying Cache.

@boschb
Copy link
Author

@boschb boschb commented Feb 12, 2017

Regarding versioning, wouldn't a @IgnoreReturnValue and ListenableFuture be safe? method is already void. I would even be ok with a completely new method. It's also entirely possible I don't know what i'm talking about though in regards to semver as I'm not that familiar.

Also FWIW, if this has to be a AsyncCache thing that's fine too (The line is a little grey regarding refresh())

@ben-manes
Copy link
Owner

@ben-manes ben-manes commented Feb 12, 2017

To clarify, is this needed for testing or the implementation? It sounds like testing only.

The API change would mean any other implanting class would not compile. I'd also like to ensure changes carry their weight, as an API is long lived.

@boschb
Copy link
Author

@boschb boschb commented Feb 12, 2017

Technically both. Though the implementation in practice I think avoids this race probably every time because of network latency. The timing hole still exists though (and tests prove it) so it does make me feel more correct to use it if I can.

I was hoping that we could just bubble up the future from the CacheLoader easy peasy:
https://github.com/ben-manes/caffeine/blob/master/caffeine/src/main/java/com/github/benmanes/caffeine/cache/LocalLoadingCache.java#L158

I see your point about breaking other implementors though. Api wise though I think it makes sense to bubble up this future from the loader, or maybe even move the insert logic to the loader itself by passing the Cache to the reload function... It's a tough call. They are pretty interlinked, but this def is a Loading/Loader specific feature... Your thoughts?

@ben-manes
Copy link
Owner

@ben-manes ben-manes commented Feb 13, 2017

I think you could use Awaitility so that your tests tolerate this race.

From your description, it also sounds like alternatives to refreshing may work. If you use ConcurrentMap<K, ImmutableSet<V>> instead, then the cache might be unnecessary. Or perhaps some of the asMap view write methods useful if you insert into the cache after updating the Multimap.

Right now I'm neutral on this change, but it would have to wait until 3.0 so hopefully we can find a workaround for now.

@boschb
Copy link
Author

@boschb boschb commented Feb 13, 2017

Awaitility wont work as this is happening in integration/loadtesting too... however yes I can probably fall back on a ConcurrentMap and just do the updates inline manually. I may just do that actually. I've gotten so used to just using a Cache instead of a Map because of the nice features.

I would say I think there is still merit to bubbling up the Future (especially since it's there). Can't hurt, and I would imagine any implementor has similar bits of code with the Loader that you do. there's always immediateFuture too. I'm not sold on putting Future stuff in the non-async version though, even though the documentation says it can happen asynchronously.

Anyways thanks for the help and let me know if it makes it in 3.0.
Cheers,
Bobby

@costimuraru
Copy link

@costimuraru costimuraru commented Apr 3, 2017

👍
I was a bit surprised to see that the refresh() method returns void and there is no way to be notified when the value is loaded. In my use case if I detect my value is stale, I want to refresh() and block until it's loaded.

@ben-manes
Copy link
Owner

@ben-manes ben-manes commented Apr 3, 2017

It never came up for Guava's Cache, perhaps because chained futures was a less commonly used idiom. Because of that, it didn't occur to me that users would take advantage of it. Most often refreshAfterWrite was relied on, instead.

I added this to the roadmap. Due to the API change it will require the 3.0 release, which will JDK9 based (to leverage VarHandles). If I can toggle between Unsafe and VarHandles, I might make it JDK8 compatible.

@costimuraru
Copy link

@costimuraru costimuraru commented Apr 3, 2017

I believe I got it to work using invalidate().

CacheValue cachedValue = lookupCache.get(cacheKey);
if (cachedValue.isExpired()) {
    // Refresh value.
    lookupCache.invalidate(cacheKey);
    cachedValue = lookupCache.get(cacheKey);
}

The only thing that bugs me are the cache statistics. In the case when the value is expired (related to #114), the above code will generate 2 cache requests (1 hit and 1 miss). Ideally, this should be 1 cache request with 1 miss.

By the way, great job with this library! This is gold.

@ben-manes
Copy link
Owner

@ben-manes ben-manes commented Apr 3, 2017

To avoid race conditions of an load/invalidation storm, you should use cache.asMap().remove(cacheKey, cachedValue).

I think variable expiration will solve your problem better, so hopefully I'll get it stabilized in a week or two.

@costimuraru
Copy link

@costimuraru costimuraru commented Apr 3, 2017

Thanks, Ben! I've updated the code with your recommendation.
The variable expiration would be cool and the hash timer wheel sounds intriguing.

@ben-manes
Copy link
Owner

@ben-manes ben-manes commented May 10, 2017

2.5.0 added support for variable expiration. This would resolve @costimuraru's use-case.

I'm leaving this issue open for @boschb's scenario, which is to return the internal refersh future to the client so that they can add their own chain. That requires changing the API, so we have to wait for v3.0. The hold off on that is to make it a JDK9 release, though with the Jigsaw debacle I don't know what the timeline is.

@ben-manes
Copy link
Owner

@ben-manes ben-manes commented Sep 21, 2017

Java 9 was released today, so my goal will be to release v3 by the end of the year. Hopefully sooner.

There are a bunch of tasks in the roadmap. Most are simple, except for batch refresh and JCache expiration overhaul. Most likely I'll keep punting on the refresh task, since that can be added scaffolded onto the interfaces using default methods.

@blizzard384
Copy link

@blizzard384 blizzard384 commented Mar 24, 2019

This feature would be much appreciated. I have an usecase when I want to send push notification after cache refresh() is finished.

@ben-manes
Copy link
Owner

@ben-manes ben-manes commented Jan 2, 2021

Sorry for not making progress on v3. I think all the blockers are resolved, so I am reviving the branch and working through these tasks.

I plan on returning CompletableFuture<V>, where this future is the instance supplied by the cache loader (via asyncReload or asyncLoad, as appropriate). There are some minor implications of this,

  1. This matches AsyncCache calls which always hand back the user supplied future. This is more flexible if the future needs to be operated on, e.g. cancelation, or if managed elsewhere where the reference equality matters. A chained future doesn't propagate upstream, so I try to not return those for less surprising usage.
  2. This means that adding chained logic goes against the original future. Because CompletableFuture chains as a stack, the last chained action is performed first (LIFO). That has a surprise where Caffeine's callbacks to update the hashmap entry is performed after the user's callbacks.
  3. The reloaded value may be discarded by the cache if the entry was updated or removed, so the refresh might not be applied anyways. This is to avoid the ABA problem and matches Guava's behavior.
  4. The feature request suggests returning Void, but I see no harm in the actual V value if the caveats are understood.
ben-manes added a commit that referenced this issue Jan 2, 2021
A mapping of in-flight refreshes is now maintained and lazily
initialized if not used. This allows the cache to ignore redundant
requests for reloads, like Guava does. It also removes disablement
of expiration during refresh and resolves an ABA problem if the
entry is modified in an undectectable way. The refresh future can
now be obtained from LoadingCache to chain operations against.

TODO: unit tests for these changes

fixes #143
fixes #193
fixes #236
fixes #282
fixes #322
fixed #373
fixes #467
ben-manes added a commit that referenced this issue Jan 2, 2021
A mapping of in-flight refreshes is now maintained and lazily
initialized if not used. This allows the cache to ignore redundant
requests for reloads, like Guava does. It also removes disablement
of expiration during refresh and resolves an ABA problem if the
entry is modified in an undectectable way. The refresh future can
now be obtained from LoadingCache to chain operations against.

TODO: unit tests for these changes

fixes #143
fixes #193
fixes #236
fixes #282
fixes #322
fixed #373
fixes #467
ben-manes added a commit that referenced this issue Jan 2, 2021
A mapping of in-flight refreshes is now maintained and lazily
initialized if not used. This allows the cache to ignore redundant
requests for reloads, like Guava does. It also removes disablement
of expiration during refresh and resolves an ABA problem if the
entry is modified in an undectectable way. The refresh future can
now be obtained from LoadingCache to chain operations against.

TODO: unit tests for these changes

fixes #143
fixes #193
fixes #236
fixes #282
fixes #322
fixed #373
fixes #467
ben-manes added a commit that referenced this issue Jan 2, 2021
A mapping of in-flight refreshes is now maintained and lazily
initialized if not used. This allows the cache to ignore redundant
requests for reloads, like Guava does. It also removes disablement
of expiration during refresh and resolves an ABA problem if the
entry is modified in an undectectable way. The refresh future can
now be obtained from LoadingCache to chain operations against.

TODO: unit tests for these changes

fixes #143
fixes #193
fixes #236
fixes #282
fixes #322
fixed #373
fixes #467
ben-manes added a commit that referenced this issue Jan 3, 2021
A mapping of in-flight refreshes is now maintained and lazily
initialized if not used. This allows the cache to ignore redundant
requests for reloads, like Guava does. It also removes disablement
of expiration during refresh and resolves an ABA problem if the
entry is modified in a previously undectectable way. The refresh
future can now be obtained from LoadingCache to chain operations
against.

fixes #143
fixes #193
fixes #236
fixes #282
fixes #322
fixed #373
fixes #467
ben-manes added a commit that referenced this issue Jan 3, 2021
A mapping of in-flight refreshes is now maintained and lazily
initialized if not used. This allows the cache to ignore redundant
requests for reloads, like Guava does. It also removes disablement
of expiration during refresh and resolves an ABA problem if the
entry is modified in a previously undectectable way. The refresh
future can now be obtained from LoadingCache to chain operations
against.

fixes #143
fixes #193
fixes #236
fixes #282
fixes #322
fixed #373
fixes #467
ben-manes added a commit that referenced this issue Jan 3, 2021
A mapping of in-flight refreshes is now maintained and lazily
initialized if not used. This allows the cache to ignore redundant
requests for reloads, like Guava does. It also removes disablement
of expiration during refresh and resolves an ABA problem if the
entry is modified in a previously undectectable way. The refresh
future can now be obtained from LoadingCache to chain operations
against.

fixes #143
fixes #193
fixes #236
fixes #282
fixes #322
fixed #373
fixes #467
ben-manes added a commit that referenced this issue Jan 4, 2021
A mapping of in-flight refreshes is now maintained and lazily
initialized if not used. This allows the cache to ignore redundant
requests for reloads, like Guava does. It also removes disablement
of expiration during refresh and resolves an ABA problem if the
entry is modified in a previously undectectable way. The refresh
future can now be obtained from LoadingCache to chain operations
against.

fixes #143
fixes #193
fixes #236
fixes #282
fixes #322
fixed #373
fixes #467
ben-manes added a commit that referenced this issue Jan 4, 2021
A mapping of in-flight refreshes is now maintained and lazily
initialized if not used. This allows the cache to ignore redundant
requests for reloads, like Guava does. It also removes disablement
of expiration during refresh and resolves an ABA problem if the
entry is modified in a previously undectectable way. The refresh
future can now be obtained from LoadingCache to chain operations
against.

fixes #143
fixes #193
fixes #236
fixes #282
fixes #322
fixed #373
fixes #467
ben-manes added a commit that referenced this issue Jan 4, 2021
A mapping of in-flight refreshes is now maintained and lazily
initialized if not used. This allows the cache to ignore redundant
requests for reloads, like Guava does. It also removes disablement
of expiration during refresh and resolves an ABA problem if the
entry is modified in a previously undectectable way. The refresh
future can now be obtained from LoadingCache to chain operations
against.

fixes #143
fixes #193
fixes #236
fixes #282
fixes #322
fixed #373
fixes #467
@boschb
Copy link
Author

@boschb boschb commented Jan 4, 2021

Hi Ben,

Happy New Year!

I think this mostly sounds all fine.

My only comment is for #2 regarding LIFO. In practice the timing of issuing the callback before the refresh completes seems problematic. This would be a timing hole (though tiny) in testing and in production it appears, but maybe i'm reading it wrong. Having a V instead of Void sorta avoids the need to access the entry via GET, but I could easily see a caller waiting for refresh() only to then access the current value in the map. If this is still racing, it could yield null or another value that got applied after the refresh(). Short of contention situation, i would expect refresh(K) == get(K) without race conditions.

I see this gets quite complex with contention, but maybe its possible to return future that delegates to the completion of future modifying the map, but uses the value from the refresh() operation? This also seems a little round about to me, but maybe it's more correct? Hard to say...

Thoughts?

ben-manes added a commit that referenced this issue Feb 8, 2021
A mapping of in-flight refreshes is now maintained and lazily
initialized if not used. This allows the cache to ignore redundant
requests for reloads, like Guava does. It also removes disablement
of expiration during refresh and resolves an ABA problem if the
entry is modified in a previously undectectable way. The refresh
future can now be obtained from LoadingCache to chain operations
against.

fixes #143
fixes #193
fixes #236
fixes #282
fixes #322
fixed #373
fixes #467
ben-manes added a commit that referenced this issue Feb 8, 2021
A mapping of in-flight refreshes is now maintained and lazily
initialized if not used. This allows the cache to ignore redundant
requests for reloads, like Guava does. It also removes disablement
of expiration during refresh and resolves an ABA problem if the
entry is modified in a previously undectectable way. The refresh
future can now be obtained from LoadingCache to chain operations
against.

fixes #143
fixes #193
fixes #236
fixes #282
fixes #322
fixed #373
fixes #467
ben-manes added a commit that referenced this issue Feb 8, 2021
A mapping of in-flight refreshes is now maintained and lazily
initialized if not used. This allows the cache to ignore redundant
requests for reloads, like Guava does. It also removes disablement
of expiration during refresh and resolves an ABA problem if the
entry is modified in a previously undectectable way. The refresh
future can now be obtained from LoadingCache to chain operations
against.

fixes #143
fixes #193
fixes #236
fixes #282
fixes #322
fixed #373
fixes #467
ben-manes added a commit that referenced this issue Feb 14, 2021
A mapping of in-flight refreshes is now maintained and lazily
initialized if not used. This allows the cache to ignore redundant
requests for reloads, like Guava does. It also removes disablement
of expiration during refresh and resolves an ABA problem if the
entry is modified in a previously undectectable way. The refresh
future can now be obtained from LoadingCache to chain operations
against.

fixes #143
fixes #193
fixes #236
fixes #282
fixes #322
fixed #373
fixes #467
ben-manes added a commit that referenced this issue Feb 14, 2021
A mapping of in-flight refreshes is now maintained and lazily
initialized if not used. This allows the cache to ignore redundant
requests for reloads, like Guava does. It also removes disablement
of expiration during refresh and resolves an ABA problem if the
entry is modified in a previously undectectable way. The refresh
future can now be obtained from LoadingCache to chain operations
against.

fixes #143
fixes #193
fixes #236
fixes #282
fixes #322
fixed #373
fixes #467
ben-manes added a commit that referenced this issue Feb 14, 2021
A mapping of in-flight refreshes is now maintained and lazily
initialized if not used. This allows the cache to ignore redundant
requests for reloads, like Guava does. It also removes disablement
of expiration during refresh and resolves an ABA problem if the
entry is modified in a previously undectectable way. The refresh
future can now be obtained from LoadingCache to chain operations
against.

fixes #143
fixes #193
fixes #236
fixes #282
fixes #322
fixed #373
fixes #467
ben-manes added a commit that referenced this issue Feb 15, 2021
A mapping of in-flight refreshes is now maintained and lazily
initialized if not used. This allows the cache to ignore redundant
requests for reloads, like Guava does. It also removes disablement
of expiration during refresh and resolves an ABA problem if the
entry is modified in a previously undectectable way. The refresh
future can now be obtained from LoadingCache to chain operations
against.

fixes #143
fixes #193
fixes #236
fixes #282
fixes #322
fixed #373
fixes #467
ben-manes added a commit that referenced this issue Feb 15, 2021
A mapping of in-flight refreshes is now maintained and lazily
initialized if not used. This allows the cache to ignore redundant
requests for reloads, like Guava does. It also removes disablement
of expiration during refresh and resolves an ABA problem if the
entry is modified in a previously undectectable way. The refresh
future can now be obtained from LoadingCache to chain operations
against.

fixes #143
fixes #193
fixes #236
fixes #282
fixes #322
fixed #373
fixes #467
ben-manes added a commit that referenced this issue Feb 15, 2021
A mapping of in-flight refreshes is now maintained and lazily
initialized if not used. This allows the cache to ignore redundant
requests for reloads, like Guava does. It also removes disablement
of expiration during refresh and resolves an ABA problem if the
entry is modified in a previously undectectable way. The refresh
future can now be obtained from LoadingCache to chain operations
against.

fixes #143
fixes #193
fixes #236
fixes #282
fixes #322
fixed #373
fixes #467
ben-manes added a commit that referenced this issue Feb 15, 2021
A mapping of in-flight refreshes is now maintained and lazily
initialized if not used. This allows the cache to ignore redundant
requests for reloads, like Guava does. It also removes disablement
of expiration during refresh and resolves an ABA problem if the
entry is modified in a previously undectectable way. The refresh
future can now be obtained from LoadingCache to chain operations
against.

fixes #143
fixes #193
fixes #236
fixes #282
fixes #322
fixed #373
fixes #467
ben-manes added a commit that referenced this issue Feb 15, 2021
A mapping of in-flight refreshes is now maintained and lazily
initialized if not used. This allows the cache to ignore redundant
requests for reloads, like Guava does. It also removes disablement
of expiration during refresh and resolves an ABA problem if the
entry is modified in a previously undectectable way. The refresh
future can now be obtained from LoadingCache to chain operations
against.

fixes #143
fixes #193
fixes #236
fixes #282
fixes #322
fixed #373
fixes #467
ben-manes added a commit that referenced this issue Feb 16, 2021
A mapping of in-flight refreshes is now maintained and lazily
initialized if not used. This allows the cache to ignore redundant
requests for reloads, like Guava does. It also removes disablement
of expiration during refresh and resolves an ABA problem if the
entry is modified in a previously undectectable way. The refresh
future can now be obtained from LoadingCache to chain operations
against.

fixes #143
fixes #193
fixes #236
fixes #282
fixes #322
fixed #373
fixes #467
@ben-manes ben-manes closed this in 551ade7 Feb 21, 2021
@ben-manes
Copy link
Owner

@ben-manes ben-manes commented Feb 21, 2021

Released in 3.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants