DataLoader support for get-if-present

[jhaber]

I'm wondering if you have any thoughts on adding a method to DataLoader to support get-if-present semantics (could be called getIfPresent, getIfCached, getCachedValue, etc.)

I have a use-case where there's an API that supports fetching either a partial or complete object. In my service, I have separate DataLoaders for these types of fetches. As you might expect, when I need to fetch the partial object I use the partial DataLoader, and when I need to fetch the full object I use the other DataLoader. However, if I've already fetched the full object then theoretically I can use that cached result to satisfy any subsequent partial fetches that come in. To support this, I want the partial DataLoader to first do a get-if-present on the complete DataLoader (without triggering a load)

Let me know if this use-case makes sense and if you would consider a PR that implements it

[bbakerman]

Can you leave some psudeo code here to explain what you want to do in more detail.

I think I know what you want but some clear psuedo could would be helpful

Off hand you can share org.dataloader.CacheMap behind the partial and full dataloaders and hence you could be loading full objects from both say if they have the same key.

The DataLoader today will ALWAYs retreive from cache if 1) its enabled and 2) the entry is in a org.dataloader.CacheMap

[jhaber]

Sure, I'll try to give some code for a simplified version. Let's say we have a UserService that supports CompletionStage<Map<Long, User>> getByIds(Set<Long> ids) and CompletionStage<Set<User>> getAll() and we have dataloaders in front of both operations:

public class UsersLoader implements MappedBatchLoader<Long, User> {

  public CompletionStage<Map<Long, User>> load(Set<Long> keys) {
    return usersService.getByIds(keys);
  }
}

public class AllUsersLoader implements BatchLoader<SingletonKey, Set<User>> {

  public CompletionStage<List<List<User>>> load(List<SingletonKey> keys) {
    return usersService.getAll().thenApply(users -> Collections.singletonList(users));
  }
}

(uses a dummy key like enum SingletonKey { INSTANCE })

Some codepaths only need individual users and do point lookups, while other codepaths need all users. However, sometimes we end up hitting both data loaders. Ideally we don't need to fetch all users, but once we do, we might as well reuse that result to satisfy any subsequent point-lookups. So I want to update the implementation for the UsersLoader to something like:

public class UsersLoader implements MappedBatchLoader<Long, User> {

  public CompletionStage<Map<Long, User>> load(Set<Long> keys) {
    if (allUsersDataLoader.alreadyFetched(SingletonKey.INSTANCE)) {
      return allUsersDataLoader.load(SingletonKey.INSTANCE)
          .thenApply(allUsers -> filterAndConvertToMap(allUsers, keys))
    } else {
      return usersService.getByIds(keys);
    }
  }
}

So my suggestion would be something in the DataLoader API that lets users determine whether a particular key has already been loaded, without triggering a load

[bbakerman]

This is very releated to #50

And as I said there - remember the cache in DataLoader is the cache to a CompleteableFuture - and not a Value

So Optional<CompleteaFuture<V>> dataLoader.getLoadPromise(K key)` would be the semantics.

That is here is the cached promise to load a value (which may or may not be satisfied yet) not the value itself.

[bbakerman]

One other thought is to have getIfPresent be a test of cached-ness AND CF completed ness

That is it only returns the CF if its both cached AND completed. eg java.util.concurrent.CompletableFuture#isDone

[bbakerman]

I started a proof of concept here

#52

is this what you are thinking?

[bbakerman]

I have released 2.2.3 with getIfPresent and getIfCompleted support

[jhaber]

Thanks so much, this should work great for my use-case