Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

IMap.values(predicateWithoutIndex) does not update TTL metadata #15896

Open
vojtechtoman opened this issue Oct 30, 2019 · 2 comments
Open

IMap.values(predicateWithoutIndex) does not update TTL metadata #15896

vojtechtoman opened this issue Oct 30, 2019 · 2 comments

Comments

@vojtechtoman
Copy link
Contributor

vojtechtoman commented Oct 30, 2019

#15136 introduced eviction support on index level. As a result, API calls such as IMap.values(predicate) update the TTL metadata (lastAccessTime, ...) of the returned records if an index is used. However, if the predicate doesn't use an index, the TTL metadata is not updated.

Reproducer:

@Test
public void reproducer() {
    String mapName = randomMapName();
    Config config = getConfig();
    HazelcastInstance node = createHazelcastInstance(config);
    IMap<Integer, String> map = node.getMap(mapName);
    map.put(0, "foo", 0L, SECONDS, 60L, SECONDS);
    assertEquals(1, map.values(Predicates.alwaysTrue()).size());
    assertTrue(map.getEntryView(0).getLastAccessTime() > 0);
}

This affects also map.values() which is implemented as map.values(Predicates.alwaysTrue()) internally.

@mmedenjak
Copy link
Contributor

As we're heavily invested in the new query engine, I don't think we should spend time on this. Still, it raises a question - if an SQL query touches an entry, should it update the statistics? As much as I would hope so, I think it might be prohibitively expensive and we might have to consider some different approach to read stats and TTL metadata. cc @devozerov

@mmedenjak mmedenjak modified the milestones: 4.1, Backlog Sep 2, 2020
@devozerov
Copy link
Contributor

Yes, the straightforward update of the TTL for every entry could be prohibitively expensive. Consider a query that requires a full scan of the map (these are all queries that cannot use indexes) - it will update TTL for the whole map on every execution of the SQL request.

Maybe some sort of batching could help here. E.g. accumulate the number of scans, and then update entries accordingly. Or track the last timestamp of the full scan and update entries TTL when it is about to evicted.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants