Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[map] IMap jmx statistics evicted together with the map entry #4321

Closed
dsukhoroslov opened this issue Dec 17, 2014 · 11 comments
Closed

[map] IMap jmx statistics evicted together with the map entry #4321

dsukhoroslov opened this issue Dec 17, 2014 · 11 comments

Comments

@dsukhoroslov
Copy link
Contributor

@dsukhoroslov dsukhoroslov commented Dec 17, 2014

Hazelcast 3.3.3 I have the following map config:

<hz:map name="xdm-xml" async-backup-count="1" backup-count="0" in-memory-format="OBJECT"
     eviction-policy="LRU" time-to-live-seconds="0" max-size-policy="USED_HEAP_PERCENTAGE" 
     max-size="25" eviction-percentage="10" max-idle-seconds="3600">
/hz:map>

When system starts and caches populated and get hits the stats present relevant values for localHits and localGetOperationCount attributes. But after 1 hour of idle some entries evicted and localHits/localGetOperationCount attributes became less and less.. till 0. Looks like you store get/hits statistics together with the entry itself and it gets evicted together with entry.

One more issue with stats: the map contains > 12M of values. Getting statistics for this map takes several minutes - looks like they're aggregated from the map entries too!

@hasancelik
Copy link
Contributor

@hasancelik hasancelik commented Dec 17, 2014

Hi Denis,
Would it be possible to provide a unit test that reproduces these issues? Reproducing issues often is a very time consuming process and when the test is already there, a lot of work is already done for us.

@dsukhoroslov
Copy link
Contributor Author

@dsukhoroslov dsukhoroslov commented Dec 18, 2014

Hi Hasan,

No, I have no unit test for this. But it should be quite simple: create an
IMap with eviction settings similair to what I posted in the issue,
populate the map with some values, get them and check map access stats.
Then for max-idle-time and check map stats again.

Thanks, Denis.
17 дек. 2014 г. 12:13 пользователь "Hasan Çelik" notifications@github.com
написал:

Hi Denis,
Would it be possible to provide a unit test that reproduces these issues?
Reproducing issues often is a very time consuming process and when the test
is already there, a lot of work is already done for us.


Reply to this email directly or view it on GitHub
#4321 (comment)
.

@pveentjer
Copy link
Member

@pveentjer pveentjer commented Dec 18, 2014

If you can create that test and post it here, then we can save a lot of time on our side.

@ahmetmircik
Copy link
Member

@ahmetmircik ahmetmircik commented Dec 18, 2014

Relevant issues:
#3538
#4253

@dsukhoroslov
Copy link
Contributor Author

@dsukhoroslov dsukhoroslov commented Dec 18, 2014

Hi Peter,

Ok, will try to find some time for this :-). Regarding the second issue
with too long time for stats retrieval - don't think I'll be able to
provide a relevant test for this. BTW, will the response time be improved
if I'll increase the number of partitions?

Thanks, Denis.
18 дек. 2014 г. 11:47 пользователь "Peter Veentjer" <
notifications@github.com> написал:

If you can create that test and post it here, then we can save a lot of
time on our side.


Reply to this email directly or view it on GitHub
#4321 (comment)
.

@hasancelik
Copy link
Contributor

@hasancelik hasancelik commented Dec 18, 2014

@dsukhoroslov
@pveentjer
Unit Test for first part of this issue:

    @Test
    public void testIssue4321() {

    final ILogger logger = Logger.getLogger(LocalMapStats.class.getName());

    String mapName = randomMapName();
    Config config = new Config();
    MapConfig mapConfig = config.getMapConfig(mapName);
    mapConfig.setEvictionPolicy(MapConfig.EvictionPolicy.LRU);
    mapConfig.setTimeToLiveSeconds(0);
    mapConfig.setEvictionPercentage(10);
    mapConfig.setMaxIdleSeconds(3);
    MaxSizeConfig maxSizeConfig = mapConfig.getMaxSizeConfig();
    maxSizeConfig.setMaxSizePolicy(MaxSizeConfig.MaxSizePolicy.USED_HEAP_PERCENTAGE);
    maxSizeConfig.setSize(25);

    HazelcastInstance instance = createHazelcastInstance(config);
    IMap<Object, Object> map = instance.getMap(mapName);
    final CountDownLatch entryEvictedLatch = new CountDownLatch(700);
    map.addEntryListener(new EntryAdapter<Object, Object>(){
        @Override
        public void entryEvicted(EntryEvent<Object, Object> event) {
            entryEvictedLatch.countDown();
        }
    },true);
    for (int i = 0; i < 1000; i++) {
        map.put(i, i);
        map.get(i);
    }
    LocalMapStats localMapStats = map.getLocalMapStats();
    assertEquals(1000, localMapStats.getHits());
    assertEquals(1000, localMapStats.getPutOperationCount());
    assertEquals(1000, localMapStats.getGetOperationCount());
    assertOpenEventually(entryEvictedLatch);
    localMapStats = map.getLocalMapStats();
    assertEquals(1000, localMapStats.getHits());
    assertEquals(1000, localMapStats.getPutOperationCount());
    assertEquals(1000, localMapStats.getGetOperationCount());

}

Failed with:
java.lang.AssertionError:
Expected :1000
Actual :265

@dsukhoroslov
Copy link
Contributor Author

@dsukhoroslov dsukhoroslov commented Dec 19, 2014

Have a look at #4253, it has the same root cause probably.

@stefano-salmaso
Copy link

@stefano-salmaso stefano-salmaso commented Mar 21, 2017

Hi all,
I fell into the same issue. Are there any updates on this?
If the number of "hits" is not related to the number of "gets" ... "hits" loses its value.

Thanks!

@samshay
Copy link

@samshay samshay commented Apr 10, 2017

Hi,
I have the same problem :(

@mmedenjak mmedenjak changed the title IMap jmx statistics evicted together with the map entry [map] IMap jmx statistics evicted together with the map entry Jul 14, 2017
@mmedenjak mmedenjak added this to the Backlog milestone Jul 14, 2017
@stefano-salmaso
Copy link

@stefano-salmaso stefano-salmaso commented Sep 7, 2017

Hi,
could we have an update on this?
For me having wrong statistics is worse than not having them!!!

Thanks
Stefano

@jerrinot
Copy link
Contributor

@jerrinot jerrinot commented Sep 7, 2017

Hi,

right now the aggregated stats indeed shows "hits/miss of entries currently present in the map". I agree this is unexpected it should be changed into "hits/miss since a map (record store) was created".

I am adding this from backlog into 3.10.

@Holmistr Holmistr added the DR1 label Nov 16, 2017
@vbekiaris vbekiaris self-assigned this Nov 22, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

10 participants
You can’t perform that action at this time.