Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

IMap with mapstore has duplicate keys #11462

Closed
tvd12 opened this issue Sep 27, 2017 · 14 comments
Closed

IMap with mapstore has duplicate keys #11462

tvd12 opened this issue Sep 27, 2017 · 14 comments

Comments

@tvd12
Copy link

@tvd12 tvd12 commented Sep 27, 2017

97043970-8cb5-4bc9-94b8-7dd1b9c127f8

@tvd12 tvd12 changed the title IMap duplicate keys IMap (has mapstore) duplicate keys Sep 27, 2017
@ahmetmircik
Copy link
Member

@ahmetmircik ahmetmircik commented Sep 27, 2017

Hi @tvd12 can you also please share a reproducer + the hazelcast version?

@tvd12
Copy link
Author

@tvd12 tvd12 commented Sep 27, 2017

I've used Hazelcast 3.8.1

@tvd12
Copy link
Author

@tvd12 tvd12 commented Sep 27, 2017

I'm sorry, my source code is private (my company policy).
I simply create a mapstore and put some value to its (IMap<Integer,JackpotBankInitValue>)

@tombujok
Copy link
Contributor

@tombujok tombujok commented Sep 27, 2017

Just share a small reproducer in a separate project - just minimal code that will reproduce it, without any of your company's code.
we don't see this behaviour in our tests.

@tombujok tombujok added the Team: Core label Sep 27, 2017
@tombujok tombujok changed the title IMap (has mapstore) duplicate keys IMap with mapstore has duplicate keys Sep 27, 2017
@tvd12
Copy link
Author

@tvd12 tvd12 commented Sep 27, 2017

Mapstore:

 @Override
    public JackpotBankInitValue load(Integer key) {
        return storeService.getInitValue(key);
    }

    @Override
    public Map<Integer, JackpotBankInitValue> loadAll(Collection<Integer> keys) {
    	Map<Integer, JackpotBankInitValue> map = new HashMap<>();
    	for(Integer key : loadAllKeys())
    		map.put(key, load(key));
    	return map;
    }

@Override
    public Iterable<Integer> loadAllKeys() {
        return Lists.newArrayList(0, 1, 2);
    }

I use hazelcast client to push value to map:

public void save(JackpotBankInitValue jackpot, Integer id) {
		IMap<Integer, JackpotBankInitValue> imap = hzInstance.getMap(Entities.JACKPOT_BANK_INIT_VALUE);
		imap.set(id, jackpot);
	}
@tvd12
Copy link
Author

@tvd12 tvd12 commented Sep 27, 2017

It's enough to you?

@ahmetmircik
Copy link
Member

@ahmetmircik ahmetmircik commented Sep 27, 2017

the map you gave the content above cannot be imap, its internal representation is different.
From where did you get the map?

@tvd12
Copy link
Author

@tvd12 tvd12 commented Sep 27, 2017

I get the map from Hazelcast client.

24a56e6c-d354-479f-934c-342853e41add

@ahmetmircik
Copy link
Member

@ahmetmircik ahmetmircik commented Sep 27, 2017

so you get the map above after calling imap#getAll ?

What was the method call on line 24?

@tvd12
Copy link
Author

@tvd12 tvd12 commented Sep 27, 2017

You can see in image, I haven't called imap#getAll, I only get imap from Hazelcast Client instance

@tvd12
Copy link
Author

@tvd12 tvd12 commented Sep 27, 2017

My source code:

public Map<Integer, JackpotBankInitValue> getJackpotInitValue() {
		IMap<Long, JackpotBankInitValue> imap = hzInstance.getMap(Entities.JACKPOT_BANK_INIT_VALUE);
		/*line 24*/Map<Integer, JackpotBankInitValue> jackpotInit = new HashMap<>();
		for (JackpotBankInitValue jp : imap.values()) {
			jackpotInit.put(jp.getBettingTypeId(), jp);
		}
		return jackpotInit;
	}
@ahmetmircik
Copy link
Member

@ahmetmircik ahmetmircik commented Sep 27, 2017

Seems you need to fix loadAll implementation, please use below fixed version:

@Override
    public Map<Integer, JackpotBankInitValue> loadAll(Collection<Integer> keys) {
    	Map<Integer, JackpotBankInitValue> map = new HashMap<>();
    	for(Integer key : keys) <-- don't call loadAllKeys(), it causes the duplicates
    		map.put(key, load(key));
    	return map;
    }
@tvd12
Copy link
Author

@tvd12 tvd12 commented Sep 27, 2017

I fixed that, but I think we shouldn't have a map with duplicated keys.

@ahmetmircik
Copy link
Member

@ahmetmircik ahmetmircik commented Sep 27, 2017

You are right, I will mark this as an enhancement issue. Let's see what we can have.
Thanks for the report.

@ahmetmircik ahmetmircik modified the milestones: 3.9.1, 3.10 Sep 27, 2017
@mmedenjak mmedenjak added the To Triage label Oct 31, 2017
@taburet taburet added Team: Core and removed Team: Core labels Jan 22, 2018
@blazember blazember self-assigned this Jan 23, 2018
blazember added a commit to blazember/hazelcast that referenced this issue Jan 23, 2018
Contract of MapLoader.loadAll(keys) defines exactly which items should be loaded from the map store.
Loading other items may lead to duplications in the map and to wasted resources in form of increased memory usage, load time and replication costs.
This fix makes loadAll failing fast if keys other than the given ones are loaded.

Fixes hazelcast#11462
blazember added a commit to blazember/hazelcast that referenced this issue Jan 23, 2018
Contract of MapLoader.loadAll(keys) defines exactly which items should be loaded from the map store.
Loading other items may lead to duplications in the map and to wasted resources in form of increased memory usage, load time and replication costs.
This fix makes loadAll failing fast if keys other than the given ones are loaded.

Fixes hazelcast#11462
blazember added a commit to blazember/hazelcast that referenced this issue Jan 23, 2018
Contract of MapLoader.loadAll(keys) defines exactly which items should be loaded from the map store.
Loading other items may lead to duplications in the map and to wasted resources in form of increased memory usage, load time and replication costs.
This fix makes loadAll failing fast if keys other than the given ones are loaded.

Fixes hazelcast#11462
@mmedenjak mmedenjak removed the To Triage label Jan 23, 2018
blazember added a commit to blazember/hazelcast that referenced this issue Jan 25, 2018
Contract of MapLoader.loadAll(keys) defines exactly which items should be loaded from the map store.
Loading other items may lead to duplications in the map and to wasted resources in form of increased memory usage, load time and replication costs.
This fix makes loadAll failing fast if keys other than the given ones are loaded.

Fixes hazelcast#11462
blazember added a commit to blazember/hazelcast that referenced this issue Jan 25, 2018
Contract of MapLoader.loadAll(keys) defines exactly which items should be loaded from the map store.
Loading other items may lead to duplications in the map and to wasted resources in form of increased memory usage, load time and replication costs.
This fix makes loadAll failing fast if keys other than the given ones are loaded.

Fixes hazelcast#11462
blazember added a commit that referenced this issue Feb 2, 2018
Fail if MapLoader.loadAll loads unexpected keys
@Holmistr Holmistr added the DR3 label Feb 2, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

8 participants
You can’t perform that action at this time.