Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Scala 2.12 map.keySet leaks reference to map #11727

Open
fanf opened this issue Sep 5, 2019 · 7 comments

Comments

@fanf
Copy link

commented Sep 5, 2019

Hello, we discovered with prod pain and crashes that Map.keySet leaks a reference to the Map object so that it is not GC'ed and leaks memory - huge memory if objects in the map are big and you wanted to only return light identifiers to them.

It can be seen for ex here: https://gist.github.com/fanf/ff39065f7f9738467e40929ad7490da1

Is this known/wanted and I'm just 13 years late in that knowledge, or is it a bug (at least a bad documentation)?

The problem is likely the references to self in DefaultKeySet (contains, size...) https://github.com/scala/scala/blob/2.12.x/src/library/scala/collection/MapLike.scala#L176

@plokhotnyuk

This comment has been minimized.

Copy link

commented Sep 5, 2019

@fanf it looks that DefaultKeySet was implemented like a view (not a separated collection of keys)...

@szeiger

This comment has been minimized.

Copy link

commented Sep 5, 2019

I think it makes sense that keySet references the source map because it is usually the fastest and most straight-forward implementation.. But you wouldn't expect so from documentation such as

Collects all keys of this map in an iterable collection.

@fanf

This comment has been minimized.

Copy link
Author

commented Sep 5, 2019

Yes, it would be very appreciated to make obvious the fact that it's a view and not a new object, AND what is the best (most efficient) method to get a fresh, independant Set.
For reference, the naive Set()++map.keySet() we tested first does not work, most likelly because ++ is smart and see that one of the set is empty and just returns the second one.

@fanf

This comment has been minimized.

Copy link
Author

commented Sep 5, 2019

(I knew about values which is never what you want it to be, keySet is a new one :)

@SethTisue

This comment has been minimized.

Copy link
Member

commented Sep 5, 2019

best (most efficient) method to get a fresh, independant Set.

.keysIterator.toSet?

@fanf

This comment has been minimized.

Copy link
Author

commented Sep 5, 2019

@SethTisue most likely from my user point of view, but I don't know if there's better way :)

@plokhotnyuk

This comment has been minimized.

Copy link

commented Sep 5, 2019

Both keySet.toSeq.toSet and keysIterator.toSet methods for getting a separated set are vulnerable for hash collision based DoS attacks even if the original map (like scala.collection.mutable.CollisionProofHashMap or java.collection.LinkedHashMap) is safe: #11203

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.