Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

Backport #2605 to 2.10.x: SI-7149 Use a WeakHashSet for type uniqueness #2901

Merged
merged 3 commits into from Sep 4, 2013

Conversation

Projects
None yet
2 participants
Member

gkossakowski commented Aug 31, 2013

This is backport of #2605.

The issue of uniques accumulating types during the whole compilation run turned out to be especially bad for ScalaTest tests. Scala 2.10.2 requires at least 6GB to compile Scalatest tests. With this patch, 800MB is enough.

Such a dramatic difference in memory consumption and low-risk of this patch makes it appealing to backport it to 2.10.x.

Review by @JamesIry.

gkossakowski and others added some commits Sep 4, 2013

SI-7150 Replace scala.reflect.internal.WeakHashSet
Replaces scala.reflect.internal.WeakHashSet with a version that
* extends the mutable.Set trait
* doesn't leak WeakReferences
* is unit tested
SI-7149 Use a WeakHashSet for type uniqueness
Currently type uniqueness is done via a HashSet[Type], but
that means the Types live through an entire compile session, even
ones that are used once. The result is a huge amount of unnecessarily
retained memory. This commit uses a WeakHashSet instead so that Types
and their WeakReferences are cleaned up when no longer in use.
Modify perRunCaches to not leak WeakReferences
perRunCaches was using a HashMap of WeakReferences which meant it would
accumulate WeakReferences over time. This commit uses a WeakHashSet
instead so that the references are cleaned up.
Member

gkossakowski commented Sep 4, 2013

I fixed the binary compatibility errors. Hopefully tests will pass now.

Owner

retronym commented Sep 4, 2013

LGTM. I re-reviewed the change itself to re-convince myself that this can't hurt us.

Member

gkossakowski commented Sep 4, 2013

@retronym: thanks for the review. I'll go and merge it then.

gkossakowski added a commit that referenced this pull request Sep 4, 2013

Merge pull request #2901 from gkossakowski/backport-uniques-memory-fix
Backport #2605 to 2.10.x: SI-7149 Use a WeakHashSet for type uniqueness

@gkossakowski gkossakowski merged commit 068b9a3 into scala:2.10.x Sep 4, 2013

1 check passed

default pr-scala Took 69 min.
Details

@gkossakowski gkossakowski deleted the gkossakowski:backport-uniques-memory-fix branch Feb 7, 2014

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment