Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stream#distinct isn't memory safe #6584

scabug opened this issue Oct 29, 2012 · 5 comments

Stream#distinct isn't memory safe #6584

scabug opened this issue Oct 29, 2012 · 5 comments


Copy link

scabug commented Oct 29, 2012

$ JAVA_OPTS='-ms512m -mx512m -XX:+PrintGCDetails' scala
Welcome to Scala version 2.9.2 (Java HotSpot(TM) 64-Bit Server VM, Java 1.6.0_33).
Type in expressions to have them evaluated.
Type :help for more information.

scala> Array.tabulate(10*1024)(x => x).distinct.size
[GC [ParNew: 52480K->6528K(59008K), 0.0943029 secs] 52480K->25772K(517760K), 0.0943769 secs] [Times: user=0.32 sys=0.10, real=0.10 secs] 
res0: Int = 10240

scala> Vector.tabulate(10*1024)(x => x).distinct.size
res1: Int = 10240

scala> List.tabulate(10*1024)(x => x).distinct.size
res2: Int = 10240

scala> Stream.tabulate(10*1024)(x => x).distinct.size
[GC [ParNew: 59008K->6528K(59008K), 0.0421088 secs] 78252K->33191K(517760K), 0.0421588 secs] [Times: user=0.14 sys=0.10, real=0.04 secs] 
[GC [ParNew: 59008K->6528K(59008K), 0.1036809 secs] 85671K->94187K(517760K), 0.1037176 secs] [Times: user=0.58 sys=0.05, real=0.10 secs] 
[GC [ParNew: 59008K->6528K(59008K), 0.1205132 secs] 146667K->149422K(517760K), 0.1205500 secs] [Times: user=0.72 sys=0.04, real=0.12 secs] 
[GC [ParNew: 59008K->6528K(59008K), 0.1134407 secs] 201902K->202718K(517760K), 0.1134693 secs] [Times: user=0.68 sys=0.04, real=0.11 secs] 
[GC [ParNew: 59008K->6528K(59008K), 0.1113360 secs] 255198K->255860K(517760K), 0.1113733 secs] [Times: user=0.65 sys=0.03, real=0.11 secs] 
[GC [1 CMS-initial-mark: 249332K(458752K)] 256833K(517760K), 0.0080780 secs] [Times: user=0.01 sys=0.00, real=0.01 secs] 
[GC [ParNew: 59008K->6528K(59008K), 0.1217139 secs] 308340K->308492K(517760K), 0.1217432 secs] [Times: user=0.71 sys=0.04, real=0.12 secs] 
[GC [ParNew: 59008K->6526K(59008K), 0.1232082 secs] 360972K->361132K(517760K), 0.1232456 secs] [Times: user=0.74 sys=0.03, real=0.12 secs] 
[GC [ParNew: 59006K->6526K(59008K), 0.1220145 secs] 413612K->413631K(517760K), 0.1220461 secs] [Times: user=0.70 sys=0.03, real=0.12 secs] 
[GC [ParNew: 59006K->59006K(59008K), 0.0000176 secs][CMS[CMS-concurrent-mark: 0.487/0.858 secs] [Times: user=3.32 sys=0.10, real=0.86 secs] 
 (concurrent mode failure): 407104K->443498K(458752K), 1.8997048 secs] 466111K->443498K(517760K), [CMS Perm : 37814K->37754K(37952K)], 1.8998418 secs] [Times: user=2.19 sys=0.02, real=1.90 secs] 
[GC [1 CMS-initial-mark: 443498K(458752K)] 444541K(517760K), 0.0004659 secs] [Times: user=0.00 sys=0.00, real=0.00 secs] 
[GC [ParNew: 52480K->52480K(59008K), 0.0000151 secs][CMS[CMS-concurrent-mark: 0.517/0.544 secs] [Times: user=1.08 sys=0.01, real=0.54 secs] 
 (concurrent mode failure): 443498K->458751K(458752K), 2.1793441 secs] 495978K->495952K(517760K), [CMS Perm : 37754K->37754K(62928K)], 2.1794330 secs] [Times: user=2.64 sys=0.01, real=2.18 secs] 
[Full GC [CMS: 458751K->458751K(458752K), 1.8047112 secs] 517759K->517586K(517760K), [CMS Perm : 37754K->37704K(62928K)], 1.8048090 secs] [Times: user=1.80 sys=0.01, real=1.80 secs] 
[GC [1 CMS-initial-mark: 458751K(458752K)] 517586K(517760K), 0.0628709 secs] [Times: user=0.07 sys=0.00, real=0.06 secs] 
[Full GC [CMS[CMS-concurrent-mark: 0.592/0.600 secs] [Times: user=1.19 sys=0.00, real=0.60 secs] 
 (concurrent mode failure): 458751K->458751K(458752K), 2.3928489 secs] 517759K->517738K(517760K), [CMS Perm : 37704K->37697K(62928K)], 2.3929299 secs] [Times: user=2.98 sys=0.00, real=2.40 secs] 
[Full GC [CMS: 458751K->458752K(458752K), 1.7668880 secs] 517759K->517759K(517760K), [CMS Perm : 37697K->37697K(62928K)], 1.7669610 secs] [Times: user=1.77 sys=0.00, real=1.77 secs] 
[Full GC [CMS: 458752K->458752K(458752K), 1.4847975 secs] 517759K->517759K(517760K), [CMS Perm : 37697K->37697K(62928K)], 1.4848679 secs] [Times: user=1.48 sys=0.01, real=1.48 secs] 
[Full GC [CMS: 458752K->458752K(458752K), 1.4837943 secs] 517759K->517759K(517760K), [CMS Perm : 37697K->37697K(62928K)], 1.4838669 secs] [Times: user=1.48 sys=0.00, real=1.48 secs] 
[GC [1 CMS-initial-mark: 458752K(458752K)] 517759K(517760K), 0.0631032 secs] [Times: user=0.07 sys=0.00, real=0.06 secs] 
[Full GC [CMS[CMS-concurrent-mark: 0.643/0.644 secs] [Times: user=1.28 sys=0.00, real=0.64 secs] 
 (concurrent mode failure): 458752K->27470K(458752K), 1.0284691 secs] 517759K->27470K(517760K), [CMS Perm : 37700K->37700K(62928K)], 1.0284995 secs] [Times: user=1.67 sys=0.00, real=1.03 secs] 
java.lang.OutOfMemoryError: Java heap space
        at scala.collection.immutable.Stream$.filteredTail(Stream.scala:1124)
        at scala.collection.immutable.Stream.filter(Stream.scala:471)
        at scala.collection.immutable.Stream$$anonfun$filteredTail$1.apply(Stream.scala:1124)
        at scala.collection.immutable.Stream$$anonfun$filteredTail$1.apply(Stream.scala:1124)
        at scala.collection.immutable.Stream$Cons.tail(Stream.scala:1060)
        at scala.collection.immutable.Stream$Cons.tail(Stream.scala:1052)
Copy link

scabug commented Oct 29, 2012

Imported From:
Reporter: Dan Brown (jdanbrown)
Affected Versions: 2.9.2

Copy link

scabug commented Oct 30, 2012

@paulp said:

Copy link

scabug commented Nov 1, 2012

@paulp said:

Copy link

scabug commented Jan 22, 2013

@adriaanm said:
reopening for 2.10.1-RC1 backport

@scabug scabug closed this as completed Feb 4, 2013
Copy link

scabug commented Feb 9, 2013

@adriaanm said (edited on Feb 9, 2013 9:48:46 PM UTC):
This change was not binary compatible with 2.10.0, so backport reverted.

See scala/scala#2103.

@scabug scabug added this to the 2.11.0-M1 milestone Apr 7, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
None yet

No branches or pull requests

2 participants