Specialize Scala collections (Vector, in particular)

[scabug]

= problem =
Scala 2.9 RC1 has been released, and there are still no collections that have been specialized as far as I can tell from the source. Missing specialization for Vector in particular is a problem because the overhead of storing an Integer instead of an Int is about 5x on a 64 bit JVM, and Vector is the most performant collection we have.

I understand adding specialization everywhere would create code bloat, but just adding it to Vector alone (and not even IndexedSeqLike and such) would greatly reduce the memory consumption of these classes.

[scabug]

Imported From: https://issues.scala-lang.org/browse/SI-4495?orig=1
Reporter: Joshua Hartman (jhartman)

[scabug]

Kipton Barros (kbarros) said:
It would also be nice to have a specialized ArrayBuffer

[scabug]

@rklaehn said:
Adding @specialized to Vector is probably a can of worms. If it was straightforward it would have been done already. But there are some collections where adding @specialized would be pretty straightforward. For example HashSet.HashSet1. That would reduce the number of objects used for a HashSet by about a factor of two.

[scabug]

@soc said (edited on Aug 18, 2012 12:47:00 PM UTC):
I'm not sure if it really counts as "specialization", but there are a few other array-based data-structures which would immensely benefit from allocating JVM's "specialized" primitive array instead of Array[AnyRef] for storing the elements internally.

This could be done without emitting specialized bytecode for primitive types, it would need a TypeTag instead.

[scabug]

Harshad (hrj) said:
I just had this idea. What if specialized collections were to be distributed as a separate modular library, in line with the 2.11 effort of modularization?

I (naively) imagine that this could be implemented by having a compiler flag called, say, --emit which can take three values: all, specialized-only, generic-only. The default would be all.

While compiling the scala standard library, the scripts would use generic-only in the first run, and then specialized-only in the second run.

For most other libraries, default would work fine.

[SethTisue]

Out of scope for Scala 2