deserializing large mutable.HashMaps results in ArrayIndexOutOfBoundsExceptions

[scabug]

Basically, put 2147484 entries in a hashmap. write to disk. try to read from disk. kaboom! ArrayIndexOutOfBoundsExceptions. 2147483 is fine. Yeah, I did the binary search.

import scala.collection.mutable.HashMap
import java.io._
import java.util.zip._

case class Feature(x: Int) 

object Foo extends App {
  def test(size: Int) { 
    println(size)
    val map = new HashMap[Feature,Double]() ++= (0 until size map {Feature(_) -> 1.0})
    val oout = new ObjectOutputStream(new GZIPOutputStream(new BufferedOutputStream(new FileOutputStream("test.ser.gz"))))
    oout.writeObject(map)
    oout.close()
    val in = new ObjectInputStream(new GZIPInputStream(new BufferedInputStream(new FileInputStream("test.ser.gz"))))
    in.readObject()
    in.close()
  }
  test(2147483)
  test(2147484)
}

java.lang.ArrayIndexOutOfBoundsException: -3733
	at scala.collection.mutable.HashTable$class.addEntry(HashTable.scala:119)
	at scala.collection.mutable.HashMap.addEntry(HashMap.scala:43)
	at scala.collection.mutable.HashTable$class.init(HashTable.scala:81)
	at scala.collection.mutable.HashMap.init(HashMap.scala:43)
	at scala.collection.mutable.HashMap.readObject(HashMap.scala:130)

[scabug]

Imported From: https://issues.scala-lang.org/browse/SI-5632?orig=1
Reporter: @dlwh
Affected Versions: 2.9.0-1, 2.9.1, 2.10.0-M2
Other Milestones: 2.10.0-M2, 2.10.0

[scabug]

@dcsobral said:
Collections of size greater than Int.MaxValue are technically illegal, even if some of them work for some things.

[scabug]

@dlwh said (edited on Mar 30, 2012 8:25:10 PM UTC):
That's all well and good, but this is .1% of that value (almost exactly, so this is probably the problem!)

scala> 2147484 * 1.0 / Int.MaxValue
res5: Double = 0.0010000001643784345

scala> 

[scabug]

@dlwh said:
Bingo. sizeForThreshold multiplies by loadFactorDenum ( == 1000) without converting to a Long.

  // Hashtable.scala
  private[collection] final def defaultLoadFactor: Int = 750 // corresponds to 75%
  private[collection] final def loadFactorDenum = 1000;  

  // ...
  
  private[collection] final def newThreshold(_loadFactor: Int, size: Int) = ((size.toLong * _loadFactor) / loadFactorDenum).toInt
  
  private[collection] final def sizeForThreshold(_loadFactor: Int, thr: Int) = thr * loadFactorDenum / _loadFactor
  

[scabug]

@non said:
Good catch! Seems like an easy fix.

[scabug]

@dlwh said:
yup. Assuming I can make Scala build I'll make a patch and send a pull request real soon now.

[scabug]

@non said:
OK. Let me know if you need help with any of it.

I encourage you to write a test that catches this issue to help confirm that you fixed it and also prevent future regressions. Makes testing your patch easier on the commiter.

[scabug]

@dlwh said:
scala/scala#345