some minor fixes to MemoryStore This is a repeat of #5, moved to its own branch in my repo. This makes all updates to on ; it skips on synchronizing the reads where it can get away with it.
Lacking this, the if/case statements never trigger on Spark 0.8.0+.
Make "currentMemory" @volatile, so that it's reads in ensureFreeSpace() are atomic and up-to-date--i.e., currentMemory can't increase while putLock is held (though it could decrease, which would only help ensureFreeSpace()).
In MapOutputTrackerSuite, the "remote fetch" test sets spark.driver.port and spark.hostPort, assuming that they will be cleared by LocalSparkContext. However, the test never sets sc, so it remains null, causing LocalSparkContext to skip clearing these properties. Subsequent tests therefore fail with java.net.BindException: "Address already in use". This commit makes LocalSparkContext clear the properties even if sc is null.
Under unknown, but occasional, circumstances, reservation.groups is empty despite reservation.instances each having groups. This means that the spark_ec2 get_existing_clusters() method would fail to find any instances. To fix it, we simply use the instances' groups as the source of truth. Note that this is actually just a revival of PR #827, now that the issue has been reproduced.