Skip to content

Commit

Permalink
[SPARK-13227] Risky apply() in OpenHashMap
Browse files Browse the repository at this point in the history
https://issues.apache.org/jira/browse/SPARK-13227

It might confuse the future developers when they use OpenHashMap.apply() with a numeric value type.

null.asInstance[Int], null.asInstance[Long], null.asInstace[Float] and null.asInstance[Double] will return 0/0.0/0L, which might confuse the developer if the value set contains 0/0.0/0L with an existing key

The current patch only adds the comments describing the issue, with the respect to apply the minimum changes to the code base

The more direct, yet more aggressive, approach is use Option as the return type

andrewor14  JoshRosen  any thoughts about how to avoid the potential issue?

Author: CodingCat <zhunansjtu@gmail.com>

Closes #11107 from CodingCat/SPARK-13227.
  • Loading branch information
CodingCat authored and rxin committed Apr 19, 2016
1 parent 2b151b6 commit 4b3d129
Showing 1 changed file with 3 additions and 0 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,9 @@ import scala.reflect.ClassTag
* space overhead.
*
* Under the hood, it uses our OpenHashSet implementation.
*
* NOTE: when using numeric type as the value type, the user of this class should be careful to
* distinguish between the 0/0.0/0L and non-exist value
*/
private[spark]
class OpenHashMap[K : ClassTag, @specialized(Long, Int, Double) V: ClassTag](
Expand Down

0 comments on commit 4b3d129

Please sign in to comment.