[SPARK-8233][SQL] misc function: hash

[qiansl127]

https://issues.apache.org/jira/browse/SPARK-8233

[chenghao-intel]

Remove the toString as the case class will handle it properly.

[chenghao-intel]

LGTM except a minor issue.
@rxin can you trigger the unit test?

[davies]

In Hive, the Hash can take multiple columns: hash(a1[, a2...])

[chenghao-intel]

Oh, yes, @qiansl127 can you update this?

[qiansl127]

@davies @chenghao-intel Done!

[sujkh85]

NAVER - http://www.naver.com/

sujkh@naver.com 님께 보내신 메일 <Re: [spark] [SPARK-8233][SQL] misc function: hash (#6971)> 이 다음과 같은 이유로 전송 실패했습니다.

---

받는 사람이 회원님의 메일을 수신차단 하였습니다.

---

[qiansl127]

@davies Could you check this PR?

[chenghao-intel]

@davies, can you please trigger the unit test? I am working on the hashCode stulff right now, it can be considered as the improvement, probably this PR would better not to be blocked by the improvement.

[chenghao-intel]

And definitely I will update the code here after this PR being merged.

[rxin]

Jenkins, retest this please.

[qiansl127]

@rxin I've done the rebase, please trigger the test.

[SparkQA]

Test build #998 has finished for PR 6971 at commit d79088b.

• This patch fails Scala style tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
  • class Word2VecModel(JavaVectorTransformer, JavaSaveable, JavaLoader):
  • case class ScalaUDF(
  • case class CreateNamedStruct(children: Seq[Expression]) extends Expression
  • case class CurrentDate() extends LeafExpression
  • case class CurrentTimestamp() extends LeafExpression
  • case class Factorial(child: Expression) extends UnaryExpression with ExpectsInputTypes
  • case class Hex(child: Expression) extends UnaryExpression with ExpectsInputTypes
  • case class UnHex(child: Expression) extends UnaryExpression with ExpectsInputTypes
  • case class ShiftLeft(left: Expression, right: Expression)
  • case class ShiftRight(left: Expression, right: Expression)
  • case class ShiftRightUnsigned(left: Expression, right: Expression)
  • case class Md5(child: Expression) extends UnaryExpression with ExpectsInputTypes
  • case class Hash(children: Expression*) extends Expression
  • case class Sha1(child: Expression) extends UnaryExpression with ExpectsInputTypes
  • case class Crc32(child: Expression) extends UnaryExpression with ExpectsInputTypes
  • case class Not(child: Expression) extends UnaryExpression with Predicate with ExpectsInputTypes
  • trait StringRegexExpression extends ExpectsInputTypes
  • trait CaseConversionExpression extends ExpectsInputTypes
  • trait StringComparison extends ExpectsInputTypes
  • case class StringLength(child: Expression) extends UnaryExpression with ExpectsInputTypes
  • case class Levenshtein(left: Expression, right: Expression) extends BinaryExpression
  • case class Ascii(child: Expression) extends UnaryExpression with ExpectsInputTypes
  • case class Base64(child: Expression) extends UnaryExpression with ExpectsInputTypes
  • case class UnBase64(child: Expression) extends UnaryExpression with ExpectsInputTypes
  • case class Decode(bin: Expression, charset: Expression) extends Expression with ExpectsInputTypes
  • case class Encode(value: Expression, charset: Expression)
  • protected[sql] abstract class AtomicType extends DataType
  • abstract class NumericType extends AtomicType
  • abstract class DataType extends AbstractDataType
  • case class UserDefinedFunction protected[sql] (

[qiansl127]

@rxin @davies Could you trigger the test?

[rxin]

Jenkins, ok to test.

[chenghao-intel]

In general, I think it's ok to leave the hashCode for further improvement, for now, I think this PR looks good to me.
Can you guys trigger the unit test?

[SparkQA]

Test build #1013 has finished for PR 6971 at commit 837f6bf.

• This patch fails to build.
• This patch merges cleanly.
• This patch adds no public classes.

[rxin]

@qiansl127 can you close this pull request?

@JoshRosen will create an implementation based on your change. Thanks.

[JoshRosen]

The ordering of the map keys in foreach is not stable, so I think this may lead to unstable hashcodes for identical maps.

[qiansl127]

OK, sorry for no following these days.