[SPARK-8235][SPARK-8236][SQL] misc functions: sha1/sha, crc32 #6970
Conversation
|
Can one of the admins verify this patch? |
|
|
||
| override def dataType: DataType = StringType | ||
|
|
||
| override def expectedChildTypes: Seq[DataType] = Seq(BinaryType, StringType) |
There was a problem hiding this comment.
This is not right, should be Seq(BinaryType)
There was a problem hiding this comment.
@tarekauel Sorry, I was wrong, didn't figure out the difference between checkInputDataTypes and expectedChildTypes of ExpectsInputTypes.
In order to get better error message, we should use checkInputDataTypes.
expectedChildTypes is used to widen the types we support.
For this case, we could use Seq(BinaryType) as expectedChildTypes, which means it will also support StringType, because StringType could be casted to BinaryType (which is cheap).
In eval and genCode, you can expect that child.dataType is BinaryType.
|
LGTM |
|
Test build #959 has started for PR 6970 at commit |
|
Test build #959 has finished for PR 6970 at commit
|
| checksum.update(value.asInstanceOf[Array[Byte]], 0, value.asInstanceOf[Array[Byte]].length) | ||
| checksum.getValue | ||
| } | ||
| } |
There was a problem hiding this comment.
Could you add codegen support?
There was a problem hiding this comment.
NAVER - http://www.naver.com/
sujkh@naver.com 님께 보내신 메일 <Re: [spark] [SPARK-8235][SPARK-8236][SQL] misc functions: sha1/sha, crc32 (#6970)> 이 다음과 같은 이유로 전송 실패했습니다.
받는 사람이 회원님의 메일을 수신차단 하였습니다.
|
LGTM, Could you rebase it to master? |
|
@davies Rebase done! Please help to check it. |
|
@qiansl127 |
|
Yes, please close this PR. |
|
Forget to follow up this one, sorry. Next time, it will be better if you guys could comment on the JIRA to avoid crush each other, thanks you all the hard work, anyway! |
https://issues.apache.org/jira/browse/SPARK-8235
https://issues.apache.org/jira/browse/SPARK-8236