Delegate to specific coder when scanning for Strings, Array[Byte], etc #116

Merged
merged 1 commit into from Apr 3, 2017

Conversation

Projects
None yet
2 participants
@rayokota
Contributor

rayokota commented Mar 31, 2017

What changes were proposed in this pull request?

The calculation of the range for the scanner is not correct for types String, Array[Byte], Byte, UTF8String. The serialized representations should be obtained from the coder. In the case of PrimitiveType and Phoenix, the serializations happen to match the code in ScanRange.scala. However, I have a custom coder that does not have the same serialization.

How was this patch tested?

All existing unit tests still pass

@weiqingy weiqingy merged commit c0635a0 into hortonworks-spark:master Apr 3, 2017

weiqingy added a commit that referenced this pull request Apr 3, 2017

Delegate to specific coder when scanning for Strings, Array[Byte], etc (
#116)

## What changes were proposed in this pull request?

The calculation of the range for the scanner is not correct for types String, Array[Byte], Byte, UTF8String. The serialized representations should be obtained from the coder. In the case of PrimitiveType and Phoenix, the serializations happen to match the code in ScanRange.scala. However, I have a custom coder that does not have the same serialization.

## How was this patch tested?

All existing unit tests still pass

weiqingy added a commit that referenced this pull request Apr 3, 2017

Delegate to specific coder when scanning for Strings, Array[Byte], etc (
#116)

## What changes were proposed in this pull request?

The calculation of the range for the scanner is not correct for types String, Array[Byte], Byte, UTF8String. The serialized representations should be obtained from the coder. In the case of PrimitiveType and Phoenix, the serializations happen to match the code in ScanRange.scala. However, I have a custom coder that does not have the same serialization.

## How was this patch tested?

All existing unit tests still pass

weiqingy added a commit that referenced this pull request Apr 3, 2017

Delegate to specific coder when scanning for Strings, Array[Byte], etc (
#116)

## What changes were proposed in this pull request?

The calculation of the range for the scanner is not correct for types String, Array[Byte], Byte, UTF8String. The serialized representations should be obtained from the coder. In the case of PrimitiveType and Phoenix, the serializations happen to match the code in ScanRange.scala. However, I have a custom coder that does not have the same serialization.

## How was this patch tested?

All existing unit tests still pass
@weiqingy

This comment has been minimized.

Show comment
Hide comment
@weiqingy

weiqingy Apr 3, 2017

Contributor

Thanks @rayokota . Merged to master/branch-1.6/branch-2.0/branch-2.1.

Contributor

weiqingy commented Apr 3, 2017

Thanks @rayokota . Merged to master/branch-1.6/branch-2.0/branch-2.1.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment