Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Delegate to specific coder when scanning for Strings, Array[Byte], etc #116

Merged

Conversation

rayokota
Copy link
Contributor

What changes were proposed in this pull request?

The calculation of the range for the scanner is not correct for types String, Array[Byte], Byte, UTF8String. The serialized representations should be obtained from the coder. In the case of PrimitiveType and Phoenix, the serializations happen to match the code in ScanRange.scala. However, I have a custom coder that does not have the same serialization.

How was this patch tested?

All existing unit tests still pass

@weiqingy weiqingy merged commit c0635a0 into hortonworks-spark:master Apr 3, 2017
weiqingy pushed a commit that referenced this pull request Apr 3, 2017
#116)

## What changes were proposed in this pull request?

The calculation of the range for the scanner is not correct for types String, Array[Byte], Byte, UTF8String. The serialized representations should be obtained from the coder. In the case of PrimitiveType and Phoenix, the serializations happen to match the code in ScanRange.scala. However, I have a custom coder that does not have the same serialization.

## How was this patch tested?

All existing unit tests still pass
weiqingy pushed a commit that referenced this pull request Apr 3, 2017
#116)

## What changes were proposed in this pull request?

The calculation of the range for the scanner is not correct for types String, Array[Byte], Byte, UTF8String. The serialized representations should be obtained from the coder. In the case of PrimitiveType and Phoenix, the serializations happen to match the code in ScanRange.scala. However, I have a custom coder that does not have the same serialization.

## How was this patch tested?

All existing unit tests still pass
weiqingy pushed a commit that referenced this pull request Apr 3, 2017
#116)

## What changes were proposed in this pull request?

The calculation of the range for the scanner is not correct for types String, Array[Byte], Byte, UTF8String. The serialized representations should be obtained from the coder. In the case of PrimitiveType and Phoenix, the serializations happen to match the code in ScanRange.scala. However, I have a custom coder that does not have the same serialization.

## How was this patch tested?

All existing unit tests still pass
@weiqingy
Copy link
Contributor

weiqingy commented Apr 3, 2017

Thanks @rayokota . Merged to master/branch-1.6/branch-2.0/branch-2.1.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants