Skip to content
Permalink
Browse files

[SPARK-23391][CORE] It may lead to overflow for some integer multipli…

…cation

## What changes were proposed in this pull request?
In the `getBlockData`,`blockId.reduceId` is the `Int` type, when it is greater than 2^28, `blockId.reduceId*8` will overflow
In the `decompress0`, `len` and  `unitSize` are  Int type, so `len * unitSize` may lead to  overflow
## How was this patch tested?
N/A

Author: liuxian <liu.xian3@zte.com.cn>

Closes #20581 from 10110346/overflow2.

(cherry picked from commit 4a4dd4f)
Signed-off-by: Sean Owen <sowen@cloudera.com>
  • Loading branch information...
10110346 authored and srowen committed Feb 12, 2018
1 parent 1e3118c commit d31c4ae7ba734356c849347b9a7b448da9a5a9a1
@@ -203,13 +203,13 @@ private[spark] class IndexShuffleBlockResolver(
// class of issue from re-occurring in the future which is why they are left here even though
// SPARK-22982 is fixed.
val channel = Files.newByteChannel(indexFile.toPath)
channel.position(blockId.reduceId * 8)
channel.position(blockId.reduceId * 8L)
val in = new DataInputStream(Channels.newInputStream(channel))
try {
val offset = in.readLong()
val nextOffset = in.readLong()
val actualPosition = channel.position()
val expectedPosition = blockId.reduceId * 8 + 16
val expectedPosition = blockId.reduceId * 8L + 16
if (actualPosition != expectedPosition) {
throw new Exception(s"SPARK-22982: Incorrect channel position after index file reads: " +
s"expected $expectedPosition but actual position was $actualPosition.")
@@ -116,7 +116,7 @@ private[columnar] case object PassThrough extends CompressionScheme {
while (pos < capacity) {
if (pos != nextNullIndex) {
val len = nextNullIndex - pos
assert(len * unitSize < Int.MaxValue)
assert(len * unitSize.toLong < Int.MaxValue)
putFunction(columnVector, pos, bufferPos, len)
bufferPos += len * unitSize
pos += len

0 comments on commit d31c4ae

Please sign in to comment.
You can’t perform that action at this time.