Skip to content

Commit

Permalink
[HOTFIX] Fix select query on varchar column with large data fails wit…
Browse files Browse the repository at this point in the history
…h jvm crash

Problem : When select query fired on varchar column having large data it results in JVM crash because when when we increase the ReusableBuffer by 30% the new size gets gets reduced because requestSize * 30 gets out of range for int which gives a negative value and the total size gets reduced.

Solution : While assigning the size to ByteBuffer we first check if total size less than the requested size then we pass requested size to to the ByteBuffer.

This closes #3104
  • Loading branch information
shardul-cr7 authored and ravipesala committed Jan 28, 2019
1 parent d149732 commit c55f1d7
Show file tree
Hide file tree
Showing 2 changed files with 15 additions and 1 deletion.
Expand Up @@ -1386,6 +1386,13 @@ private CarbonCommonConstants() {
*/
public static final int CARBON_DYNAMIC_ALLOCATION_SCHEDULER_THREAD_SLEEP_TIME = 250;

/**
* We increment the requested page size by 30% only if the requested size is less than 10MB.
* Otherwise we take the original requested page size.This parameter will be used till the
* size based page implementation comes in carbon.
*/
public static final int REQUESTED_PAGE_SIZE_MAX = 10485760;

/**
* It allows queries on hive metastore directly along with filter information, otherwise first
* fetches all partitions from hive and apply filters on it.
Expand Down
Expand Up @@ -19,6 +19,7 @@

import org.apache.carbondata.common.annotations.InterfaceAudience;
import org.apache.carbondata.common.annotations.InterfaceStability;
import org.apache.carbondata.core.constants.CarbonCommonConstants;

/**
* class holds the reusable data buffer based on request it will resize.
Expand Down Expand Up @@ -47,7 +48,13 @@ public class ReusableDataBuffer {
*/
public byte[] getDataBuffer(int requestedSize) {
if (dataBuffer == null || requestedSize > size) {
this.size = requestedSize + ((requestedSize * 30) / 100);
// increase by 30% only if the requestedSize less than 10 MB
// otherwise take the original requestedSize.
if (requestedSize < CarbonCommonConstants.REQUESTED_PAGE_SIZE_MAX) {
this.size = requestedSize + ((requestedSize * 30) / 100);
} else {
this.size = requestedSize;
}
dataBuffer = new byte[size];
}
return dataBuffer;
Expand Down

0 comments on commit c55f1d7

Please sign in to comment.