Skip to content

Commit fe54ebd

Browse files
jbappleImpala Public Jenkins
authored andcommitted
IMPALA-5031: widen Thrift enum to placate UBSAN
This fixes an instance of undefined behavior in the end-to-end tests in which an enum value is outside of the allowable values for that enum according to the C++14 standard. Representative backtrace: exec/parquet/parquet-metadata-utils.cc:293:26: runtime error: load of value 49, which is not a valid value for type 'Type::type' #0 ParquetMetadataUtils::ValidateRowGroupColumn( parquet::FileMetaData const&, char const*, int, int, parquet::SchemaElement const&, RuntimeState*) exec/parquet/parquet-metadata-utils.cc:293:26 #1 BaseScalarColumnReader::Reset(HdfsFileDesc const&, parquet::ColumnChunk const&, int) exec/parquet/parquet-column-readers.cc:1077:43 #2 HdfsParquetScanner::InitScalarColumns() exec/parquet/hdfs-parquet-scanner.cc:1679:60 #3 HdfsParquetScanner::NextRowGroup() exec/parquet/hdfs-parquet-scanner.cc:648:45 #4 HdfsParquetScanner::GetNextInternal(RowBatch*) exec/parquet/hdfs-parquet-scanner.cc:437:45 #5 HdfsParquetScanner::ProcessSplit() exec/parquet/hdfs-parquet-scanner.cc:353:21 #6 HdfsScanNode::ProcessSplit(vector<FilterContext> const&, MemPool*, io::ScanRange*, long*) exec/hdfs-scan-node.cc:514:21 #7 HdfsScanNode::ScannerThread(bool, long) exec/hdfs-scan-node.cc:415:7 #8 HdfsScanNode::ThreadTokenAvailableCb(ThreadResourcePool*):: $_0::operator()() const exec/hdfs-scan-node.cc:337:13 Change-Id: I48090e8e0c6c6f18bb1ad3c32c1f5fbffc908844 Reviewed-on: http://gerrit.cloudera.org:8080/13940 Reviewed-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com> Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
1 parent 521b152 commit fe54ebd

1 file changed

Lines changed: 22 additions & 0 deletions

File tree

common/thrift/parquet.thrift

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,28 @@ enum Type {
3838
DOUBLE = 5;
3939
BYTE_ARRAY = 6;
4040
FIXED_LEN_BYTE_ARRAY = 7;
41+
42+
/**
43+
* UBSAN_FORCE_WIDTH forces the values of the C++ enum Type to include (1u << 31) - 1.
44+
* That prevents the undefined behavior in the [expr] and [dcl.enum] sections of the
45+
* C++14 standard:
46+
*
47+
* If during the evaluation of an expression, the result is not mathematically
48+
* defined or not in the range of representable values for its type, the behavior is
49+
* undefined.
50+
*
51+
* and
52+
*
53+
* [F]or an enumeration where emin is the smallest enumerator and emax is the
54+
* largest, the values of the enumeration are the values in the range bmin to bmax,
55+
* defined as follows: Let K be 1 for a two's complement representation and 0 for a
56+
* one's complement or sign-magnitude representation. bmax is the smallest value
57+
* greater than or equal to max(|emin| - K, |emax|) and equal to 2^M-1, where M is
58+
* a non-negative integer. bmin is zero if emin is non-negative and -(bmax+K)
59+
* otherwise.
60+
*/
61+
62+
UBSAN_FORCE_WIDTH = 0x7ffffff;
4163
}
4264

4365
/**

0 commit comments

Comments
 (0)