Real-time Query for Hadoop; mirror of Apache Impala
C++ Java Python JavaScript C Shell Other
Latest commit 68660ca Feb 17, 2017 @abehm abehm committed with jenkins IMPALA-4902: Copy parameters map in HdfsPartition.toThrift().
The bug: When generating the toThrift() of an HdfsTable,
each THdfsPartition used to contain a reference to its
partition's parameters map. As a result, one thread trying
to serialize a thrift table returned by toThrift() could
conflict with another thread updating the parameters maps of
the table partitions. Here are a few examples of operations
that may modify the parameters map:
COMPUTE [INCREMENTAL] STATS, DROP STATS,
ALTER TABLE SET TBLPROPERTIES, ALTER TABLE SET CACHED, etc.

The fix: Create a shallow copy of the parameters map in
HdfsPartition.toThrift(). This means that toThrift() itself
must be protected from concurrent modifications to the
parameters map. Callers of toThrift() are now required
to hold the table lock. One place where the lock was not
already held needed to be adjusted.

Testing:
- I was unable to reproduce the issue locally, but the stacks
  from the JIRAs point directly to the parameters map, and
  the races are pretty obvious from looking at the code.
- Passed a core/hdfs private run.

Change-Id: Ic11277ad5512d2431cd3cc791715917c95395ddf
Reviewed-on: http://gerrit.cloudera.org:8080/6127
Reviewed-by: Alex Behm <alex.behm@cloudera.com>
Tested-by: Impala Public Jenkins
Permalink
Failed to load latest commit information.
be IMPALA-2020, 4915, 4936: Add rounding for decimal casts Feb 24, 2017
bin IMPALA-4934: Disable Kudu OpenSSL initialization Feb 22, 2017
cmake_modules IMPALA-4652: Add crcutil to build Jan 12, 2017
common IMPALA-2328: Read support for min/max Parquet statistics Feb 23, 2017
docs [DOCS] Construct "fixed issues" topic for 2.8 release notes Feb 24, 2017
ext-data-source Add all build targets to CMake and speed up builds Dec 15, 2016
fe IMPALA-4902: Copy parameters map in HdfsPartition.toThrift(). Feb 24, 2017
infra IMPALA-4934: Disable Kudu OpenSSL initialization Feb 22, 2017
shell IMPALA-4570: shell tarball breaks with certain setuptools versions Dec 3, 2016
ssh_keys Move ssh keys from bin directory to fix packaging build break Jan 8, 2014
testdata IMPALA-2328: Read support for min/max Parquet statistics Feb 23, 2017
tests IMPALA-2328: Read support for min/max Parquet statistics Feb 23, 2017
thirdparty CDH-48469: Update thirdparty dependencies to cdh5.11.0-SNAPSHOT. Dec 21, 2016
www Three misc webpage changes Feb 23, 2017
.clang-format Match .clang-format more closely to actual practice. Oct 14, 2016
.clang-tidy IMPALA-3200: move bufferpool under runtime Nov 22, 2016
.gitignore IMPALA-4653: fix sticky config variable problem Jan 5, 2017
CMakeLists.txt IMPALA-4959: Avoid picking up the system's boost cmake module Feb 23, 2017
DISCLAIMER IMPALA-3808: Add incubating DISCLAIMER from the Incubator Branding Guide Sep 2, 2016
EXPORT_CONTROL.md IMPALA-4406: Add cryptography export control notice Nov 6, 2016
LICENSE.txt IMPALA-4230: ASF policy issues from 2.7.0 rc3. Oct 21, 2016
LOGS.md Consolidate test and cluster logs under a single directory. Mar 28, 2016
NOTICE.txt Update copyright year to 2017 Jan 29, 2017
README.md IMPALA-4512: Add a script that builds Impala on stock Ubuntu 14.04. Nov 29, 2016
buildall.sh IMPALA-4904,IMPALA-4914: add targeted-stress to exhaustive tests Feb 21, 2017

README.md

Welcome to Impala

Lightning-fast, distributed SQL queries for petabytes of data stored in Apache Hadoop clusters.

Impala is a modern, massively-distributed, massively-parallel, C++ query engine that lets you analyze, transform and combine data from a variety of data sources:

  • Best of breed performance and scalability.
  • Support for data stored in HDFS, Apache HBase and Amazon S3.
  • Wide analytic SQL support, including window functions and subqueries.
  • On-the-fly code generation using LLVM to generate CPU-efficient code tailored specifically to each individual query.
  • Support for the most commonly-used Hadoop file formats, including the Apache Parquet (incubating) project.
  • Apache-licensed, 100% open source.

More about Impala

To learn more about Impala as a business user, or to try Impala live or in a VM, please visit the Impala homepage.

If you are interested in contributing to Impala as a developer, or learning more about Impala's internals and architecture, visit the Impala wiki.

Supported Platforms

Impala only supports Linux at the moment.

Build Instructions

See bin/bootstrap_build.sh.

Export Control Notice

This distribution uses cryptographic software and may be subject to export controls. Please refer to EXPORT_CONTROL.md for more information.