Permalink
Switch branches/tags
v1.2.2 v0.7.1refresh v0.7refresh v0.6refresh v0.5refresh v0.4refresh v0.3refresh v0.1refresh impala-v1.2.1 impala-v1.1.1 impala-v1.1 impala-v1.0 cdh5.15.0-release cdh5.14.4-release cdh5.14.2-release cdh5.14.0-release cdh5.13.3-release cdh5.13.2-release cdh5.13.1-release cdh5.13.0-release cdh5.12.2-release cdh5.12.1-release cdh5.12.0-release cdh5.11.2-release cdh5.11.1-release cdh5.11.0-release cdh5.10.2-release cdh5.10.1-release cdh5.10.0-release cdh5.9.3-release cdh5.9.2-release cdh5.9.1-release cdh5.9.0-release cdh5.8.5-release cdh5.8.4-release cdh5.8.3-release cdh5.8.2-release cdh5.8.0-release cdh5.7.6-release cdh5.7.5-release cdh5.7.4-release cdh5.7.3-release cdh5.7.2-release cdh5.7.1-release cdh5.7.0-release cdh5.6.1-release cdh5.6.0-release cdh5.5.6-release cdh5.5.5-release cdh5.5.4-release cdh5.5.2-release cdh5.5.1-release cdh5.5.0-release cdh5.4.11-release cdh5.4.10-release cdh5.4.9-release cdh5.4.8-release cdh5.4.7-release cdh5.4.5-release cdh5.4.4-release cdh5.4.3-release cdh5.4.2-release cdh5.4.1-release cdh5.4.0-release cdh5.3.10-release cdh5.3.9-release cdh5.3.8-release cdh5.3.6-release cdh5.3.5-release cdh5.3.4-release cdh5.3.3-release cdh5.3.2-release cdh5.3.1-release cdh5.3.0-release cdh5.2.6-release cdh5.2.5-release cdh5.2.4-release cdh5.2.3-release cdh5.2.2-release cdh5.2.1-release cdh5.2.0-release cdh5.1.5-release cdh5.1.4-release cdh5.1.3-release cdh5.1.2-release cdh5.1.0_v1.4 cdh5.1.0-release cdh5.0.6-release cdh5.0.5-release cdh5.0.3-release cdh5.0.2_v1.3.1 cdh5.0.2-release cdh5.0.1-release cdh4_v2.0 cdh4_v1.4 cdh4_v1.3.1
Nothing to show
Commits on Aug 8, 2018
Commits on Aug 2, 2018
  1. IMPALA-6998: test_bloom_wait_time fails due to late arrival of filter…

    smukil committed May 10, 2018
    …s on Isilon
    
    This test has been failing on Isilon runs, most likely due to timing issues
    which makes it a test issue rather than a product bug.
    
    This patch disables the test for Isilon. We should revisit what tests we run
    on non-HDFS filesystems later on, but until then, this should unblock
    the build.
    
    ==C5_APPROVED_BUGFIX==
    
    Change-Id: I2df6983a65a50b7efdd482124b70f518ee4c3229
    Reviewed-on: http://gerrit.cloudera.org:8080/10366
    Reviewed-by: Sailesh Mukil <sailesh@cloudera.com>
    Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
    Reviewed-on: http://gerrit.sjc.cloudera.com:8080/35637
    Reviewed-by: Fnu Sailesh Mukil <sailesh@cloudera.com>
    Tested-by: Fnu Sailesh Mukil <sailesh@cloudera.com>
Commits on Aug 1, 2018
  1. IMPALA-7008: Rewrite query to make it not return 100M rows

    lekv committed Jun 1, 2018
    One query in spilling.test is expected to fail. When it does not fail,
    it returned 100M rows, which would then cause the Python test code to
    consume memory until it gets OOM-killed by the kernel.
    
    To fix this, we rewrite the query. I tested this locally to make sure
    that the query still fails as expected on HDFS.
    
    ==C5_APPROVED_BUGFIX==
    
    Change-Id: I31956d3092a7e69b979f631df3a6dfda14ebe140
    Reviewed-on: http://gerrit.cloudera.org:8080/10597
    Reviewed-by: Tim Armstrong <tarmstrong@cloudera.com>
    Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
    Reviewed-on: http://gerrit.sjc.cloudera.com:8080/35624
    Tested-by: Lars Volker <lv@cloudera.com>
  2. IMPALA-7051: Serialize Maven invocations.

    Philip Zeyliger
    Philip Zeyliger committed May 18, 2018
    I've observed some rare cases where Impala fails to build. I believe
    it's because two Maven targets (yarn-extras and ext-data-source) are
    being executed simultaneously. Maven's handling of ~/.m2/repository,
    for example, is known to be not safe.
    
    This patch serializes the Maven builds with the following
    dependency graph:
      fe -> yarn-extras -> ext-data-source -> impala-parent
    The ordering of yarn-extras -> ext-data-source is arbitrary.
    
    I decided that this artificial dependency was the clearest
    way to prevent parallel executions. Having mvn-quiet.sh
    take a lock seemed considerably more complex.
    
    ==C5_APPROVED_BUGFIX==
    
    Change-Id: Ie24f34f421bc7dcf9140938464d43400da95275e
    Reviewed-on: http://gerrit.cloudera.org:8080/10460
    Reviewed-by: Tim Armstrong <tarmstrong@cloudera.com>
    Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
    Reviewed-on: http://gerrit.sjc.cloudera.com:8080/35223
    Reviewed-by: Philip Zeyliger <philip@cloudera.com>
    Tested-by: Philip Zeyliger <philip@cloudera.com>
  3. IMPALA-7145: fix leak of OpenSSL context when spilling

    timarmstrong authored and glglwty committed Jun 8, 2018
    Add a RAII wrapper for the OpenSSL context that automatically frees
    on all exit paths from the function.
    
    Add a backend test wrapper that enables LeakSanitizer for an individual
    test. This is a step towards IMPALA-2746.
    
    Fix version check bug in asan.h.
    
    Testing:
    Enable LeakSanitizer for openssl-util-test. This reliably found the bug.
    
    Ran core tests under ASAN.
    
    ==C5_APPROVED_BUGFIX==
    
    Change-Id: I98760ed8f31b18b489a156f945c29c95c9bf3184
    Reviewed-on: http://gerrit.cloudera.org:8080/10666
    Reviewed-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
    Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
    (cherry picked from commit 73c0786)
  4. IMPALA-7146: log session to query mapping

    timarmstrong authored and glglwty committed Jun 7, 2018
    Example output:
      I0607 11:12:25.542639 127653 impala-server.cc:962] Registered query query_id=894557dcebc637dd:743b75d100000000 session_id=c648cfc1ec86e3ae:7a071baf052f51ae
    
    Testing:
    Manually checked that both HS2 and Beeswax queries get the right log
    message.
    
    ==C5_APPROVED_BUGFIX==
    
    Change-Id: I6d66f41b829ab49b2e0670626bb79241dddff11a
    Reviewed-on: http://gerrit.cloudera.org:8080/10638
    Reviewed-by: Philip Zeyliger <philip@cloudera.com>
    Tested-by: Tim Armstrong <tarmstrong@cloudera.com>
    (cherry picked from commit 7c16905)
  5. IMPALA-6956: deflake and logging for query_expiration test

    vercegovac authored and glglwty committed Jun 4, 2018
    There was a recent flake where the number of in-flight queries
    that were executing differed from the expected number.
    The reason for the false negative is that one of the queries
    expired before the check for in-flight queries: it took too long
    to issue the queries.
    
    This change modifies the timeout for the expired query to not expire
    so quickly. Additional logging is added to the check for in-flight
    queries so that we can distinguish the case of too few queries vs.
    queries that have the wrong state.
    
    ==C5_APPROVED_BUGFIX==
    
    Change-Id: I01a8762d28ad920b9ec8a0b1b82469618c66768f
    Reviewed-on: http://gerrit.cloudera.org:8080/10602
    Reviewed-by: Tim Armstrong <tarmstrong@cloudera.com>
    Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
    Reviewed-on: http://gerrit.cloudera.org:8080/10621
    Tested-by: Tim Armstrong <tarmstrong@cloudera.com>
    (cherry picked from commit 50fa6e3)
  6. Guard S3 access check with variable

    lekv authored and glglwty committed Jun 5, 2018
    We've recently seen automated build failures where multiple inclusions
    of impala-config.sh led to the S3 access check being performed multiple
    times. This would trigger rate limiting on S3's side, which then caused
    the jobs to fail.
    
    This change adds a guard to only perform the check once, even when
    including impala-config.sh multiple times.
    
    To test this I included impala-config.sh multiple times on my local
    shell and observed that the check was only performed once.
    
    ==C5_APPROVED_BUGFIX==
    
    Change-Id: I1df16b2d037afafb7748f52b64732f680883f16c
    Reviewed-on: http://gerrit.cloudera.org:8080/10607
    Reviewed-by: Lars Volker <lv@cloudera.com>
    Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
    Reviewed-on: http://gerrit.cloudera.org:8080/10616
    Tested-by: Tim Armstrong <tarmstrong@cloudera.com>
    (cherry picked from commit 4a0a1e6)
  7. Collect hs_err_pid*.log files in logs/

    timarmstrong authored and glglwty committed Jun 4, 2018
    These JVM error dumps are often useful for debugging job failures. We
    should put them into the logs/ directory so they can be collected
    along with other logs.
    
    This was motivated by IMPALA-7069.
    
    Testing:
    Induced a segfault by changing the code, checked that the file ended up
    in the logs/ subdirectory.
    
    ==C5_APPROVED_BUGFIX==
    
    Change-Id: I753f6acb63cec600432675cfb6e5dd1e1427b5ae
    Reviewed-on: http://gerrit.cloudera.org:8080/10594
    Reviewed-by: Lars Volker <lv@cloudera.com>
    Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
    Reviewed-by: Tim Armstrong <tarmstrong@cloudera.com>
    (cherry picked from commit ac301f7)
  8. IMPALA-7104: test_bloom_wait_time failing with timeout on asan

    Thomas Tauber-Marshall authored and glglwty committed May 31, 2018
    test_bloom_wait_time checks that its query runs in less than a certain
    amount of time, to ensure we aren't waiting on bloom filters that will
    never arrive.
    
    The test sometimes goes slightly over the timeout and fails on ASAN.
    This patch uses a specific_build_type_timeout to extend the allowed
    timeout for ASAN.
    
    It also fixes a bug in a similar test where we were comparing a
    duration in seconds to a timeout in milliseconds, and renames some
    variables to make it clearer why this was a bug.
    
    ==C5_APPROVED_BUGFIX==
    
    Change-Id: I0271eb83e65b3b5f17e32f6501032f0d12dde38e
    Reviewed-on: http://gerrit.cloudera.org:8080/10565
    Reviewed-by: Michael Brown <mikeb@cloudera.com>
    Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
    (cherry picked from commit 278e0e4)
  9. IMPALA-6990: TestClientSsl.test_tls_v12 failing due to Python SSL error

    smukil authored and glglwty committed May 29, 2018
    When we upgraded to thrift-0.9.3, the TSSLSocket.py logic changed quite
    a bit. Our RHEL7 machines come equipped with Python 2.7.5. Looking at
    these comments, that means that we'll be unable to create a 'SSLContext'
    but be able to explicitly specify ciphers:
    https://github.com/apache/thrift/blob/88591e32e710a0524327153c8b629d5b461e35e0/lib/py/src/transport/TSSLSocket.py#L37-L41
        # SSLContext is not available for Python < 2.7.9
        _has_ssl_context = sys.hexversion >= 0x020709F0
    
        # ciphers argument is not available for Python < 2.7.0
        _has_ciphers = sys.hexversion >= 0x020700F0
    
    If we cannot create a 'SSLContext', then we cannot use TLSv1.2 and have
    to use TLSv1:
    https://github.com/apache/thrift/blob/88591e32e710a0524327153c8b629d5b461e35e0/lib/py/src/transport/TSSLSocket.py#L48-L49
        # For python >= 2.7.9, use latest TLS that both client and server
        # supports.
        # SSL 2.0 and 3.0 are disabled via ssl.OP_NO_SSLv2 and ssl.OP_NO_SSLv3.
        # For python < 2.7.9, use TLS 1.0 since TLSv1_X nor OP_NO_SSLvX is
        # unavailable.
        _default_protocol = ssl.PROTOCOL_SSLv23 if _has_ssl_context else \
            ssl.PROTOCOL_TLSv1
    
    Our custom cluster test forces the server to use TLSv1.2 and also forces
    a specific cipher:
    https://github.com/apache/impala/blob/2f22a6f67ff363a0832a7ceee5d0020c8fd9b15a/tests/custom_cluster/test_client_ssl.py#L118-L119
    
    So this combination of configuration values causes a failure in RHEL7
    because we only allow a specific cipher which works with TLSv1.2, but
    the client cannot use TLSv1.2 due to the Python version as mentioned above.
    
    We've not noticed these failures on older-than-RHEL7-systems since the
    OpenSSL versions on those systems don't support TLSv1.2. (< OpenSSL 1.0.1)
    
    To fix this, we need to change the Python version on RHEL 7 to be
    >= Python 2.7.9. This patch skips the test if an older version of
    Python than 2.7.9 is detected.
    
    ==C5_APPROVED_BUGFIX==
    
    Change-Id: I92c66ecaeb94b0c83ee6f1396c082709c21b3187
    Reviewed-on: http://gerrit.cloudera.org:8080/10529
    Reviewed-by: Sailesh Mukil <sailesh@cloudera.com>
    Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
    (cherry picked from commit 4193e6d)
  10. IMPALA-7090: Limit the size of expr created by EqualityDisjunctsToInRule

    glglwty committed May 26, 2018
    Currently EqualityDisjunctsToInRule introduced in IMPALA-5280 might
    create an expr with unlimited number of children and fails a query,
    which should be avoided. The easy solution is to not apply the
    rewrite when the number of children is large.
    
    ==C5_APPROVED_BUGFIX==
    
    Change-Id: Ie40c3210271a9e3c7f1b2b869a8c2ec8bacaa72a
    Reviewed-on: http://gerrit.cloudera.org:8080/10528
    Reviewed-by: Tianyi Wang <twang@cloudera.com>
    Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
    (cherry picked from commit 2c363f6)
  11. IMPALA-6338: Disable more flaky bloom filter tests

    Thomas Tauber-Marshall authored and glglwty committed May 29, 2018
    Until IMPALA-6338 is fixed, temporarily disable tests that are
    affected by it - any test that has a 'limit' and relies on the
    contents of the runtime profile. This patch disables the runtime
    profile check for all such tests in bloom_filter.test
    
    ==C5_APPROVED_BUGFIX==
    
    Change-Id: Ifc9da892efa3b27d63056ad8e3befac82808ffdb
    Reviewed-on: http://gerrit.cloudera.org:8080/10530
    Reviewed-by: Bikramjeet Vig <bikramjeet.vig@cloudera.com>
    Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
    (cherry picked from commit 7156101)
  12. IMPALA-6933: Avoids db name collisions for Kudu tests

    vercegovac authored and glglwty committed May 25, 2018
    Kudu tests generate temporary db names in a way so that its
    unlikely, yet possible to collide. A recent test failure
    indicates such a collision came up. The fix changes the
    way that the name is generated so that it includes the
    classes name for which the db name is generated. This db name
    will make it easier to see which test created it and the name
    will not collide with other names generated by other tests.
    
    Testing:
    - ran the updated test locally
    
    ==C5_APPROVED_BUGFIX==
    
    Change-Id: I7c2f8a35fec90ae0dabe80237d83954668b47f6e
    Reviewed-on: http://gerrit.cloudera.org:8080/10513
    Reviewed-by: Michael Brown <mikeb@cloudera.com>
    Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
    (cherry picked from commit e4d950a)
  13. IMPALA-6813: Hedged reads metrics broken when scanning non-HDFS based…

    smukil authored and glglwty committed Apr 9, 2018
    … table
    
    We realized that the libHDFS API call hdfsGetHedgedReadMetrics() crashes
    when the 'fs' argument passed to it is not a HDFS filesystem.
    
    There is an open bug for it on the HDFS side: HDFS-13417
    However, it looks like we won't be getting a fix for it in the short term,
    so our only option at this point is to skip it.
    
    Testing: Made sure that enabling preads and scanning from S3 doesn't
    cause a crash.
    Also, added a custom cluster test to exercise the pread code path. We
    are unable to verify hedged reads in a minicluster, but we can at least
    exercise the code path to make sure that nothing breaks.
    
    ==C5_APPROVED_BUGFIX==
    
    Change-Id: I48fe80dfd9a1ed68a8f2b7038e5f42b5a3df3baa
    Reviewed-on: http://gerrit.cloudera.org:8080/9966
    Reviewed-by: Sailesh Mukil <sailesh@cloudera.com>
    Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
    (cherry picked from commit 9c83fc8)
  14. IMPALA-6317: Add -cmake_only option to buildall.sh

    dknupp authored and glglwty committed Mar 6, 2018
    It's sometimes useful to be able to build a complete Impala dev
    environment without necessarily building the Impala binary itself
    -- e.g., when one wants to use the internal test framework to run
    tests against an instance of Impala running on a remote cluster.
    
    - This patch adds a -cmake_only flag to buildall.sh, which then
      gets propagated to the make_impala.sh.
    
    - Added a missing line to the help text re: passing the -ninja
      command line option
    
    ==C5_APPROVED_BUGFIX==
    
    Change-Id: If31a4e29425a6a20059cba2f43b72e4fb908018f
    Reviewed-on: http://gerrit.cloudera.org:8080/10455
    Reviewed-by: David Knupp <dknupp@cloudera.com>
    Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
    (cherry picked from commit 4c84050)
  15. Remove python dependency binaries from cdh5.

    dknupp authored and glglwty committed May 23, 2018
    The dependencies should be pip downloaded according to the
    requirements files. In some cases, the binaries in the repo
    didn't even match the version that was specified in the
    file.
    
    ==C5_APPROVED_BUGFIX==
    
    Change-Id: I3092ba9c35fda86666270aca2e65b13a678ebc98
    Reviewed-on: http://gerrit.sjc.cloudera.com:8080/33206
    Reviewed-by: Lars Volker <lv@cloudera.com>
    Tested-by: David Knupp <dknupp@cloudera.com>
    (cherry picked from commit 4b3c302)
  16. IMPALA-7035: Configure jceks.key.serialFilter for KMS.

    Philip Zeyliger authored and glglwty committed May 15, 2018
    Configures a Java property for KMS to account for JDK 8u171's security fixes. I
    was seeing impala-py.test tests/metadata/test_hdfs_encryption.py fail with the
    following error:
    
      AssertionError: Error creating encryption zone: RemoteException: Can't recover key for testkey1 from keystore file:/home/impdev/Impala/testdata/cluster/cdh6/node-1/data/kms.keystore
    
    The issue is described in HDFS-13494, and I imagine it'll be fixed in due time. In the
    meanwhile, setting this property seems to do the trick.
    
    ==C5_APPROVED_BUGFIX==
    
    Change-Id: I2d21c9cce3b91e8fd8b2b4f1cda75e3958c977d5
    Reviewed-on: http://gerrit.cloudera.org:8080/10418
    Reviewed-by: Joe McDonnell <joemcdonnell@cloudera.com>
    Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
    Reviewed-on: http://gerrit.cloudera.org:8080/10446
    (cherry picked from commit a0edf8e)
  17. IMPALA-7017: deflake/fix test_catalog_restart test

    vercegovac authored and glglwty committed May 14, 2018
    The custom_cluster/test_metadata_replicas.py:test_catalog_restart
    test has been recently flaky/broken for two reasons:
    
    1) Variable support for Hive and non-hdfs filesystems. Other tests that
    depend on Hive have disabled tests for non-hdfs filesystems. Since the
    functionality tested is not intended for all filesystems, this change
    disables this test for all filesystems other than hdfs.
    
    2) Several builds have been flaky when looking up catalogd's version.
    This change adds a retry for obtaining the version.
    
    ==C5_APPROVED_BUGFIX==
    
    Change-Id: Iab6edb01f0bd7f5408cfef28fd05fdc95fb78469
    Reviewed-on: http://gerrit.cloudera.org:8080/10397
    Reviewed-by: Joe McDonnell <joemcdonnell@cloudera.com>
    Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
    (cherry picked from commit 698161f)
  18. IMPALA-4464: Remove /bin/remote_data_load.py

    dknupp authored and glglwty committed May 14, 2018
    This file was started before the ASF project was set up, and
    committed as-is. However, it relies on some internal resources
    not generally available to the external Apache community at large,
    and so serves no purpose in that context.
    
    ==C5_APPROVED_BUGFIX==
    
    Change-Id: I002efae6ad538d371680ce23099277708ed67e0e
    Reviewed-on: http://gerrit.cloudera.org:8080/10388
    Reviewed-by: Philip Zeyliger <philip@cloudera.com>
    Tested-by: David Knupp <dknupp@cloudera.com>
    (cherry picked from commit 4cea23f)
  19. IMPALA-6907: Close stale connections to removed cluster members

    michaelhkw authored and glglwty committed May 2, 2018
    Previously, ImpalaServer::MembershipCallback() is used by each
    Impala backend node to update cluster membership. It also removes
    stale connections to nodes which are no longer members of the cluster.
    However, the way it detects removed member is flawed as it relies
    on query_locations_ to determine whether stale connections may
    exist to the removed members. query_locations_ is a map of host
    name to a set of queries running on that host. A entry for a remote
    node only exists in query_locations_ if an Impalad node has acted
    as coordinator of a query with fragment instances scheduled to run
    on that remote node.
    
    This change fixes this problem by closing connections to remote
    hosts which are removed from the cluster regardless of whether
    it can be found in query_locations_. A new test is added to
    exercise this path by restarting Impalad backend nodes between
    queries. Also change impala_cluster.py to use bin/start-impala.sh
    to start Impala demon instead of directly forking and exec'ing
    Impalad. This is needed as start-impala.sh sets up the proper
    Java related environment variables.
    
    ==C5_APPROVED_BUGFIX==
    
    Change-Id: I41b7297cf665bf291b09b23524d19b1d10ab281d
    Reviewed-on: http://gerrit.cloudera.org:8080/10327
    Reviewed-by: Michael Ho <kwho@cloudera.com>
    Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
    (cherry picked from commit 36b82ea)
  20. IMPALA-6993: Don't log stack trace in Coordinator::BackendState::Exec()

    dhecht authored and glglwty committed May 8, 2018
    We should use Status::Expected() here, just like it's used
    above. The stack trace isn't interesting and the error is expected
    once we get down this path. (I'm not sure why we don't just use
    exec_status though, but presumably the prefix was added for a
    reason).
    
    ==C5_APPROVED_BUGFIX==
    
    Change-Id: I9cd015ce0a5af51de73d591f07c01a80cda942e0
    Reviewed-on: http://gerrit.cloudera.org:8080/10348
    Reviewed-by: Tim Armstrong <tarmstrong@cloudera.com>
    Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
    (cherry picked from commit 17e1b58)
  21. IMPALA-6975: TestRuntimeRowFilters.test_row_filters failing with Memo…

    smukil authored and glglwty committed May 4, 2018
    …ry limit exceeded
    
    This test has started failing relatively frequently. We think that
    this may be due to timing differences of when RPCs arrive from the
    recent changes with KRPC.
    
    Increasing the memory limit should allow this test to pass
    consistently.
    
    ==C5_APPROVED_BUGFIX==
    
    Change-Id: Ie39482e2a0aee402ce156b11cce51038cff5e61a
    Reviewed-on: http://gerrit.cloudera.org:8080/10315
    Reviewed-by: Tim Armstrong <tarmstrong@cloudera.com>
    Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
    (cherry picked from commit fab08fb)
  22. IMPALA-6931: reduces races in query expiration tests

    vercegovac authored and glglwty committed May 1, 2018
    Recent tests ran into flakiness when testing query expiration.
    This change makes two changes:
    1) query state is retrieved earlier; a flaky test skipped
    the expected state.
    2) bump the timing; a flaky test had queries expire before
    it could check them
    
    ==C5_APPROVED_BUGFIX==
    
    Change-Id: I93f4ec450fc7e5a685c135b444e90d37e632831d
    Reviewed-on: http://gerrit.cloudera.org:8080/10279
    Reviewed-by: Dan Hecht <dhecht@cloudera.com>
    Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
    (cherry picked from commit bd413c2)
  23. Update version to 2.13.0-SNAPSHOT

    smukil authored and glglwty committed Apr 30, 2018
    ==C5_APPROVED_BUGFIX==
    
    Change-Id: Iac447c57a273e0c334e0e63d6e740c2a00d5c762
    Reviewed-on: http://gerrit.cloudera.org:8080/10241
    Reviewed-by: Lars Volker <lv@cloudera.com>
    Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
    (cherry picked from commit d06de71)
  24. IMPALA-6885: handle empty s3 dirs in recover_partitions test

    vercegovac authored and glglwty committed Apr 24, 2018
    Two tests (test_duplicate_partitions, test_support_all_types)
    added only directories, which is a no-op on s3. This results
    in s3 test failures.
    
    Another test (test_recover_partitions) falsely passed for s3
    since its test for a malformed partition name is ambiguous
    (did it pass because recovering correctly excluded or because
    the directory does not exist, as in the case of s3?).
    
    A recent change fixed the asserts in this test, so these tests
    likely were falsely reporting success.
    
    This change bundles directory creation with adding to it a new
    file, which is the common case in this test. As a result, the
    peculiarity with s3 is not handled via a special case each time
    the filesystem is modified for new partitions.
    
    This change also adds an explicit test for empty partition
    directories, since the first change leaves this case untested.
    
    Testing:
    - tested against a local minicluster and s3
    
    ==C5_APPROVED_BUGFIX==
    
    Change-Id: Ic6c74f4f2466b5870322bdf373e8a6a4d1d679f4
    Reviewed-on: http://gerrit.cloudera.org:8080/10210
    Reviewed-by: Alex Behm <alex.behm@cloudera.com>
    Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
    (cherry picked from commit 0238ebc)
  25. IMPALA-6889: Fix race around terminating processes

    lekv authored and glglwty committed Apr 23, 2018
    A process with empty cmdline can trip up the Process class in our test
    code. When a process dies it becomes a zombie until it gets reaped by
    its parent.
    
    From 'man proc':
    
    /proc/[pid]/cmdline
      This read-only file holds the complete command line for the
      process, unless the process is a zombie.  In the latter case,
      there is nothing in this file: that is, a read on this file
      will return 0 characters.  The command-line arguments appear
      in this file as a set of strings separated by null bytes
      ('\0'), with a further null byte after the last string.
    
    To fix this, we take a copy of the cmdline before passing it to the
    Process ctor to prevent it from changing.
    
    I couldn't come up with a test for this.
    
    ==C5_APPROVED_BUGFIX==
    
    Change-Id: Iecf630e0b71d91469650636e81f940a7bec07113
    Reviewed-on: http://gerrit.cloudera.org:8080/10156
    Reviewed-by: Michael Brown <mikeb@cloudera.com>
    Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
    (cherry picked from commit c2255fd)
  26. IMPALA-6887: Typo in authz-policy.ini.template

    fredyw authored and glglwty committed Apr 19, 2018
    Before: alter_functionl_text_lzo
    After: alter_functional_text_lzo
    
    This patch also adds missing test cases for ALTER privilege on
    functional_text_lzo database.
    
    Testing:
    - Ran all front-end tests
    
    ==C5_APPROVED_BUGFIX==
    
    Change-Id: I6aea8d71dda39838e9e70160018ce2c5fc73df21
    Reviewed-on: http://gerrit.cloudera.org:8080/10113
    Reviewed-by: Bharath Vissapragada <bharathv@cloudera.com>
    Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
    (cherry picked from commit cfe2c43)
  27. IMPALA-6804: Allow SELECT and INSERT privileges on SERVER

    holleyism authored and glglwty committed Apr 10, 2018
    This change add the last two privileges at the SERVER level.
    
    GRANT SELECT ON SERVER TO ROLE test_role;
    REVOKE SELECT ON SERVER FROM ROLE test_role;
    GRANT INSERT ON SERVER TO ROLE test_role
    REVOKE INSERT ON SERVER FROM ROLE test_role
    
    Testing:
    Added tests to cover server level select and insert.
    Ran all front-end tests.
    
    ==C5_APPROVED_BUGFIX==
    
    Change-Id: If6510b4d54c9902029a357f1aebb2570cd75855d
    Reviewed-on: http://gerrit.cloudera.org:8080/9972
    Reviewed-by: Alex Behm <alex.behm@cloudera.com>
    Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
    (cherry picked from commit 855dc46)
  28. IMPALA-6650: Add fine-grained DROP privilege

    fredyw authored and glglwty committed Apr 3, 2018
    Add support for executing DROP statements by granting DROP privilege.
    
    These are the new GRANT/REVOKE statements introduced at server,
    database, and table scopes.
    
    GRANT DROP on SERVER svr TO ROLE testrole;
    GRANT DROP on DATABASE db TO ROLE testrole;
    GRANT DROP on TABLE tbl TO ROLE testrole;
    
    REVOKE DROP on SERVER svr FROM ROLE testrole;
    REVOKE DROP on DATABASE db FROM ROLE testrole;
    REVOKE DROP on TABLE tbl FROM ROLE testrole;
    
    Testing:
    - Added new front-end tests
    - Ran front-end tests
    
    ==C5_APPROVED_BUGFIX==
    
    Change-Id: Iea1dfb0b117cb1725e9656b9a79a9aebee82b5e4
    Reviewed-on: http://gerrit.cloudera.org:8080/9911
    Reviewed-by: Alex Behm <alex.behm@cloudera.com>
    Tested-by: Impala Public Jenkins
    (cherry picked from commit a7cfdb7)
  29. IMPALA-6800: Add test cases for statements that require ALTER privilege

    fredyw authored and glglwty committed Apr 3, 2018
    Testing:
    - Added new test cases
    - Ran front-end tests
    
    ==C5_APPROVED_BUGFIX==
    
    Change-Id: Ie9fc1249420771ee857259c01acc3f9e7c9ed383
    Reviewed-on: http://gerrit.cloudera.org:8080/9905
    Reviewed-by: Alex Behm <alex.behm@cloudera.com>
    Tested-by: Impala Public Jenkins
    (cherry picked from commit 99f001f)
  30. IMPALA-6649: Add fine-grained ALTER privilege

    holleyism authored and glglwty committed Mar 26, 2018
    Updated support and analysis files to provide ALTER privilege.
    
    Example statements:
    GRANT ALTER on SERVER svr TO ROLE testrole;
    GRANT ALTER on DATABASE db TO ROLE testrole;
    GRANT ALTER on TABLE tbl TO ROLE testrole;
    
    REVOKE ALTER on SERVER svr FROM ROLE testrole;
    REVOKE ALTER on DATABASE db FROM ROLE testrole;
    REVOKE ALTER on TABLE tbl FROM ROLE testrole;
    
    ALTER TABLE ... RENAME requires TABLE level privileges and
    CREATE at the DATABASE level.
    
    Tests:
    Added ALTER tests to cover scope of existing ALTER tests but in
    the context of only having ALTER privilege.
    Ran all fe tests.
    
    ==C5_APPROVED_BUGFIX==
    
    Change-Id: I0b25d10a8634829fbe90e308dfc7efc8182fef2d
    Reviewed-on: http://gerrit.cloudera.org:8080/9805
    Reviewed-by: Alex Behm <alex.behm@cloudera.com>
    Tested-by: Impala Public Jenkins
    (cherry picked from commit d852e63)
  31. IMPALA-6647: Add CREATE fine-grained privilege

    fredyw authored and glglwty committed Mar 20, 2018
    This patch allows executing CREATE statements by granting CREATE
    privilege.
    
    These are the new GRANT/REVOKE statements introduced at server and
    database scopes.
    
    GRANT CREATE on SERVER svr TO ROLE testrole;
    GRANT CREATE on DATABASE db TO ROLE testrole;
    
    REVOKE CREATE on SERVER svr FROM ROLE testrole;
    REVOKE CREATE on DATABASE db FROM ROLE testrole;
    
    Testing:
    - Ran front-end tests
    
    ==C5_APPROVED_BUGFIX==
    
    Change-Id: Id540e78fc9201fc1b4e6cac9b81ea54b8ae9eecd
    Reviewed-on: http://gerrit.cloudera.org:8080/9738
    Reviewed-by: Alex Behm <alex.behm@cloudera.com>
    Tested-by: Impala Public Jenkins
    (cherry picked from commit 2502ff9)
  32. IMPALA-6643: Add REFRESH fine-grained privilege

    fredyw authored and glglwty committed Mar 12, 2018
    Before this patch, ALL privilege was required to execute INVALIDATE
    METADATA and having any privilege allowed executing REFRESH <table>
    and INVALIDATE METADATA <table>. With this patch, REFRESH privilege
    is now required to execute INVALIDATE METADATA or REFRESH statement.
    
    These are the new GRANT/REVOKE statements introduced at server,
    database, and table scopes.
    
    GRANT REFRESH on SERVER svr TO ROLE testrole;
    GRANT REFRESH on DATABASE db TO ROLE testrole;
    GRANT REFRESH on TABLE db.tbl TO ROLE testrole;
    
    REVOKE REFRESH on SERVER svr FROM ROLE testrole;
    REVOKE REFRESH on DATABASE db FROM ROLE testrole;
    REVOKE REFRESH on TABLE db.tbl FROM ROLE testrole;
    
    Testing:
    - Ran front-end tests
    
    ==C5_APPROVED_BUGFIX==
    
    Change-Id: I4c3c5a51fe493d39fd719c7a388d4d5760049ce4
    Reviewed-on: http://gerrit.cloudera.org:8080/9589
    Reviewed-by: Alex Behm <alex.behm@cloudera.com>
    Tested-by: Impala Public Jenkins
    (cherry picked from commit 1d66417)
  33. IMPALA-6902: Tighten udf test to look for crashes

    vercegovac authored and glglwty committed Apr 24, 2018
    test_native_functions_race assumes that any exception
    is grounds for failing the test. However, the flake described
    in IMPALA-6902 shows an example of an exception that causes
    a false negative (it should be ignored). This change tightens
    the test to fail when an impalad crashes, which was the original
    intent of the test.
    
    Tests:
    - re-ran test_udfs.py
    
    ==C5_APPROVED_BUGFIX==
    
    Change-Id: I7780ff328fde5cf462cf718a1467b8a032d2c17b
    Reviewed-on: http://gerrit.cloudera.org:8080/10180
    Reviewed-by: Thomas Tauber-Marshall <tmarshall@cloudera.com>
    Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
    (cherry picked from commit e2f9a8f)