Permalink
Commits on Apr 25, 2017
  1. KAFKA-4840; BufferPool errors can cause buffer pool to go into a bad …

    …state
    
    Author: Sean McCauliff <smccauliff@linkedin.com>
    
    Reviewers: Dong Lin <lindong28@gmail.com>, Ismael Juma <ismael@juma.me.uk>, Jiangjie Qin <becket.qin@gmail.com>
    
    Closes #2659 from smccauliff/kafka-4840
    smccauliff committed with becketqin Apr 25, 2017
Commits on Apr 24, 2017
  1. MINOR: Replication system tests should cover compressed path

    Author: Jason Gustafson <jason@confluent.io>
    
    Reviewers: Apurva Mehta <apurva@confluent.io>, Ismael Juma <ismael@juma.me.uk>
    
    Closes #2745 from hachikuji/add-replication-testcase-for-compression
    hachikuji committed with ijuma Apr 24, 2017
Commits on Apr 23, 2017
  1. MINOR: ProducerPerformance should work with older client jars

    Author: Jun Rao <junrao@gmail.com>
    
    Reviewers: Ismael Juma <ismael@juma.me.uk>
    
    Closes #2896 from junrao/minor
    junrao committed with ijuma Apr 23, 2017
  2. KAFKA-4564; Follow-up to fix test_timeout_on_pre_010_brokers system t…

    …est failure
    
    Author: Matthias J. Sax <matthias@confluent.io>
    
    Reviewers: Eno Thereska <eno@confluent.io>, Ismael Juma <ismael@juma.me.uk>
    
    Closes #2897 from mjsax/KAFKA-4564-follow-up
    mjsax committed with ijuma Apr 23, 2017
Commits on Apr 22, 2017
  1. MINOR: Move `Os` class to utils package and rename it to OperatingSystem

    The `common` package is public and this class is
    internal.
    
    Author: Ismael Juma <ismael@juma.me.uk>
    
    Reviewers: Jason Gustafson <jason@confluent.io>
    
    Closes #2759 from ijuma/move-os-to-utils
    ijuma committed with hachikuji Apr 22, 2017
Commits on Apr 21, 2017
  1. MINOR: fix javadoc comment of Transformer

    Here's where it's called:
    https://github.com/apache/kafka/blob/trunk/streams/src/main/java/org/apache/kafka/streams/kstream/internals/KStreamTransform.java#L64-L67
    
    Author: mihbor <mbor81@gmail.com>
    
    Reviewers: Matthias J. Sax, Damian Guy, Guozhang Wang
    
    Closes #2884 from mihbor/patch-3
    mihbor committed with guozhangwang Apr 21, 2017
  2. KAFKA-5090: Fix Kafka Streams SessionStore.findSessions javadoc broken

    Author: mihbor <mbor81@gmail.com>
    
    Reviewers: Matthias J. Sax, Guozhang Wang
    
    Closes #2874 from mihbor/patch-1
    mihbor committed with guozhangwang Apr 21, 2017
  3. MINOR: improve test cleanup for Streams

     - call close() on Metrics to join created threads
    
    Author: Matthias J. Sax <matthias@confluent.io>
    
    Reviewers: Eno Thereska, Damian Guy, Guozhang Wang
    
    Closes #2788 from mjsax/minor-improve-test-metric-cleanup
    mjsax committed with guozhangwang Apr 21, 2017
  4. KAFKA-5100; ProducerPerformanceService failing due to parsing error

    Author: Jun Rao <junrao@gmail.com>
    
    Reviewers: Ismael Juma <ismael@juma.me.uk>
    
    Closes #2890 from junrao/kafka-5100
    junrao committed Apr 21, 2017
  5. KAFKA-4564: Add system test for pre-0.10 brokers

    Author: Matthias J. Sax <matthias@confluent.io>
    
    Reviewers: Ismael Juma, Eno Thereska, Matthias J. Sax, Guozhang Wang
    
    Closes #2837 from mjsax/kafka-4564-fail-fast-test-stream-compatibility
    mjsax committed with guozhangwang Apr 21, 2017
  6. MINOR: fix record collector to stick with streams partitioner behavio…

    …r if it is specified
    
    If `partition==null` and `partitioner!=null` we should not fall back to default partitioner (as we do before the patch if `producer.partitionsFor(...)` returns empty list. Falling back to default partitioner might corrupt hash partitioning.
    
    Author: Matthias J. Sax <matthias@confluent.io>
    
    Reviewers: Eno Thereska, Damian Guy, Guozhang Wang
    
    Closes #2868 from mjsax/minor-fix-RecordCollector
    mjsax committed with guozhangwang Apr 21, 2017
  7. MINOR: Remove redundant CRC validation for non-compressed records in …

    …older message formats
    
    Author: Jason Gustafson <jason@confluent.io>
    
    Reviewers: Ismael Juma <ismael@juma.me.uk>
    
    Closes #2881 from hachikuji/fix-redundant-crc-check
    hachikuji committed Apr 21, 2017
  8. MINOR: Fix logging in ControlRecordType

    - typo error corrected (spelling)
    
    Author: Kamal C <kamal.chandraprakash@gmail.com>
    
    Reviewers: Ismael Juma <ismael@juma.me.uk>
    
    Closes #2885 from Kamal15/log
    Kamal15 committed with ijuma Apr 21, 2017
  9. MINOR: findbugs should generate XML reports in Jenkins

    Author: Colin P. Mccabe <cmccabe@confluent.io>
    
    Reviewers: Ismael Juma <ismael@juma.me.uk>
    
    Closes #2842 from cmccabe/findbugs-xml
    cmccabe committed with ijuma Apr 21, 2017
  10. KAFKA-5094; Replace SCRAM credentials in broker logs with tag hidden

    Author: Rajini Sivaram <rajinisivaram@googlemail.com>
    
    Reviewers: Ismael Juma <ismael@juma.me.uk>
    
    Closes #2879 from rajinisivaram/KAFKA-5094
    rajinisivaram committed with ijuma Apr 21, 2017
  11. KAFKA-5088: some spelling error in code comment

    fix some spelling errors
    
    Author: xinlihua <xin.lihua1@zte.com.cn>
    
    Reviewers: Matthias J. Sax, Guozhang Wang
    
    Closes #2871 from auroraxlh/fix_spellingerror
    auroraxlh committed with guozhangwang Apr 21, 2017
Commits on Apr 20, 2017
  1. KAFKA-5095: Adjust accepted overhead for ThreadCacheTest

    Author: Eno Thereska <eno@confluent.io>
    
    Reviewers: Damian Guy, Guozhang Wang
    
    Closes #2877 from enothereska/KAFKA-5095-cacheOverheads
    enothereska committed with guozhangwang Apr 20, 2017
  2. KAFKA-5073: Kafka Streams stuck rebalancing after exception thrown in…

    … rebalance listener
    
    Author: Matthias J. Sax <matthias@confluent.io>
    
    Reviewers: Ismael Juma, Eno Thereska, Guozhang Wang
    
    Closes #2856 from mjsax/kafka-5073
    mjsax committed with guozhangwang Apr 20, 2017
  3. KAFKA-5047: NullPointerException while using GlobalKTable in KafkaStr…

    …eams
    
    Skip null keys when initializing GlobalKTables. This is inline with what happens during normal processing.
    
    Author: Damian Guy <damian.guy@gmail.com>
    
    Reviewers: Michael G. Noll, Eno Thereska, Matthias J. Sax, Guozhang Wang
    
    Closes #2834 from dguy/kafka-5047
    dguy committed with guozhangwang Apr 20, 2017
  4. MINOR: reduce commit interval and cache size for integration test

    Author: Matthias J. Sax <matthias@confluent.io>
    
    Reviewers: Damian Guy, Eno Thereska, Guozhang Wang
    
    Closes #2789 from mjsax/minor-improve-integration-test
    mjsax committed with guozhangwang Apr 20, 2017
  5. KAFKA-5014; NetworkClient.leastLoadedNode should check if channel is …

    …ready
    
    Author: Ismael Juma <ismael@juma.me.uk>
    
    Reviewers: Jason Gustafson <jason@confluent.io>
    
    Closes #2813 from ijuma/kafka-5014-least-loaded-node-should-check-if-channel-is-ready
    ijuma committed with hachikuji Apr 20, 2017
  6. MINOR: improve test stability for Streams broker-compatibility test

    Author: Matthias J. Sax <matthias@confluent.io>
    
    Reviewers: Magnus Edenhill, Eno Thereska, Damian Guy, Guozhang Wang
    
    Closes #2836 from mjsax/minor-broker-comp-test
    mjsax committed with guozhangwang Apr 20, 2017
  7. KAFKA-4937: Batch offset fetches in the Consumer

    change `consumer.position` so that it always updates any partitions that need an update. Keep track of partitions that `seekToBeginning` in `StoreChangeLogReader` and do the `consumer.position` call after all `seekToBeginning` calls.
    
    Author: Damian Guy <damian.guy@gmail.com>
    
    Reviewers: Guozhang Wang, Jason Gustafson, Ismael Juma
    
    Closes #2769 from dguy/kafka-4937
    dguy committed with guozhangwang Apr 20, 2017
  8. MINOR: Improvements to PID snapshot management

    Author: Jason Gustafson <jason@confluent.io>
    
    Reviewers: Ismael Juma <ismael@juma.me.uk>, Jun Rao <junrao@gmail.com>
    
    Closes #2866 from hachikuji/improve-snapshot-management
    hachikuji committed with junrao Apr 20, 2017
  9. MINOR: Fix open file leak in log cleaner integration tests

    Author: Jason Gustafson <jason@confluent.io>
    
    Reviewers: Ismael Juma <ismael@juma.me.uk>
    
    Closes #2870 from hachikuji/fix-log-cleaner-test-leak
    hachikuji committed with ijuma Apr 20, 2017
  10. KAFKA-4814; Enable ZK ACLs only when zookeeper.set.acl is set

    Author: Rajini Sivaram <rajinisivaram@googlemail.com>
    
    Reviewers: Ismael Juma <ismael@juma.me.uk>
    
    Closes #2845 from rajinisivaram/KAFKA-4814
    rajinisivaram committed with ijuma Apr 20, 2017
  11. MINOR: Update dependencies for 0.11

    Worth special mention:
    
    1. Update Scala to 2.11.11 and 2.12.2
    2. Update Gradle to 3.5
    3. Update ZooKeeper to 3.4.10
    4. Update reflections to 0.9.11, which:
        * Switches to jsr305 annotations with a provided scope
        * Updates Guava from 18 to 20
        * Updates javaassist from 3.18 to 3.21
    
    There’s a separate PR for updating RocksDb, so
    I didn’t include that here.
    
    Author: Ismael Juma <ismael@juma.me.uk>
    
    Reviewers: Jason Gustafson <jason@confluent.io>
    
    Closes #2872 from ijuma/update-deps-for-0.11
    ijuma committed Apr 20, 2017
Commits on Apr 19, 2017
  1. KAFKA-5068; Optionally print out metrics after running the perf tests

    junrao added a config `--print.metrics` to control whether ProducerPerformance prints out metrics at the end of the test. If its okay, will add the code counterpart for consumer.
    
    Author: huxi <huxi@zhenrongbao.com>
    
    Reviewers: Jun Rao <junrao@gmail.com>
    
    Closes #2860 from amethystic/kafka-5068_print_metrics_in_perf_tests
    amethystic committed with junrao Apr 19, 2017
  2. MINOR: Fix some re-raising of exceptions in system tests

    Author: Ewen Cheslack-Postava <me@ewencp.org>
    
    Reviewers: Ismael Juma <ismael@juma.me.uk>
    
    Closes #2852 from ewencp/minor-re-raise-exceptions
    ewencp committed with ijuma Apr 19, 2017
  3. KAFKA-5049; Chroot check should be done for each ZkUtils instance

    Author: anukin <anukin2611@gmail.com>
    
    Reviewers: Ismael Juma <ismael@juma.me.uk>
    
    Closes #2857 from anukin/KAFKA_5049_zkroot_check
    anukin committed with ijuma Apr 19, 2017
Commits on Apr 18, 2017
  1. KAFKA-5069; add controller integration tests

    Test the various controller protocols by observing zookeeper and broker state.
    
    Author: Onur Karaman <okaraman@linkedin.com>
    
    Reviewers: Ismael Juma <ismael@juma.me.uk>, Jun Rao <junrao@gmail.com>
    
    Closes #2853 from onurkaraman/KAFKA-5069
    onurkaraman committed with junrao Apr 18, 2017
  2. KAFKA-5036; Second part: Points 2 -> 5): Refactor caching of Latest E…

    …poch
    
    This PR covers point (2) and point (5) from KAFKA-5036:
    
    **Commit 1:**
    2. Currently, we update the leader epoch in epochCache after log append in the follower but before log append in the leader. It would be more consistent to always do this after log append. This also avoids issues related to failure in log append.
    5. The constructor of LeaderEpochFileCache has the following:
    lock synchronized { ListBuffer(checkpoint.read(): _*) }
    But everywhere else uses a read or write lock. We should use consistent locking.
    This is a refactor to the way epochs are cached, replacing the code to cache the latest epoch in the LeaderEpochFileCache by reusing the cached value in Partition. There is no functional change.
    
    **Commit 2:**
    Adds an assert(epoch >=0) as epochs are written. Refactors tests so they never hit this assert.
    
    Author: Ben Stopford <benstopford@gmail.com>
    
    Reviewers: Jun Rao <junrao@gmail.com>
    
    Closes #2831 from benstopford/KAFKA-5036-part2-second-try
    benstopford committed with junrao Apr 18, 2017
Commits on Apr 17, 2017
  1. KAFKA-5075; Defer exception to the next pollOnce() if consumer's fetc…

    …h position has already increased
    
    Author: Dong Lin <lindong28@gmail.com>
    Author: Dong Lin <lindong28@users.noreply.github.com>
    
    Reviewers: Jiangjie Qin <becket.qin@gmail.com>
    
    Closes #2859 from lindong28/KAFKA-5075
    lindong28 committed with becketqin Apr 17, 2017
Commits on Apr 14, 2017
  1. KAFKA-4986; Producer per StreamTask support (KIP-129)

    Enable producer per task if exactly-once config is enabled.
    
    Author: Matthias J. Sax <matthias@confluent.io>
    
    Reviewers: Eno Thereska <eno@confluent.io>, Damian Guy <damian.guy@gmail.com>, Ismael Juma <ismael@juma.me.uk>
    
    Closes #2773 from mjsax/exactly-once-streams-producer-per-task
    mjsax committed with ijuma Apr 14, 2017
Commits on Apr 12, 2017
  1. KAFKA-5038; Throw correct exception of locking of state directory fails

    Author: Eno Thereska <eno.thereska@gmail.com>
    
    Reviewers: Damian Guy <damian.guy@gmail.com>, Matthias J. Sax <matthias@confluent.io>, Ismael Juma <ismael@juma.me.uk>
    
    Closes #2848 from enothereska/KAFKA-5038-trunk
    enothereska committed with ijuma Apr 12, 2017