KAFKA-4719: Consumption timeout should take into account producer request timeout

[hachikuji]

No description provided.

[hachikuji]

cc @ewencp

Single run here: https://jenkins.confluent.io/view/All/job/system-test-kafka-branch-builder-2/190/console. If successful, I'll repeat the test run a few more times.

[asfbot]

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk7-scala2.10/1402/
Test FAILed (JDK 7 and Scala 2.10).

[asfbot]

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk8-scala2.11/1408/
Test PASSed (JDK 8 and Scala 2.11).

[asfbot]

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk8-scala2.12/1405/
Test PASSed (JDK 8 and Scala 2.12).

[asfbot]

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk7-scala2.10/1405/
Test PASSed (JDK 7 and Scala 2.10).

[hachikuji]

@ewencp I ran the offset validation tests 5 times here: https://jenkins.confluent.io/view/All/job/system-test-kafka-branch-builder-2/191/console. There was no recurrence of the specific failure that this patch is targeting. The two other failures are difficult to understand and do not seem to fit with the event output. If the changes here seem reasonable, I suggest we merge this patch for the increased logging visibility, and I can continue investigating the other failures.

[hachikuji]

Oh, I almost forgot. One of the test failures seems to have been caused by this:

[INFO  - 2017-02-01 19:02:37,216 - background_thread - _protected_worker - lineno:39]: Traceback (most recent call last):
  File "/var/lib/jenkins/workspace/system-test-kafka-branch-builder-2/kafka/venv/local/lib/python2.7/site-packages/ducktape-0.6.0-py2.7.egg/ducktape/services/
background_thread.py", line 35, in _protected_worker
    self._worker(idx, node)
  File "/var/lib/jenkins/workspace/system-test-kafka-branch-builder-2/kafka/tests/kafkatest/services/verifiable_consumer.py", line 169, in _worker
    log_config = self.render('tools_log4j.properties', log_file=VerifiableConsumer.LOG_FILE)
  File "/var/lib/jenkins/workspace/system-test-kafka-branch-builder-2/kafka/venv/local/lib/python2.7/site-packages/ducktape-0.6.0-py2.7.egg/ducktape/template.py", line 90, in render
    template = self.template_env.get_template(path)
AttributeError: 'VerifiableConsumer' object has no attribute 'template_env'

[asfbot]

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk7-scala2.10/1421/
Test FAILed (JDK 7 and Scala 2.10).

[asfbot]

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk8-scala2.12/1421/
Test PASSed (JDK 8 and Scala 2.12).

[asfbot]

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk8-scala2.11/1424/
Test FAILed (JDK 8 and Scala 2.11).

[hachikuji]

5 more runs, this time all passing: https://jenkins.confluent.io/view/All/job/system-test-kafka-branch-builder-2/192/console.

[ewencp]

LGTM, merging to trunk and 0.10.2