KafkaReceiver does not account for consumer group joining

[ddingel]

Description

Following up the discussions in this issue Kafka-produce and consume JSON message, Assert fails for 'consume' operation
KafkaReceiver currently does not track if the consumer group joining has finished.

The Problem Statement

If the consumer group join takes longer than maxNoOfRetryPollsOrTimeouts * pollingTime the consumer will not process any messages and report an empty record set. This situation also aften expects the pollingTime to be adjusted while running in different hardware configuations such as inside a docker or a standalone Jenkins or laptop etc, which is not a very good user/automation experience.

Solution

AC1 :
A fix could be to modify the receiving part, that the consumer tries to successfully join the consumer group and if that fails err out with an explanation.

AC2:
As explained here

• Do a first poll outside of the loop to trigger the consumer group rebalancing
• Ignore any runs if there is no assignment for that consumer
  • i.e. Do not increase maxNoOfRetryPollsOrTimeouts if there is no assignment
  • i.e. Increase the maxNoOfRetryPollsOrTimeouts only if there was an successful assignment

[authorjapps]

Available in 1.3.24. Yet to release.

• Re-Scheduled for Sunday: 6-Sep-2020, Sunday
• Attempted twice to release today: Error occurred:

[ERROR] The git-push command failed.
[ERROR] Command output:
[ERROR] fatal: Could not read from remote repository.
...
debug1: /etc/ssh/ssh_config line 48: Applying options for *
debug2: ssh_connect_direct: needpriv 0
debug1: Connecting to github.com port 22.
ssh: connect to host github.com port 22: Operation timed out

[authorjapps]

Released.
See Release Notes here.