Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RangeError in consumer group syncGroup response processing for member without assignment #550

Closed
ankon opened this issue Nov 7, 2019 · 1 comment · Fixed by #567
Closed

Comments

@ankon
Copy link
Contributor

ankon commented Nov 7, 2019

 Crash: KafkaJSNumberOfRetriesExceeded: Attempt to access memory outside buffer bounds ({"groupId":"XXXX","retryCount":0,"stack":"KafkaJSNumberOfRetriesExceeded
  Caused by: RangeError [ERR_BUFFER_OUT_OF_BOUNDS]: Attempt to access memory outside buffer bounds
    at boundsError (internal/buffer.js:70:11)
    at Buffer.readInt16BE (internal/buffer.js:492:5)
    at Decoder.readInt16 (/app/node_modules/kafkajs/src/protocol/decoder.js:36:31)
    at Object.decode (/app/node_modules/kafkajs/src/consumer/assignerProtocol.js:71:24)
    at ConsumerGroup.sync (/app/node_modules/kafkajs/src/consumer/consumerGroup.js:160:48)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (internal/process/task_queues.js:93:5)
    at /app/node_modules/kafkajs/src/consumer/runner.js:52:9
    at /app/node_modules/kafkajs/src/consumer/runner.js:354:11"})

From my logs it seems that at that point this member didn't receive an assignment, so this piece of code fails:

version: decoder.readInt16(),

I think there needs to be a check in any of these places, I'm just not sure which one:

  1. Check in ConsumerGroup.sync whether the memberAssignment is empty
  2. Check in MemberAssignment.decode whether the given buffer is empty
@ankon
Copy link
Contributor Author

ankon commented Nov 7, 2019

FWIW, The reason I'm getting this now is that I changed my assigner to no longer pre-fill the assignments:

sortedMembers.forEach(memberId => {
assignment[memberId] = {}
})

Nevon added a commit that referenced this issue Nov 28, 2019
Nevon added a commit that referenced this issue Nov 29, 2019
The test assigner only returns assignments for the first member of the
group, and not the second one. Without #567 the second consumer will
crash when trying to decode the missing assignment.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
1 participant