UserError: Out of bounds when consuming messages

[xxzefgh]

Follow-up to #110

I copied message schema and partitions count from our prod instance where error happens consistently, not sure if that has anything to do with the bug though.

Repo: https://github.com/mbrunjadze-crocobet/platformatic-kafka-out-of-bounds-bug

Error:

node:internal/modules/run_main:123
    triggerUncaughtException(
    ^

UserError: Out of bounds.
    at DynamicBuffer.slice (file:///platformatic-kafka-out-of-bounds-bug/node_modules/@platformatic/kafka/dist/protocol/dynamic-buffer.js:96:19)
    at Reader.readVarIntBytes (file:///platformatic-kafka-out-of-bounds-bug/node_modules/@platformatic/kafka/dist/protocol/reader.js:215:35)
    at readRecord (file:///platformatic-kafka-out-of-bounds-bug/node_modules/@platformatic/kafka/dist/protocol/records.js:57:23)
    at readRecordsBatch (file:///platformatic-kafka-out-of-bounds-bug/node_modules/@platformatic/kafka/dist/protocol/records.js:149:28)
    at file:///platformatic-kafka-out-of-bounds-bug/node_modules/@platformatic/kafka/dist/apis/consumer/fetch-v17.js:111:52
    at Reader.readNullableArray (file:///platformatic-kafka-out-of-bounds-bug/node_modules/@platformatic/kafka/dist/protocol/reader.js:236:24)
    at Reader.readArray (file:///platformatic-kafka-out-of-bounds-bug/node_modules/@platformatic/kafka/dist/protocol/reader.js:269:21)
    at file:///platformatic-kafka-out-of-bounds-bug/node_modules/@platformatic/kafka/dist/apis/consumer/fetch-v17.js:87:31
    at Reader.readNullableArray (file:///platformatic-kafka-out-of-bounds-bug/node_modules/@platformatic/kafka/dist/protocol/reader.js:236:24)
    at Reader.readArray (file:///platformatic-kafka-out-of-bounds-bug/node_modules/@platformatic/kafka/dist/protocol/reader.js:269:21) {
  code: 'PLT_KFK_USER',
  canRetry: false
}

[ShogunPanda]

@xxzefgh Where you using compression? In that case #123 might fix it.

[xxzefgh]

@ShogunPanda We don't use compression, and unfortunately 1.15.1 didn't fix it.
One thing I should note is that error doesn't happen when we use mode: latest (about 1000 msgs/sec). but with mode: earliest or custom offset, let's say from 1 hour before, it fails before we get the 1st message.

[simone-sanfratello]

Thanks for the code @xxzefgh, it has been really helpful to detect the error!