Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

panic: runtime error: slice bounds out of range (1.2.0) #10

Closed
whatsupbros opened this issue Mar 6, 2023 · 2 comments · Fixed by #13
Closed

panic: runtime error: slice bounds out of range (1.2.0) #10

whatsupbros opened this issue Mar 6, 2023 · 2 comments · Fixed by #13
Assignees
@EladLeev
Labels
bug Something isn't working

Comments

@whatsupbros
Copy link

whatsupbros commented Mar 6, 2023 •
edited
Loading

Follow-up for #7

When attempt to connect to the cluster and consume a topic data to analyze, getting this fatal error all the time in specific environments (in one of the environments the same topic can be consumed without technical issues by the tool, but there is fewer than 100 messages in the topic in that environment).

Expected Behavior

The topic data is consumed without issues, and schema IDs stats provided as the result.

Current Behavior

Getting this exception on any attempt to use the application:

$ ./schema-registry-statistics --bootstrap "<broker host>:<broker port>" --topic "<topic name>" --group "schema-stats-client" --tls --cert /path/to/ca.cer --user "<user>" --password "<password>" --store --path "./output.json" --oldest --verbose
[sr-stats] 2023/03/06 10:25:44 Starting to consume from <topic name>
[sr-stats DEBUG] 2023/03/06 10:25:44 Initializing new client
[sr-stats DEBUG] 2023/03/06 10:25:44 ClientID is the default of 'sarama', you should consider setting it to something application-specific.
[sr-stats DEBUG] 2023/03/06 10:25:44 ClientID is the default of 'sarama', you should consider setting it to something application-specific.
[sr-stats DEBUG] 2023/03/06 10:25:44 client/metadata fetching metadata for all topics from broker <broker host>:<broker port>
[sr-stats DEBUG] 2023/03/06 10:25:44 Completed pre-auth SASL handshake. Available mechanisms: [PLAIN]
[sr-stats DEBUG] 2023/03/06 10:25:44 SASL authentication successful with broker <broker host>:<broker port>:4 - [0 0 0 0]
[sr-stats DEBUG] 2023/03/06 10:25:44 Connected to broker at <broker host>:<broker port> (unregistered)
[sr-stats DEBUG] 2023/03/06 10:25:44 client/brokers registered new broker #1 at <broker host>:<broker port>
[sr-stats DEBUG] 2023/03/06 10:25:44 Successfully initialized new client
[sr-stats DEBUG] 2023/03/06 10:25:44 client/metadata fetching metadata for [<topic name>] from broker <broker host>:<broker port>
[sr-stats DEBUG] 2023/03/06 10:25:44 client/coordinator requesting coordinator for schema-stats-client from <broker host>:<broker port>
[sr-stats DEBUG] 2023/03/06 10:25:44 client/coordinator coordinator for schema-stats-client is #1 (<broker host>:<broker port>)
[sr-stats DEBUG] 2023/03/06 10:25:44 ClientID is the default of 'sarama', you should consider setting it to something application-specific.
[sr-stats DEBUG] 2023/03/06 10:25:44 Completed pre-auth SASL handshake. Available mechanisms: [PLAIN]
[sr-stats DEBUG] 2023/03/06 10:25:44 SASL authentication successful with broker <broker host>:<broker port>:4 - [0 0 0 0]
[sr-stats DEBUG] 2023/03/06 10:25:44 Connected to broker at <broker host>:<broker port> (registered as #1)
[sr-stats DEBUG] 2023/03/06 10:25:47 client/coordinator requesting coordinator for schema-stats-client from <broker host>:<broker port>
[sr-stats DEBUG] 2023/03/06 10:25:47 client/coordinator coordinator for schema-stats-client is #1 (<broker host>:<broker port>)
[sr-stats] 2023/03/06 10:25:47 Consumer up and running!...
[sr-stats] 2023/03/06 10:25:47 Use SIGINT to stop consuming.
[sr-stats DEBUG] 2023/03/06 10:25:48 consumer/broker/1 accumulated 12 new subscriptions
[sr-stats DEBUG] 2023/03/06 10:25:48 consumer/broker/1 added subscription to <topic name>/2
[sr-stats DEBUG] 2023/03/06 10:25:48 consumer/broker/1 added subscription to <topic name>/1
[sr-stats DEBUG] 2023/03/06 10:25:48 consumer/broker/1 added subscription to <topic name>/4
[sr-stats DEBUG] 2023/03/06 10:25:48 consumer/broker/1 added subscription to <topic name>/5
[sr-stats DEBUG] 2023/03/06 10:25:48 consumer/broker/1 added subscription to <topic name>/3
[sr-stats DEBUG] 2023/03/06 10:25:48 consumer/broker/1 added subscription to <topic name>/6
[sr-stats DEBUG] 2023/03/06 10:25:48 consumer/broker/1 added subscription to <topic name>/7
[sr-stats DEBUG] 2023/03/06 10:25:48 consumer/broker/1 added subscription to <topic name>/10
[sr-stats DEBUG] 2023/03/06 10:25:48 consumer/broker/1 added subscription to <topic name>/8
[sr-stats DEBUG] 2023/03/06 10:25:48 consumer/broker/1 added subscription to <topic name>/11
[sr-stats DEBUG] 2023/03/06 10:25:48 consumer/broker/1 added subscription to <topic name>/9
[sr-stats DEBUG] 2023/03/06 10:25:48 consumer/broker/1 added subscription to <topic name>/0
panic: runtime error: slice bounds out of range [:5] with capacity 0

goroutine 7 [running]:
main.(*Consumer).ConsumeClaim(0xc000104c30, {0x824380, 0xc0001e2120}, {0x822cc8, 0xc0005887e0})
        /home/runner/work/schema-registry-statistics/schema-registry-statistics/main.go:111 +0x275
github.com/Shopify/sarama.(*consumerGroupSession).consume(0xc0001e2120, {0xc00018c1e0, 0x2b}, 0x0)
        /home/runner/go/pkg/mod/github.com/!shopify/sarama@v1.37.2/consumer_group.go:868 +0x26e
github.com/Shopify/sarama.newConsumerGroupSession.func2({0xc00018c1e0?, 0x0?}, 0x0?)
        /home/runner/go/pkg/mod/github.com/!shopify/sarama@v1.37.2/consumer_group.go:793 +0x76
created by github.com/Shopify/sarama.newConsumerGroupSession
        /home/runner/go/pkg/mod/github.com/!shopify/sarama@v1.37.2/consumer_group.go:785 +0x4b4

Context (Environment)

schema-registry-statistics version: 1.2.0

Distribution: Confluent Platform Enterprise
Confluent Platform version: 7.2.2
Apache Kafka version: 3.2.2

SASL_PLAIN with TLS/SSL endpoint is used to connect to the Broker.
With other consumers I can read the topic data without issues.
There are 12 partitions of the topic.
There are 6 versions of schemas registered for the topic value subject in Schema Registry.
@whatsupbros
Copy link
Author

In another envronment, the issue appears after consumption of some number of records:

$ ./schema-registry-statistics --bootstrap "<broker host>:<broker port>" --topic "<topic name>" --group "schema-stats-client-2" --tls --cert /path/to/ca.cer --user "<user>" --password "<password>" --store --path "./output.json" --oldest --verbose
[sr-stats] 2023/03/06 18:05:45 Starting to consume from <topic name>
[sr-stats DEBUG] 2023/03/06 18:05:45 Initializing new client
[sr-stats DEBUG] 2023/03/06 18:05:45 ClientID is the default of 'sarama', you should consider setting it to something application-specific.
[sr-stats DEBUG] 2023/03/06 18:05:45 ClientID is the default of 'sarama', you should consider setting it to something application-specific.
[sr-stats DEBUG] 2023/03/06 18:05:45 client/metadata fetching metadata for all topics from broker <broker host>:<broker port>
[sr-stats DEBUG] 2023/03/06 18:05:45 Completed pre-auth SASL handshake. Available mechanisms: [PLAIN]
[sr-stats DEBUG] 2023/03/06 18:05:45 SASL authentication successful with broker <broker host>:<broker port>:4 - [0 0 0 0]
[sr-stats DEBUG] 2023/03/06 18:05:45 Connected to broker at <broker host>:<broker port> (unregistered)
[sr-stats DEBUG] 2023/03/06 18:05:45 client/brokers registered new broker #1 at <broker host>:<broker port>
[sr-stats DEBUG] 2023/03/06 18:05:45 Successfully initialized new client
[sr-stats DEBUG] 2023/03/06 18:05:45 client/metadata fetching metadata for [<topic name>] from broker <broker host>:<broker port>
[sr-stats DEBUG] 2023/03/06 18:05:45 client/coordinator requesting coordinator for schema-stats-client-2 from <broker host>:<broker port>
[sr-stats DEBUG] 2023/03/06 18:05:45 client/coordinator coordinator for schema-stats-client-2 is #1 (<broker host>:<broker port>)
[sr-stats DEBUG] 2023/03/06 18:05:45 ClientID is the default of 'sarama', you should consider setting it to something application-specific.
[sr-stats DEBUG] 2023/03/06 18:05:45 Completed pre-auth SASL handshake. Available mechanisms: [PLAIN]
[sr-stats DEBUG] 2023/03/06 18:05:45 SASL authentication successful with broker <broker host>:<broker port>:4 - [0 0 0 0]
[sr-stats DEBUG] 2023/03/06 18:05:45 Connected to broker at <broker host>:<broker port> (registered as #1)
[sr-stats DEBUG] 2023/03/06 18:05:48 client/coordinator requesting coordinator for schema-stats-client-2 from <broker host>:<broker port>
[sr-stats DEBUG] 2023/03/06 18:05:48 client/coordinator coordinator for schema-stats-client-2 is #1 (<broker host>:<broker port>)
[sr-stats] 2023/03/06 18:05:48 Consumer up and running!...
[sr-stats] 2023/03/06 18:05:48 Use SIGINT to stop consuming.
[sr-stats DEBUG] 2023/03/06 18:05:48 consumer/broker/1 accumulated 9 new subscriptions
[sr-stats DEBUG] 2023/03/06 18:05:48 consumer/broker/1 added subscription to <topic name>/6
[sr-stats DEBUG] 2023/03/06 18:05:48 consumer/broker/1 added subscription to <topic name>/7
[sr-stats DEBUG] 2023/03/06 18:05:48 consumer/broker/1 added subscription to <topic name>/11
[sr-stats DEBUG] 2023/03/06 18:05:48 consumer/broker/1 added subscription to <topic name>/1
[sr-stats DEBUG] 2023/03/06 18:05:48 consumer/broker/1 added subscription to <topic name>/4
[sr-stats DEBUG] 2023/03/06 18:05:48 consumer/broker/1 added subscription to <topic name>/0
[sr-stats DEBUG] 2023/03/06 18:05:48 consumer/broker/1 added subscription to <topic name>/10
[sr-stats DEBUG] 2023/03/06 18:05:48 consumer/broker/1 added subscription to <topic name>/3
[sr-stats DEBUG] 2023/03/06 18:05:48 consumer/broker/1 added subscription to <topic name>/2
[sr-stats DEBUG] 2023/03/06 18:05:48 consumer/broker/1 accumulated 3 new subscriptions
[sr-stats] 2023/03/06 18:05:50 acked 100 messages
[sr-stats] 2023/03/06 18:05:50 acked 100 messages
[sr-stats] 2023/03/06 18:05:50 acked 100 messages
[sr-stats] 2023/03/06 18:05:50 acked 100 messages
[sr-stats] 2023/03/06 18:05:50 acked 100 messages
[sr-stats] 2023/03/06 18:05:50 acked 100 messages
[sr-stats] 2023/03/06 18:05:50 acked 100 messages
[sr-stats] 2023/03/06 18:05:50 acked 100 messages
[sr-stats] 2023/03/06 18:05:50 acked 100 messages
[sr-stats] 2023/03/06 18:05:50 acked 100 messages
[sr-stats] 2023/03/06 18:05:50 acked 100 messages
[sr-stats] 2023/03/06 18:05:50 acked 100 messages
[sr-stats] 2023/03/06 18:05:50 acked 100 messages
[sr-stats] 2023/03/06 18:05:50 acked 100 messages
[sr-stats] 2023/03/06 18:05:50 acked 100 messages
[sr-stats] 2023/03/06 18:05:50 acked 100 messages
[sr-stats] 2023/03/06 18:05:50 acked 100 messages
[sr-stats] 2023/03/06 18:05:50 acked 100 messages
[sr-stats] 2023/03/06 18:05:50 acked 100 messages
[sr-stats] 2023/03/06 18:05:50 acked 100 messages
[sr-stats] 2023/03/06 18:05:50 acked 100 messages
[sr-stats] 2023/03/06 18:05:50 acked 100 messages
[sr-stats] 2023/03/06 18:05:50 acked 100 messages
[sr-stats] 2023/03/06 18:05:50 acked 100 messages
[sr-stats] 2023/03/06 18:05:50 acked 100 messages
[sr-stats] 2023/03/06 18:05:50 acked 100 messages
[sr-stats] 2023/03/06 18:05:50 acked 100 messages
[sr-stats] 2023/03/06 18:05:50 acked 100 messages
[sr-stats] 2023/03/06 18:05:50 acked 100 messages
[sr-stats] 2023/03/06 18:05:50 acked 100 messages
[sr-stats DEBUG] 2023/03/06 18:05:50 consumer/broker/1 added subscription to <topic name>/5
[sr-stats] 2023/03/06 18:05:50 acked 100 messages
[sr-stats DEBUG] 2023/03/06 18:05:50 consumer/broker/1 added subscription to <topic name>/9
[sr-stats] 2023/03/06 18:05:50 acked 100 messages
[sr-stats DEBUG] 2023/03/06 18:05:50 consumer/broker/1 added subscription to <topic name>/8
[sr-stats] 2023/03/06 18:05:50 acked 100 messages
[sr-stats] 2023/03/06 18:05:50 acked 100 messages
[sr-stats] 2023/03/06 18:05:50 acked 100 messages
[sr-stats] 2023/03/06 18:05:50 acked 100 messages
[sr-stats] 2023/03/06 18:05:50 acked 100 messages
[sr-stats] 2023/03/06 18:05:50 acked 100 messages
[sr-stats] 2023/03/06 18:05:50 acked 100 messages
[sr-stats] 2023/03/06 18:05:50 acked 100 messages
[sr-stats] 2023/03/06 18:05:50 acked 100 messages
[sr-stats] 2023/03/06 18:05:50 acked 100 messages
[sr-stats] 2023/03/06 18:05:50 acked 100 messages
[sr-stats] 2023/03/06 18:05:50 acked 100 messages
[sr-stats] 2023/03/06 18:05:50 acked 100 messages
[sr-stats] 2023/03/06 18:05:50 acked 100 messages
[sr-stats] 2023/03/06 18:05:50 acked 100 messages
[sr-stats] 2023/03/06 18:05:52 acked 100 messages
[sr-stats] 2023/03/06 18:05:52 acked 100 messages
[sr-stats] 2023/03/06 18:05:52 acked 100 messages
[sr-stats] 2023/03/06 18:05:52 acked 100 messages
[sr-stats] 2023/03/06 18:05:52 acked 100 messages
[sr-stats] 2023/03/06 18:05:52 acked 100 messages
[sr-stats] 2023/03/06 18:05:52 acked 100 messages
[sr-stats] 2023/03/06 18:05:52 acked 100 messages
[sr-stats] 2023/03/06 18:05:52 acked 100 messages
[sr-stats] 2023/03/06 18:05:52 acked 100 messages
[sr-stats] 2023/03/06 18:05:52 acked 100 messages
[sr-stats] 2023/03/06 18:05:52 acked 100 messages
[sr-stats] 2023/03/06 18:05:52 acked 100 messages
[sr-stats] 2023/03/06 18:05:52 acked 100 messages
[sr-stats] 2023/03/06 18:05:52 acked 100 messages
[sr-stats] 2023/03/06 18:05:52 acked 100 messages
[sr-stats] 2023/03/06 18:05:52 acked 100 messages
[sr-stats] 2023/03/06 18:05:52 acked 100 messages
[sr-stats] 2023/03/06 18:05:52 acked 100 messages
[sr-stats] 2023/03/06 18:05:52 acked 100 messages
[sr-stats] 2023/03/06 18:05:52 acked 100 messages
[sr-stats] 2023/03/06 18:05:52 acked 100 messages
[sr-stats] 2023/03/06 18:05:52 acked 100 messages
[sr-stats] 2023/03/06 18:05:52 acked 100 messages
[sr-stats] 2023/03/06 18:05:52 acked 100 messages
[sr-stats] 2023/03/06 18:05:52 acked 100 messages
[sr-stats] 2023/03/06 18:05:52 acked 100 messages
[sr-stats] 2023/03/06 18:05:52 acked 100 messages
[sr-stats] 2023/03/06 18:05:52 acked 100 messages
[sr-stats] 2023/03/06 18:05:52 acked 100 messages
[sr-stats] 2023/03/06 18:05:52 acked 100 messages
[sr-stats] 2023/03/06 18:05:52 acked 100 messages
[sr-stats] 2023/03/06 18:05:52 acked 100 messages
[sr-stats] 2023/03/06 18:05:52 acked 100 messages
[sr-stats] 2023/03/06 18:05:52 acked 100 messages
[sr-stats] 2023/03/06 18:05:52 acked 100 messages
[sr-stats] 2023/03/06 18:05:52 acked 100 messages
[sr-stats] 2023/03/06 18:05:52 acked 100 messages
[sr-stats] 2023/03/06 18:05:52 acked 100 messages
[sr-stats] 2023/03/06 18:05:52 acked 100 messages
[sr-stats] 2023/03/06 18:05:52 acked 100 messages
[sr-stats] 2023/03/06 18:05:52 acked 100 messages
[sr-stats] 2023/03/06 18:05:52 acked 100 messages
[sr-stats] 2023/03/06 18:05:52 acked 100 messages
[sr-stats] 2023/03/06 18:05:52 acked 100 messages
[sr-stats] 2023/03/06 18:05:52 acked 100 messages
[sr-stats] 2023/03/06 18:05:55 acked 100 messages
[sr-stats] 2023/03/06 18:05:55 acked 100 messages
[sr-stats] 2023/03/06 18:05:55 acked 100 messages
[sr-stats] 2023/03/06 18:05:55 acked 100 messages
[sr-stats] 2023/03/06 18:05:55 acked 100 messages
[sr-stats] 2023/03/06 18:05:55 acked 100 messages
[sr-stats] 2023/03/06 18:05:55 acked 100 messages
[sr-stats] 2023/03/06 18:05:55 acked 100 messages
[sr-stats] 2023/03/06 18:05:55 acked 100 messages
[sr-stats] 2023/03/06 18:05:55 acked 100 messages
[sr-stats] 2023/03/06 18:05:55 acked 100 messages
[sr-stats] 2023/03/06 18:05:55 acked 100 messages
[sr-stats] 2023/03/06 18:05:55 acked 100 messages
[sr-stats] 2023/03/06 18:05:55 acked 100 messages
[sr-stats] 2023/03/06 18:05:55 acked 100 messages
[sr-stats] 2023/03/06 18:05:55 acked 100 messages
[sr-stats] 2023/03/06 18:05:55 acked 100 messages
[sr-stats] 2023/03/06 18:05:55 acked 100 messages
[sr-stats] 2023/03/06 18:05:55 acked 100 messages
[sr-stats] 2023/03/06 18:05:55 acked 100 messages
[sr-stats] 2023/03/06 18:05:55 acked 100 messages
[sr-stats] 2023/03/06 18:05:55 acked 100 messages
[sr-stats] 2023/03/06 18:05:55 acked 100 messages
[sr-stats] 2023/03/06 18:05:55 acked 100 messages
[sr-stats] 2023/03/06 18:05:55 acked 100 messages
[sr-stats] 2023/03/06 18:05:55 acked 100 messages
[sr-stats] 2023/03/06 18:05:55 acked 100 messages
[sr-stats] 2023/03/06 18:05:55 acked 100 messages
[sr-stats] 2023/03/06 18:05:55 acked 100 messages
[sr-stats] 2023/03/06 18:05:55 acked 100 messages
[sr-stats] 2023/03/06 18:05:55 acked 100 messages
[sr-stats] 2023/03/06 18:05:55 acked 100 messages
[sr-stats] 2023/03/06 18:05:55 acked 100 messages
[sr-stats] 2023/03/06 18:05:55 acked 100 messages
[sr-stats] 2023/03/06 18:05:55 acked 100 messages
[sr-stats] 2023/03/06 18:05:55 acked 100 messages
[sr-stats] 2023/03/06 18:05:55 acked 100 messages
[sr-stats] 2023/03/06 18:05:55 acked 100 messages
[sr-stats] 2023/03/06 18:05:55 acked 100 messages
[sr-stats] 2023/03/06 18:05:55 acked 100 messages
[sr-stats] 2023/03/06 18:05:55 acked 100 messages
[sr-stats] 2023/03/06 18:05:55 acked 100 messages
[sr-stats] 2023/03/06 18:05:55 acked 100 messages
[sr-stats] 2023/03/06 18:05:55 acked 100 messages
[sr-stats] 2023/03/06 18:05:55 acked 100 messages
[sr-stats] 2023/03/06 18:05:55 acked 100 messages
[sr-stats] 2023/03/06 18:05:55 acked 100 messages
[sr-stats] 2023/03/06 18:05:55 acked 100 messages
[sr-stats] 2023/03/06 18:05:55 acked 100 messages
[sr-stats] 2023/03/06 18:05:55 acked 100 messages
[sr-stats] 2023/03/06 18:05:56 acked 100 messages
[sr-stats] 2023/03/06 18:05:56 acked 100 messages
[sr-stats] 2023/03/06 18:05:56 acked 100 messages
[sr-stats] 2023/03/06 18:05:56 acked 100 messages
[sr-stats] 2023/03/06 18:05:56 acked 100 messages
[sr-stats] 2023/03/06 18:05:56 acked 100 messages
[sr-stats] 2023/03/06 18:05:56 acked 100 messages
[sr-stats] 2023/03/06 18:05:56 acked 100 messages
[sr-stats] 2023/03/06 18:05:56 acked 100 messages
[sr-stats] 2023/03/06 18:05:56 acked 100 messages
[sr-stats] 2023/03/06 18:05:56 acked 100 messages
[sr-stats] 2023/03/06 18:05:56 acked 100 messages
[sr-stats] 2023/03/06 18:05:56 acked 100 messages
[sr-stats] 2023/03/06 18:05:56 acked 100 messages
[sr-stats] 2023/03/06 18:05:56 acked 100 messages
panic: runtime error: slice bounds out of range [:5] with capacity 0

goroutine 64 [running]:
main.(*Consumer).ConsumeClaim(0xc00014a680, {0x824380, 0xc00021a120}, {0x822cc8, 0xc000490f90})
        /home/runner/work/schema-registry-statistics/schema-registry-statistics/main.go:111 +0x275
github.com/Shopify/sarama.(*consumerGroupSession).consume(0xc00021a120, {0xc000220510, 0x2b}, 0xa)
        /home/runner/go/pkg/mod/github.com/!shopify/sarama@v1.37.2/consumer_group.go:868 +0x26e
github.com/Shopify/sarama.newConsumerGroupSession.func2({0xc000220510?, 0x0?}, 0x0?)
        /home/runner/go/pkg/mod/github.com/!shopify/sarama@v1.37.2/consumer_group.go:793 +0x76
created by github.com/Shopify/sarama.newConsumerGroupSession
        /home/runner/go/pkg/mod/github.com/!shopify/sarama@v1.37.2/consumer_group.go:785 +0x4b4

@whatsupbros whatsupbros changed the title panic: runtime error: slice bounds out of range panic: runtime error: slice bounds out of range (1.2.0) Mar 6, 2023
@EladLeev EladLeev self-assigned this Mar 7, 2023
@EladLeev EladLeev added the bug Something isn't working label Mar 7, 2023
@EladLeev EladLeev mentioned this issue Apr 6, 2023
Merged
@EladLeev
Copy link
Owner

EladLeev commented Apr 6, 2023

@whatsupbros - Sorry it took time, can you please check v1.2.1?
I assume that the error happened due to empty messages, or messages that doesn't contains schemas.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@EladLeev EladLeev

Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix slice bound EladLeev/schema-registry-statistics
2 participants
@whatsupbros @EladLeev