Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[YCQL][LST] Error decoding response from Cassandra. ver(4); flags(0000); stream(57); op(8); offset(9); len(37); buffer: b'\x84\x00\x009\x08\x00\x00\x00%\x00\x00\x00\x02\x00\x00\x00\x01\x00\x00\x00\x00\x00\rks_lst_833429\x00\x04ct_0\x00\x00\x00\x00' #13946

Open
def- opened this issue Sep 9, 2022 · 2 comments
Open

[YCQL][LST] Error decoding response from Cassandra. ver(4); flags(0000); stream(57); op(8); offset(9); len(37); buffer: b'\x84\x00\x009\x08\x00\x00\x00%\x00\x00\x00\x02\x00\x00\x00\x01\x00\x00\x00\x00\x00\rks_lst_833429\x00\x04ct_0\x00\x00\x00\x00' #13946

def- opened this issue Sep 9, 2022 · 2 comments
Labels
area/ycql Yugabyte CQL (YCQL) kind/bug This issue is a bug kind/failing-test Tests and testing infra LST_LRT LRT bugs encountered within long-system-test priority/medium Medium priority issue qa_automation Bugs identified via itest-system, LST, Stress automation or causing automation failures qa_lst Bugs identified using lst automation

Comments

@def-
Copy link
Contributor

def- commented Sep 9, 2022 •
edited by yugabyte-ci

Jira Link: DB-3442

Description

On a local macOS run of the new LST YCQL feature @vijayjoshi16 is working on I ran this error that can be reproduced easily:

$ cd ~/code/yugabyte-db
$ git checkout dbb4a340fa6cf7ad9698ca57dfce8ae8085fb636
$ ./yb_build.sh release
$ bin/yb-ctl create
$ cd ~/code/yb-long-system-test
$ git checkout a66d6d77
$ ./long_system_test.py --ycql --runtime=1 --threads=10 --seed=833429
2022-09-09 11:03:53,936 MainThread INFO
2022-09-09 11:03:53,936 MainThread INFO     --------------------------------------------------------------------------------
2022-09-09 11:03:53,936 MainThread INFO     Running Long System Test 0.1 - https://github.com/yugabyte/yb-long-system-test/
2022-09-09 11:03:53,936 MainThread INFO     --------------------------------------------------------------------------------
2022-09-09 11:03:53,936 MainThread INFO
2022-09-09 11:03:54,013 MainThread INFO     Reproduce with: git checkout a66d6d77 && ./long_system_test.py --ycql --runtime=1 --threads=10 --seed=833429
2022-09-09 11:03:54,017 MainThread WARNING  Cluster.__init__ called with contact_points specified, but no load_balancing_policy. In the next major version, this will raise an error; please specify a load-balancing policy. (contact_points = ['localhost'], lbp = None)
2022-09-09 11:03:54,022 MainThread WARNING  Downgrading core protocol version from 66 to 65 for 127.0.0.1:9042. To avoid this, it is best practice to explicitly set Cluster(protocol_version) to the version supported by your cluster. http://datastax.github.io/python-driver/api/cassandra/cluster.html#cassandra.cluster.Cluster.protocol_version
2022-09-09 11:03:54,022 MainThread WARNING  Downgrading core protocol version from 65 to 5 for 127.0.0.1:9042. To avoid this, it is best practice to explicitly set Cluster(protocol_version) to the version supported by your cluster. http://datastax.github.io/python-driver/api/cassandra/cluster.html#cassandra.cluster.Cluster.protocol_version
2022-09-09 11:03:54,023 MainThread WARNING  Downgrading core protocol version from 5 to 4 for 127.0.0.1:9042. To avoid this, it is best practice to explicitly set Cluster(protocol_version) to the version supported by your cluster. http://datastax.github.io/python-driver/api/cassandra/cluster.html#cassandra.cluster.Cluster.protocol_version
2022-09-09 11:03:54,029 MainThread INFO     Using datacenter 'datacenter1' for DCAwareRoundRobinPolicy (via host '127.0.0.1:9042'); if incorrect, please specify a local_dc to the constructor, or limit contact points to local cluster nodes
2022-09-09 11:03:54,029 MainThread INFO     Cassandra host ::1:9042 removed
2022-09-09 11:03:54,172 MainThread INFO     Starting worker_0: YCQLRandomSelectAction, YCQLSingleInsertAction, YCQLSingleUpdateAction, YCQLBulkInsertAction, YCQLBulkUpdateAction, YCQLSingleDeleteAction
2022-09-09 11:03:54,173 MainThread INFO     Starting worker_1: YCQLRandomSelectAction, YCQLSingleInsertAction, YCQLSingleUpdateAction, YCQLBulkInsertAction, YCQLBulkUpdateAction, YCQLSingleDeleteAction
2022-09-09 11:03:54,173 MainThread INFO     Starting worker_2: YCQLRandomSelectAction, YCQLSingleInsertAction, YCQLSingleUpdateAction, YCQLBulkInsertAction, YCQLBulkUpdateAction, YCQLSingleDeleteAction
2022-09-09 11:03:54,173 MainThread INFO     Starting worker_3: YCQLRandomSelectAction, YCQLSingleInsertAction, YCQLSingleUpdateAction, YCQLBulkInsertAction, YCQLBulkUpdateAction, YCQLSingleDeleteAction
2022-09-09 11:03:54,176 MainThread INFO     Starting worker_4: YCQLRandomSelectAction, YCQLSingleInsertAction, YCQLSingleUpdateAction, YCQLBulkInsertAction, YCQLBulkUpdateAction, YCQLSingleDeleteAction
2022-09-09 11:03:54,177 MainThread INFO     Starting worker_5: YCQLRandomSelectAction, YCQLSingleInsertAction, YCQLSingleUpdateAction, YCQLBulkInsertAction, YCQLBulkUpdateAction, YCQLSingleDeleteAction
2022-09-09 11:03:54,177 MainThread INFO     Starting worker_6: YCQLRandomSelectAction, YCQLSingleInsertAction, YCQLSingleUpdateAction, YCQLBulkInsertAction, YCQLBulkUpdateAction, YCQLSingleDeleteAction
2022-09-09 11:03:54,178 MainThread INFO     Starting worker_7: YCQLRandomSelectAction, YCQLSingleInsertAction, YCQLSingleUpdateAction, YCQLBulkInsertAction, YCQLBulkUpdateAction, YCQLSingleDeleteAction
2022-09-09 11:03:54,179 MainThread INFO     Starting worker_8: YCQLRandomSelectAction, YCQLSingleInsertAction, YCQLSingleUpdateAction, YCQLBulkInsertAction, YCQLBulkUpdateAction, YCQLSingleDeleteAction
2022-09-09 11:03:54,181 MainThread INFO     Starting worker_9: YCQLRandomSelectAction, YCQLSingleInsertAction, YCQLSingleUpdateAction, YCQLBulkInsertAction, YCQLBulkUpdateAction, YCQLSingleDeleteAction
2022-09-09 11:03:54,183 asyncore_cassandra_driver_event_loop ERROR    Error decoding response from Cassandra. ver(4); flags(0000); stream(57); op(8); offset(9); len(37); buffer: b'\x84\x00\x009\x08\x00\x00\x00%\x00\x00\x00\x02\x00\x00\x00\x01\x00\x00\x00\x00\x00\rks_lst_833429\x00\x04ct_0\x00\x00\x00\x00'
Traceback (most recent call last):
  File "cassandra/connection.py", line 1229, in cassandra.connection.Connection.process_msg
  File "cassandra/protocol.py", line 1196, in cassandra.protocol._ProtocolHandler.decode_message
  File "cassandra/protocol.py", line 744, in cassandra.protocol.ResultMessage.recv_body
  File "cassandra/protocol.py", line 730, in cassandra.protocol.ResultMessage.recv
  File "cassandra/row_parser.pyx", line 34, in cassandra.row_parser.make_recv_results_rows.recv_results_rows
  File "cassandra/deserializers.pyx", line 478, in cassandra.deserializers.make_deserializers
  File "cassandra/deserializers.pyx", line 519, in cassandra.deserializers.obj_array
  File "stringsource", line 154, in View.MemoryView.array.__cinit__
ValueError: Invalid shape in axis 0: 0.

I'll try to get further information as to what's going wrong. We are not using Vector and Array. I am using yb-cassandra-driver Python driver so that should work too.
@def- def- added area/ycql Yugabyte CQL (YCQL) status/awaiting-triage Issue awaiting triage labels Sep 9, 2022
@yugabyte-ci yugabyte-ci added kind/bug This issue is a bug priority/medium Medium priority issue labels Sep 9, 2022
@def-
Copy link
Contributor Author

def- commented Sep 9, 2022 •
edited

This seems to occur with a wide variety of types

2022-09-09 16:34:28,098 asyncore_cassandra_driver_event_loop ERROR    Error decoding response from Cassandra. ver(4); flags(0000); stream(57); op(8); offset(9); len(37); buffer: b'\x84\x00\x009\x08\x00\x00\x00%\x00\x00\x00\x02\x00\x00\x00\x01\x00\x00\x00\x00\x00\rks_lst_758905\x00\x04ct_1\x00\x00\x00\x00\x84\x00\x001\x08\x00\x00\x00:\x00\x00\x00\x02\x00\x00\x00\x01\x00\x00\x00\x02\x00\rks_lst_758905\x00\x04ct_0\x00\x06c1_int\x00\t\x00\x07c0_time\x00\x12\x00\x00\x00\x00\x84\x00\x005\x08\x00\x00\x00\x04\x00\x00\x00\x01\x84\x00\x008\x08\x00\x00\x00\x04\x00\x00\x00\x01\x84\x00\x006\x08\x00\x00\x00\x04\x00\x00\x00\x01\x84\x00\x002\x08\x00\x00\x00\x04\x00\x00\x00\x01\x84\x00\x003\x08\x00\x00\x00\x04\x00\x00\x00\x01\x84\x00\x007\x08\x00\x00\x00n\x00\x00\x00\x02\x00\x00\x00\x01\x00\x00\x00\x05\x00\rks_lst_758905\x00\x04ct_1\x00\x0bc4_set_text\x00"\x00\r\x00\x0bc0_timeuuid\x00\x0f\x00\nc1_boolean\x00\x04\x00\x07c3_text\x00\r\x00\x0cc2_timestamp\x00\x0b\x00\x00\x00\x00\x84\x00\x000\x08\x00\x00\x00\x04\x00\x00\x00\x01\x84\x00\x004\x08\x00\x00\x00\x04\x00\x00\x00\x01'
Traceback (most recent call last):
  File "cassandra/connection.py", line 1229, in cassandra.connection.Connection.process_msg
  File "cassandra/protocol.py", line 1196, in cassandra.protocol._ProtocolHandler.decode_message
  File "cassandra/protocol.py", line 744, in cassandra.protocol.ResultMessage.recv_body
  File "cassandra/protocol.py", line 730, in cassandra.protocol.ResultMessage.recv
  File "cassandra/row_parser.pyx", line 34, in cassandra.row_parser.make_recv_results_rows.recv_results_rows
  File "cassandra/deserializers.pyx", line 478, in cassandra.deserializers.make_deserializers
  File "cassandra/deserializers.pyx", line 519, in cassandra.deserializers.obj_array
  File "stringsource", line 154, in View.MemoryView.array.__cinit__
ValueError: Invalid shape in axis 0: 0.
2022-09-09 16:34:28,098 ThreadPoolExecutor-0_1 WARNING  Host 127.0.0.1:9042 has been marked down
2022-09-09 16:34:28,098 worker_5   ERROR    Unexpected query failure: ValueError
Query: SELECT c4_set_text, c1_boolean, c0_timeuuid, c3_text from ct_1 LIMIT 0 OFFSET 11
  values: None
  runtime: 2022-09-09 16:34:28.079 - 2022-09-09 16:34:28.098
  supports explain: False
  supports rollback: False
  affected rows: None
Action: YCQLRandomSelectAction
Error class: ValueError
Error message: Invalid shape in axis 0: 0.

Just removing some doesn't fix it.

@def-
Copy link
Contributor Author

def- commented Sep 9, 2022

The common pattern is that this always happens with LIMIT 0. I'll use a minimum limit 1 for now in LST.

@kripasreenivasan kripasreenivasan added the qa_automation Bugs identified via itest-system, LST, Stress automation or causing automation failures label Sep 13, 2022
@yugabyte-ci yugabyte-ci removed the status/awaiting-triage Issue awaiting triage label Sep 28, 2022
@yugabyte-ci yugabyte-ci added the kind/failing-test Tests and testing infra label Oct 12, 2022
@Karvy-yb Karvy-yb added the LST_LRT LRT bugs encountered within long-system-test label Mar 15, 2023
@Arjun-yb Arjun-yb added the qa_lst Bugs identified using lst automation label Dec 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area/ycql Yugabyte CQL (YCQL) kind/bug This issue is a bug kind/failing-test Tests and testing infra LST_LRT LRT bugs encountered within long-system-test priority/medium Medium priority issue qa_automation Bugs identified via itest-system, LST, Stress automation or causing automation failures qa_lst Bugs identified using lst automation
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@def- @m-iancu @yugabyte-ci @Arjun-yb @kripasreenivasan @Karvy-yb