Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

serialization of negative varints is broken #5656

Closed
espindola opened this issue Jan 27, 2020 · 3 comments
Closed

serialization of negative varints is broken #5656

espindola opened this issue Jan 27, 2020 · 3 comments

Comments

@espindola
Copy link
Contributor

Running the following CQL on an empty database

CREATE KEYSPACE test_ks WITH REPLICATION = { 'class' : 'SimpleStrategy' , 'replication_factor' : 1 };                                                                                                                                      
use test_ks;                                                                                                                                                                                                                               
CREATE TABLE my_table (val varint PRIMARY KEY) with compression = {'sstable_compression': ''};                                                                                                                                             
INSERT INTO my_table (val) VALUES (-2147483648); 

In scylla produces the sstable

00 05                   key_length                                                                                                                                                                                                         
ff 80 00 00 00          key                                                                                                                                                                                                                
7f ff ff ff             local_deletion_time                                                                                                                                                                                                
80 00 00 00 00 00 00 00 marked_for_delete_at                                                                                                                                                                                               
24                      HAS_ALL_COLUMNS | HAS_TIMESTAMP                                                                                                                                                                                                          
02                      row_body_size
13                      prev_unfiltered_size
00                      delta_timestamp
01                      END_OF_PARTITION

With cassandra I get

00 04                   key_length                                                                                                                                                                                                         
80 00 00 00             key                                                                                                                                                                                                                
7f ff ff ff             local_deletion_time                                                                                                                                                                                                
80 00 00 00 00 00 00 00 marked_for_delete_at                                                                                                                                                                                               
24                      HAS_ALL_COLUMNS | HAS_TIMESTAMP                                                                                                                                                                                                          
02                      row_body_size
12                      prev_unfiltered_size
00                      delta_timestamp
01                      END_OF_PARTITION

Note how the scylla sstable has an extra 0xff at the start of the varint value.

When prepared statements are used, scylla writes the value produced by the driver, so the bug is hidden.

@espindola
Copy link
Contributor Author

Should this be backported?

avikivity pushed a commit that referenced this issue Feb 2, 2020
We would sometimes produce an unnecessary extra 0xff prefix byte.

The new encoding matches what cassandra does.

This was both a efficiency and correctness issue, as using varint in a
key could produce different tokens.

Fixes #5656

Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
(cherry picked from commit c89c90d)
avikivity pushed a commit that referenced this issue Feb 2, 2020
We would sometimes produce an unnecessary extra 0xff prefix byte.

The new encoding matches what cassandra does.

This was both a efficiency and correctness issue, as using varint in a
key could produce different tokens.

Fixes #5656

Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
(cherry picked from commit c89c90d)
avikivity pushed a commit that referenced this issue Feb 2, 2020
We would sometimes produce an unnecessary extra 0xff prefix byte.

The new encoding matches what cassandra does.

This was both a efficiency and correctness issue, as using varint in a
key could produce different tokens.

Fixes #5656

Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
(cherry picked from commit c89c90d)
@avikivity
Copy link
Member

Backported to 3.3, 3.2, 3.1.

@tzach
Copy link
Contributor

tzach commented Feb 25, 2020

reproducer by @espindola

#!/usr/bin/python

from cassandra.cluster import Cluster

cluster = Cluster()
session = cluster.connect()

session.execute("CREATE KEYSPACE test_ks WITH REPLICATION = { 'class' : 'SimpleStrategy' , 'replication_factor' : 1 };")
session.execute("use test_ks;")
session.execute("CREATE TABLE my_table (val varint PRIMARY KEY) with compression = {'sstable_compression': ''};")

p = session.prepare("INSERT INTO my_table (val) VALUES (?);")

session.execute(p.bind([-2147483648]))

session.execute("INSERT INTO my_table (val) VALUES (-2147483648);")

print(session.execute("SELECT * FROM my_table;"))

avikivity pushed a commit to avikivity/scylladb that referenced this issue Apr 28, 2020
We would sometimes produce an unnecessary extra 0xff prefix byte.

The new encoding matches what cassandra does.

This was both a efficiency and correctness issue, as using varint in a
key could produce different tokens.

Fixes scylladb#5656

Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
(cherry picked from commit c89c90d)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants