Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Slight different between Scylla and Apache Cassandra TTL #4263

Closed
tzach opened this issue Feb 24, 2019 · 5 comments
Closed

Slight different between Scylla and Apache Cassandra TTL #4263

tzach opened this issue Feb 24, 2019 · 5 comments

Comments

@tzach
Copy link
Contributor

@tzach tzach commented Feb 24, 2019

Installation details
Scylla version (or git commit hash): 3.0.3
Apache Cassandra 3.11

Just before a row is TTLed, Scylla return a null value, while Cassandra does not

Setup

cqlsh> CREATE KEYSPACE mykeyspace WITH replication = {'class': 'SimpleStrategy', 'replication_factor': '1'}  AND durable_writes = true;
cqlsh> USE mykeyspace;
cqlsh:mykeyspace> CREATE TABLE heartrate_ttl (
              ...  pet_chip_id  uuid,
              ...  time timestamp,
              ...  heart_rate int,
              ... PRIMARY KEY (pet_chip_id, time));
cqlsh:mykeyspace>  INSERT INTO heartrate_ttl(pet_chip_id, time, heart_rate) VALUES (123e4567-e89b-12d3-a456-426655440b23, '2019-03-04 07:04:00', 131) USING TTL 21;
cqlsh:mykeyspace> select pet_chip_id,time, heart_rate, TTL(heart_rate) from heartrate_ttl;

pet_chip_id                          | time                            | heart_rate | ttl(heart_rate)
--------------------------------------+---------------------------------+------------+-----------------
 123e4567-e89b-12d3-a456-426655440b23 | 2019-03-04 07:04:00.000000+0000 |        131 |              10

Just before TTL is over Scylla will return

cqlsh:mykeyspace> select pet_chip_id,time, heart_rate, TTL(heart_rate) from heartrate_ttl;

 pet_chip_id                          | time                            | heart_rate | ttl(heart_rate)
--------------------------------------+---------------------------------+------------+-----------------
 123e4567-e89b-12d3-a456-426655440b23 | 2019-03-04 07:04:00.000000+0000 |       null |            null

and after a second

pet_chip_id | time | heart_rate | ttl(heart_rate)
-------------+------+------------+-----------------

I cound not repreduce this null value with Cassandra. Not sure its a bug

@avikivity

This comment has been minimized.

Copy link
Contributor

@avikivity avikivity commented Feb 24, 2019

Maybe the row marker's liveness and the column's liveness are evaluted using different base timestamps (at two different points in the query lifecycle).

@haaawk

This comment has been minimized.

Copy link
Member

@haaawk haaawk commented Nov 18, 2019

@eliransin Can we reassign to @piodul since he's working on #5290 which seems related?

@eliransin

This comment has been minimized.

Copy link
Contributor

@eliransin eliransin commented Nov 18, 2019

@haaawk NP. Let me know if it turns out to be unrelated :)

@haaawk haaawk assigned piodul and haaawk and unassigned eliransin Nov 18, 2019
piodul added a commit to piodul/scylla that referenced this issue Nov 18, 2019
Corrects condition on which a row was considered expired by its TTL.
The previous, invalid logic caused a situation in which a row inserted
with TTL changed all its non-key cells to null one second before the row
disappeared.

Fixes: scylladb#4263, scylladb#5290.

Tests: unit(dev)
nyh added a commit that referenced this issue Nov 19, 2019
Merged patch set by Piotr Dulikowski:

This change corrects condition on which a row was considered expired by its
TTL.

The logic that decides when a row becomes expired was inconsistent with the
logic that decides if a single cell is expired. A single cell becomes expired
when expiry_timestamp <= now, while a row became expired when
expiry_timestamp < now (notice the strict inequality). For rows inserted
with TTL, this caused non-key cells to expire (change their values to null)
one second before the row disappeared. Now, row expiry logic uses non-strict
inequality.

Fixes #4263,
Fixes #5290.

Tests:

    unit(dev)
    python test described in issue #5290
avikivity added a commit that referenced this issue Nov 20, 2019
Merged patch set by Piotr Dulikowski:

This change corrects condition on which a row was considered expired by its
TTL.

The logic that decides when a row becomes expired was inconsistent with the
logic that decides if a single cell is expired. A single cell becomes expired
when expiry_timestamp <= now, while a row became expired when
expiry_timestamp < now (notice the strict inequality). For rows inserted
with TTL, this caused non-key cells to expire (change their values to null)
one second before the row disappeared. Now, row expiry logic uses non-strict
inequality.

Fixes #4263,
Fixes #5290.

Tests:

    unit(dev)
    python test described in issue #5290

(cherry picked from commit 9b9609c)
avikivity added a commit that referenced this issue Nov 20, 2019
Merged patch set by Piotr Dulikowski:

This change corrects condition on which a row was considered expired by its
TTL.

The logic that decides when a row becomes expired was inconsistent with the
logic that decides if a single cell is expired. A single cell becomes expired
when expiry_timestamp <= now, while a row became expired when
expiry_timestamp < now (notice the strict inequality). For rows inserted
with TTL, this caused non-key cells to expire (change their values to null)
one second before the row disappeared. Now, row expiry logic uses non-strict
inequality.

Fixes #4263,
Fixes #5290.

Tests:

    unit(dev)
    python test described in issue #5290

(cherry picked from commit 9b9609c)
avikivity added a commit that referenced this issue Nov 20, 2019
Merged patch set by Piotr Dulikowski:

This change corrects condition on which a row was considered expired by its
TTL.

The logic that decides when a row becomes expired was inconsistent with the
logic that decides if a single cell is expired. A single cell becomes expired
when expiry_timestamp <= now, while a row became expired when
expiry_timestamp < now (notice the strict inequality). For rows inserted
with TTL, this caused non-key cells to expire (change their values to null)
one second before the row disappeared. Now, row expiry logic uses non-strict
inequality.

Fixes #4263,
Fixes #5290.

Tests:

    unit(dev)
    python test described in issue #5290

(cherry picked from commit 9b9609c)
(cherry picked from commit 95acf71)
@avikivity

This comment has been minimized.

Copy link
Contributor

@avikivity avikivity commented Nov 20, 2019

Backported to 3.0, 3.1, 3.2

@avikivity

This comment has been minimized.

Copy link
Contributor

@avikivity avikivity commented Jan 5, 2020

Already backported everywhere.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
7 participants
You can’t perform that action at this time.