Cassandra 3.11.10 uses "class" instead of "sstable_compression" for compression settings by default #8948

agalue · 2021-06-29T15:16:41Z

This is Scylla's bug tracker, to be used for reporting bugs only.
If you have a question about Scylla, and not a bug, please ask it in
our mailing-list at scylladb-dev@googlegroups.com or in our slack channel.

I have read the disclaimer above, and I am reporting a suspected malfunction in Scylla.

Installation details
Scylla version (or git commit hash): 4.4.3-0.20210609.bfafb8456)
Cluster size: 1 (test environment)
OS (RHEL/CentOS/Ubuntu/AWS AMI): CentOS

Hardware details (for performance issues) Delete if unneeded
Platform (physical/VM/cloud instance type/docker):
Hardware: sockets= cores= hyperthreading= memory=
Disks: (SSD/HDD, count)

I was trying to migrate to Scylla on a test environment by letting it use the same data directory as Cassandra left with all the data (i.e., /var/lib/cassandra) to see if that works.

After doing that, Scylla refused to start because of the following error:

Jun 28 15:19:59 scylla-apex1 scylla[33784]:  [shard 0] init - Startup failed: exceptions::configuration_exception (Unknown compression option 'class'.)

I found that Cassandra deprecated sstable_compression (and other sibling fields), and it now uses class, as described here. Interestingly, Cassandra allows both, but complains when sstable_compression is used with a warning on the logs.

I'm wondering if there is a way to fix this in Scylla, to allow it to use the same Cassandra Data directory without the need of using the sstableloader tool to migrate the data and reduce the downtime.

The text was updated successfully, but these errors were encountered:

…sstable_compression' Cassandra 3.0 deprecated the 'sstable_compression' attribute and added 'class' as a replacement. Follow by supporting both. The SSTABLE_COMPRESSION variable is renamed to SSTABLE_COMPRESSION_DEPRECATED to detect all uses and prevent future misuse. Existing unit tests are modified to use the new name, and a test is added to ensure the old name is still supported. Fixes scylladb#8948.

slivne · 2021-07-06T15:28:13Z

@avikivity if you want me to reassign we can

…sstable_compression' Cassandra 3.0 deprecated the 'sstable_compression' attribute and added 'class' as a replacement. Follow by supporting both. The SSTABLE_COMPRESSION variable is renamed to SSTABLE_COMPRESSION_DEPRECATED to detect all uses and prevent future misuse. To prevent old-version nodes from seeing the new name, the compression_parameters class preserves the key name when it is constructed from an options map, and emits the same key name when asked to generate an options map. Existing unit tests are modified to use the new name, and a test is added to ensure the old name is still supported. Fixes scylladb#8948.

…recate 'sstable_compression'" This reverts commit 5571ef0. It causes rolling upgrade failures. Fixes #9055. Reopens #8948.

avikivity · 2021-07-28T11:15:13Z

Reopened by 331eb57

…sstable_compression' Cassandra 3.0 deprecated the 'sstable_compression' attribute and added 'class' as a replacement. Follow by supporting both. The SSTABLE_COMPRESSION variable is renamed to SSTABLE_COMPRESSION_DEPRECATED to detect all uses and prevent future misuse. To prevent old-version nodes from seeing the new name, the compression_parameters class preserves the key name when it is constructed from an options map, and emits the same key name when asked to generate an options map. Existing unit tests are modified to use the new name, and a test is added to ensure the old name is still supported. Fixes scylladb#8948. Closes scylladb#8949

nyh · 2022-02-06T13:20:22Z

This difference is reproduced by the translated Cassandra unit test cassandra_tests/validation/operations/alter_test.py::testAlterTableWithCompression:

Whereas the default setting for Cassandra is {'chunk_length_in_kb': '16', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}, in Scylla it is {'sstable_compression': 'org.apache.cassandra.io.compress.LZ4Compressor'} so the first check in that test fails.

tzach · 2022-02-06T16:03:29Z

See more schema incompatibles here #9859

This is a translation of Cassandra's CQL unit test source file validation/operations/AlterTest.java into our our cql-pytest framework. This test file includes 24 tests for various types of ALTER operations (of keyspaces, tables and types). Two additional tests which required multiple data centers to test were dropped with a comment explaining why. All 24 tests pass on Cassandra, with 8 failing on Scylla reproducing one already known Scylla issue and 5 previously-unknown ones: Refs #8948: Cassandra 3.11.10 uses "class" instead of "sstable_compression" for compression settings by default Refs #9929: Cassandra added "USING TIMESTAMP" to "ALTER TABLE", we didn't. Refs #9930: Forbid re-adding static columns as regular and vice versa Refs #9935: Scylla stores un-expanded compaction class name in system tables. Refs #10036: Reject empty options while altering a keyspace Refs #10037: If there are multiple values for a key, CQL silently chooses last value Signed-off-by: Nadav Har'El <nyh@scylladb.com> Message-Id: <20220206163820.1875410-2-nyh@scylladb.com>

jmaixl · 2022-05-16T18:53:05Z

@agalue if you migrate just the sstables for keyspaces, does that work?
I'm able to migrate specific keyspaces' sstables (so everything but system, system_auth, system_distributed, system_traces, hints).

agalue · 2022-05-25T15:51:41Z

I apologize for the delay in my response.

That could work, but what I wanted is precisely avoid the need to migrate SSTables manually.

I wanted to see if it is possible to stop and uninstall Cassandra, install ScyllaDB, point it to the Cassandra folder where the SSTables live, start ScyllaDB, and have it working, which is what triggered the problem I reported. That would minimize the downtime without the need for a migration, which is difficult considering the constraints we have.

If that's not possible, and the only way to have it working is to migrate the SSTables before starting using Scylla (as dual-writes are not an option for us), I'll let my team know.

nyh · 2023-06-15T08:39:52Z

Note that this issue is not just about different printing of the schema - it's also about a valid CREATE TABLE command from Cassandra (e.g., create table ... with compression = { 'class': 'LZ4Compressor' }) not working in Scylla. This can break applications who try to create such tables.

I Added two cql-pytest tests to reproduce these two aspects of this issue:

test_sstable_compression.py::test_compression_class is about a Cassandra application attempting to use the "class" parameter in a CREATE TABLE .. WITH COMPRESSION and failing.
test_sstable_compression.py::test_read_compression_class is about an application reading the schema of an existing table, hoping to find a "class" there (as is the case in modern Cassandra).

We have an old PR fixing this from @avikivity - #8949 - but as explained above it was reverted because of upgrade issues. Can we fix those and re-apply a new version of this fix?

This patch adds some minimal tests for the "with compression = {..}" table configuration. These tests reproduce three known bugs: Refs scylladb#6442: Always print all schema parameters (including default values) Scylla doesn't return the default chunk_length_in_kb, but Cassandra does. Refs scylladb#8948: Cassandra 3.11.10 uses "class" instead of "sstable_compression" for compression settings by default Cassandra switched, long ago, the "sstable_compression" attribute's name to "class". This can break Cassandra applications that create tables (where we won't understand the "class" parameter) and applications that inquire about the configuration of existing tables. This patch adds tests for both problems. Refs scylladb#9933: ALTER TABLE with "chunk_length_kb" (compression) of 1MB caused a core dump on all nodes Our test for this issue hangs Scylla (or crashes, depending on the test environment configuration), when a huge allocation is attempted during memtable flush. So this test is marked "skip" instead of xfail. The tests included here also uncovered a new minor/insignificant bug, where Scylla allows floating point numbers as chunk_length_in_kb - this number is truncated to an integer, and allowed, unlike Cassandra or common sense. Signed-off-by: Nadav Har'El <nyh@scylladb.com>

This patch adds some minimal tests for the "with compression = {..}" table configuration. These tests reproduce three known bugs: Refs #6442: Always print all schema parameters (including default values) Scylla doesn't return the default chunk_length_in_kb, but Cassandra does. Refs #8948: Cassandra 3.11.10 uses "class" instead of "sstable_compression" for compression settings by default Cassandra switched, long ago, the "sstable_compression" attribute's name to "class". This can break Cassandra applications that create tables (where we won't understand the "class" parameter) and applications that inquire about the configuration of existing tables. This patch adds tests for both problems. Refs #9933: ALTER TABLE with "chunk_length_kb" (compression) of 1MB caused a core dump on all nodes Our test for this issue hangs Scylla (or crashes, depending on the test environment configuration), when a huge allocation is attempted during memtable flush. So this test is marked "skip" instead of xfail. The tests included here also uncovered a new minor/insignificant bug, where Scylla allows floating point numbers as chunk_length_in_kb - this number is truncated to an integer, and allowed, unlike Cassandra or common sense. Signed-off-by: Nadav Har'El <nyh@scylladb.com> Closes #14261

nyh · 2023-10-18T20:28:28Z

This issue is a duplicate of #2061. One of them should be closed. The other issue is older, but this one has better explanations and also refers to a reverted patch.

This is a translation of Cassandra's CQL unit test source file validation/operations/CreateTest.java into our cql-pytest framework. The 15 tests did not reproduce any previously-unknown bug, but did provide additional reproducers for several known issues: Refs scylladb#6442: Always print all schema parameters (including default values) Refs scylladb#8001: Documented unit "µs" not supported for assigning a duration" type. Refs scylladb#8892: Add an option for default RF for new keyspaces. Refs scylladb#8948: Cassandra 3.11.10 uses "class" instead of "sstable_compression" for compression settings by default Unfortunately, I also had to comment out - and not translate - several tests which weren't real "CQL tests" (tests that use only the CQL driver), and instead relied on Cassandra's Java implementation details: 1. Tests for CREATE TRIGGER were commented out because testing them in Cassandra requires adding a Java class for the test. We're also not likely to ever add this feature to Scylla (Refs scylladb#2205). 2. Similarly, tests for CEP-11 (Pluggable memtable implementations) used internal Java APIs instead of CQL, and it also unlikely we'll ever implement it in a way compatible with Cassandra because of its Java reliance. 3. One test for data center names used internal Cassandra Java APIs, not CQL to create mock data centers and snitches. Signed-off-by: Nadav Har'El <nyh@scylladb.com>

This is a translation of Cassandra's CQL unit test source file validation/operations/CreateTest.java into our cql-pytest framework. The 15 tests did not reproduce any previously-unknown bug, but did provide additional reproducers for several known issues: Refs #6442: Always print all schema parameters (including default values) Refs #8001: Documented unit "µs" not supported for assigning a duration" type. Refs #8892: Add an option for default RF for new keyspaces. Refs #8948: Cassandra 3.11.10 uses "class" instead of "sstable_compression" for compression settings by default Unfortunately, I also had to comment out - and not translate - several tests which weren't real "CQL tests" (tests that use only the CQL driver), and instead relied on Cassandra's Java implementation details: 1. Tests for CREATE TRIGGER were commented out because testing them in Cassandra requires adding a Java class for the test. We're also not likely to ever add this feature to Scylla (Refs #2205). 2. Similarly, tests for CEP-11 (Pluggable memtable implementations) used internal Java APIs instead of CQL, and it also unlikely we'll ever implement it in a way compatible with Cassandra because of its Java reliance. 3. One test for data center names used internal Cassandra Java APIs, not CQL to create mock data centers and snitches. Signed-off-by: Nadav Har'El <nyh@scylladb.com> Closes #15791

agalue changed the title ~~Cassandra uses "class" instead of~~ Cassandra 3.11.10 uses "class" instead of "sstable_compression" for compression settings by default Jun 29, 2021

avikivity mentioned this issue Jun 29, 2021

compression: define 'class' attribute for compression and deprecate 'sstable_compression' #8949

Closed

slivne assigned avikivity Jul 6, 2021

slivne added this to the 4.7 milestone Jul 6, 2021

slivne added user request area/cassandra 3.x compatibility area/cassandra 4.x compatibility labels Jul 6, 2021

scylladb-promoter closed this as completed in 5571ef0 Jul 8, 2021

scylladb-promoter added the Backport candidate label Jul 8, 2021

avikivity added a commit that referenced this issue Jul 28, 2021

Revert "compression: define 'class' attribute for compression and dep…

331eb57

…recate 'sstable_compression'" This reverts commit 5571ef0. It causes rolling upgrade failures. Fixes #9055. Reopens #8948.

avikivity reopened this Jul 28, 2021

nyh removed the Backport candidate label Feb 6, 2022

slivne unassigned avikivity Apr 14, 2022

slivne modified the milestones: 5.0, 5.2 Apr 14, 2022

nyh mentioned this issue Oct 30, 2022

docs: Issue in page Data Definition #11858

Open

DoronArazii modified the milestones: 5.2, 5.x Nov 22, 2022

nyh mentioned this issue Jun 15, 2023

Always print all schema parameters (including default values) #6442

Open

nyh mentioned this issue Jun 15, 2023

test/cql-pytest: add tests reproducing bugs in compression configuration #14261

Closed

nyh mentioned this issue Oct 18, 2023

Support / Change create table syntax for compression options #2061

Open

nyh mentioned this issue Oct 22, 2023

cql-pytest: translate Cassandra's test for CREATE operations #15791

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Cassandra 3.11.10 uses "class" instead of "sstable_compression" for compression settings by default #8948

Cassandra 3.11.10 uses "class" instead of "sstable_compression" for compression settings by default #8948

agalue commented Jun 29, 2021 •

edited

slivne commented Jul 6, 2021

avikivity commented Jul 28, 2021

nyh commented Feb 6, 2022 •

edited

tzach commented Feb 6, 2022

jmaixl commented May 16, 2022

agalue commented May 25, 2022

nyh commented Jun 15, 2023 •

edited

nyh commented Oct 18, 2023

Cassandra 3.11.10 uses "class" instead of "sstable_compression" for compression settings by default #8948

Cassandra 3.11.10 uses "class" instead of "sstable_compression" for compression settings by default #8948

Comments

agalue commented Jun 29, 2021 • edited

slivne commented Jul 6, 2021

avikivity commented Jul 28, 2021

nyh commented Feb 6, 2022 • edited

tzach commented Feb 6, 2022

jmaixl commented May 16, 2022

agalue commented May 25, 2022

nyh commented Jun 15, 2023 • edited

nyh commented Oct 18, 2023

agalue commented Jun 29, 2021 •

edited

nyh commented Feb 6, 2022 •

edited

nyh commented Jun 15, 2023 •

edited