Skip to content

Commit

Permalink
Fix the correspondingMessagingVersion of SSTable format and improve T…
Browse files Browse the repository at this point in the history
…TL overflow tests coverage

Patch by Jacek Lewandowski; reviewed by Berenguer Blasi for CASSANDRA-19197
  • Loading branch information
jacek-lewandowski committed Dec 15, 2023
1 parent f9ebec9 commit 2fc2be5
Show file tree
Hide file tree
Showing 11 changed files with 432 additions and 225 deletions.
2 changes: 2 additions & 0 deletions CHANGES.txt
@@ -1,4 +1,5 @@
5.0-beta2
* Fix the correspondingMessagingVersion of SSTable format and improve TTL overflow tests coverage (CASSANDRA-19197)
* Fix resource cleanup after SAI query timeouts (CASSANDRA-19177)
* Suppress CVE-2023-6481 (CASSANDRA-19184)
Merged from 4.1:
Expand All @@ -7,6 +8,7 @@ Merged from 4.0:
Merged from 3.11:
Merged from 3.0:


5.0-beta1
* Fix SAI intersection queries (CASSANDRA-19011)
* Clone EndpointState before sending GossipShutdown message (CASSANDRA-19115)
Expand Down
7 changes: 7 additions & 0 deletions build.xml
Expand Up @@ -1237,12 +1237,16 @@
<macrodef name="testlist">
<attribute name="test.file.list"/>
<sequential>
<condition property="maxMemory" value="8G" else="4G">
<equals arg1="${test.classlistprefix}" arg2="distributed"/>
</condition>
<testmacrohelper inputdir="${test.dir}/${test.classlistprefix}" filelist="@{test.file.list}" exclude="**/*.java" timeout="${test.timeout}">
<jvmarg value="-Dlegacy-sstable-root=${test.data}/legacy-sstables"/>
<jvmarg value="-Dinvalid-legacy-sstable-root=${test.data}/invalid-legacy-sstables"/>
<jvmarg value="-Dcassandra.ring_delay_ms=1000"/>
<jvmarg value="-Dcassandra.tolerate_sstable_size=true"/>
<jvmarg value="-Dcassandra.skip_sync=true" />
<jvmarg value="-Xmx${maxMemory}"/>
</testmacrohelper>
</sequential>
</macrodef>
Expand Down Expand Up @@ -1764,6 +1768,7 @@
<jvmarg value="-Dcassandra.ring_delay_ms=10000"/>
<jvmarg value="-Dcassandra.tolerate_sstable_size=true"/>
<jvmarg value="-Dcassandra.skip_sync=true" />
<jvmarg value="-Xmx8G"/>
</testmacro>
</target>

Expand Down Expand Up @@ -1800,6 +1805,7 @@
<jvmarg value="-Dcassandra.ring_delay_ms=10000"/>
<jvmarg value="-Dcassandra.tolerate_sstable_size=true"/>
<jvmarg value="-Dcassandra.skip_sync=true" />
<jvmarg value="-Xmx8G"/>
</testmacro>
</target>

Expand All @@ -1815,6 +1821,7 @@
<jvmarg value="-Dcassandra.ring_delay_ms=10000"/>
<jvmarg value="-Dcassandra.tolerate_sstable_size=true"/>
<jvmarg value="-Dcassandra.skip_sync=true" />
<jvmarg value="-Xmx8G"/>
</testmacro>
</target>

Expand Down
32 changes: 16 additions & 16 deletions conf/cassandra.yaml
Expand Up @@ -2113,27 +2113,27 @@ drop_compact_storage_enabled: false

# This property indicates with what Cassandra major version the storage format will be compatible with.
#
# The chosen storage compatiblity mode will determine the versions of the written sstables, commitlogs, hints,
# etc. Those storage elements will use the higher minor versions of the major version that corresponds to the
# Cassandra version we want to stay compatible with. For example, if we want to stay compatible with Cassandra 4.0
# or 4.1, the value of this property should be 4, and that will make us use 'oa' sstables.
# The chosen storage compatibility mode will determine the versions of the written sstables, commitlogs, hints, etc.
# For example, if we're going to remain compatible with Cassandra 4.x, the value of this property should be 4, which
# will make us use sstables in the latest N version of the BIG format.
#
# This will also determine if certain features depending on newer formats are available. For example, extended TTLs
# up to 2106 depend on the sstable, commitlog, hints and messaging versions that were introduced by Cassandra 5.0,
# so that feature won't be available if this property is set to CASSANDRA_4. See upgrade guides for details. Currently
# the only supported major is CASSANDRA_4.
# This will also determine if certain features that depend on newer formats are available. For example, extended TTL
# (up to 2106) depends on the sstable, commit-log, hints, and messaging versions introduced by Cassandra 5.0, so that
# feature won't be available if this property is set to CASSANDRA_4. See the upgrade guide for more details.
#
# Possible values are in the StorageCompatibilityMode.java file accessible online. At the time of writing these are:
# Possible values are:
# - CASSANDRA_4: Stays compatible with the 4.x line in features, formats and component versions.
# - UPGRADING: The cluster monitors nodes versions during this interim stage. _This has a cost_ but ensures any new features,
# formats, versions, etc are enabled safely.
# - UPGRADING: The cluster monitors the version of each node during this interim stage. This has a cost but ensures
# all new features, formats, versions, etc. are enabled safely.
# - NONE: Start with all the new features and formats enabled.
#
# A typical upgrade would be:
# - Do a rolling upgrade starting all nodes in CASSANDRA_Y compatibility mode.
# - Once the new binary is rendered stable do a rolling restart with UPGRADING. The cluster will enable new features in a safe way
# until all nodes are started in UPGRADING, then all new features are enabled.
# - Do a rolling restart with all nodes starting with NONE. This sheds the extra cost of checking nodes versions and ensures
# a stable cluster. If a node from a previous version was started by accident we won't any longer toggle behaviors as when UPGRADING.
# - Do a rolling upgrade, starting all nodes in CASSANDRA_X compatibility mode.
# - Once the new binary is rendered stable, do a rolling restart with the UPGRADING mode. The cluster will keep new
# features disabled until all nodes are started in the UPGRADING mode; when that happens, new features controlled by
# the storage compatibility mode are enabled.
# - Do a rolling restart with all nodes starting with the NONE mode. This eliminates the cost of checking node versions
# and ensures stability. If Cassandra was started at the previous version by accident, a node with disabled
# compatibility mode would no longer toggle behaviors as when it was running in the UPGRADING mode.
#
storage_compatibility_mode: CASSANDRA_4
3 changes: 2 additions & 1 deletion ide/idea/workspace.xml
Expand Up @@ -200,8 +200,9 @@
-Dmigration-sstable-root=$PROJECT_DIR$/test/data/migration-sstables
-XX:ActiveProcessorCount=2
-XX:HeapDumpPath=build/test
-XX:MaxMetaspaceSize=1G
-XX:MaxMetaspaceSize=2G
-XX:SoftRefLRUPolicyMSPerMB=0
-Xmx4G
-ea" />
<option name="PARAMETERS" value="" />
<fork_mode value="class" />
Expand Down
Expand Up @@ -385,7 +385,9 @@ static class BigVersion extends Version
super(format, version);

isLatestVersion = version.compareTo(current_version) == 0;
correspondingMessagingVersion = MessagingService.VERSION_30;

// Note that, we probably forgot to change that to 40 for N version, and therefore we cannot do it now.
correspondingMessagingVersion = version.compareTo("oa") >= 0 ? MessagingService.VERSION_50 : MessagingService.VERSION_30;

hasCommitLogLowerBound = version.compareTo("mb") >= 0;
hasCommitLogIntervals = version.compareTo("mc") >= 0;
Expand Down
Expand Up @@ -302,7 +302,7 @@ static class BtiVersion extends Version
super(format, version);

isLatestVersion = version.compareTo(current_version) == 0;
correspondingMessagingVersion = MessagingService.VERSION_40;
correspondingMessagingVersion = MessagingService.VERSION_50;
}

@Override
Expand Down
@@ -0,0 +1,60 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package org.apache.cassandra.distributed.upgrade;

import org.junit.Test;

import static org.apache.cassandra.utils.StorageCompatibilityMode.NONE;
import static org.apache.cassandra.utils.StorageCompatibilityMode.UPGRADING;

public class MixedModeTTLOverflowAfterUpgradeTest extends MixedModeTTLOverflowUpgradeTestBase
{
@Test
public void testTTLOverflowAfterUpgrade() throws Throwable
{
testTTLOverflow((cluster, node) -> {
cluster.disableAutoCompaction(KEYSPACE);
if (node == 1) // only node1 is upgraded, and the cluster is in mixed versions mode
{
verify(Step.NODE1_40_NODE2_PREV, cluster, true);
}
else // both nodes have been upgraded, and the cluster isn't in mixed version mode anymore
{
verify(Step.NODE1_40_NODE2_40, cluster, true);

// We restart node1 with compatibility mode UPGRADING
restartNodeWithCompatibilityMode(cluster, 1, UPGRADING);
// since node2 is still in 4.0 compatibility mode, the limit should remain 2038
verify(Step.NODE1_UPGRADING_NODE2_40, cluster, true);

// We restart node2 in UPGRADING compatibility mode
restartNodeWithCompatibilityMode(cluster, 2, UPGRADING);
// Both nodes are in UPGRADING compatibility mode, so the limit should be 2106
verify(Step.NODE1_UPGRADING_NODE2_UPGRADING, cluster, false);

// We restart the cluster out of compatibility mode, so the limit should be 2106
restartNodeWithCompatibilityMode(cluster, 1, NONE);
verify(Step.NODE1_NONE_NODE2_UPGRADING, cluster, false);

restartNodeWithCompatibilityMode(cluster, 2, NONE);
verify(Step.NODE1_NONE_NODE2_NONE, cluster, false);
}
});
}
}
@@ -0,0 +1,61 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package org.apache.cassandra.distributed.upgrade;

import org.junit.Test;

import static org.apache.cassandra.utils.StorageCompatibilityMode.NONE;
import static org.apache.cassandra.utils.StorageCompatibilityMode.UPGRADING;

public class MixedModeTTLOverflowDuringUpgradeTest extends MixedModeTTLOverflowUpgradeTestBase
{
@Test
public void testTTLOverflowDuringUpgrade() throws Throwable
{
testTTLOverflow((cluster, node) -> {
cluster.disableAutoCompaction(KEYSPACE);
if (node == 1) // only node1 is upgraded, and the cluster is in mixed versions mode
{
verify(Step.NODE1_40_NODE2_PREV, cluster, true);

// We restart the upgraded node 1 with compatibility mode = UPGRADING
restartNodeWithCompatibilityMode(cluster, 1, UPGRADING);
// 2038 should still be the limit, because node2 is not upgraded yet
verify(Step.NODE1_UPGRADING_NODE2_PREV, cluster, true);
}
else // both nodes have been upgraded, and the cluster isn't in mixed version mode anymore
{
// Once we have completed the upgrade, 2038 should still be the limit because
// node2 is still in 4.x compatibility mode
verify(Step.NODE1_UPGRADING_NODE2_40, cluster, true);

// We restart the last upgraded node in UPGRADING compatibility mode
restartNodeWithCompatibilityMode(cluster, 2, UPGRADING);
// Both nodes are in UPGRADING compatibility mode, so the limit should be 2106
verify(Step.NODE1_UPGRADING_NODE2_UPGRADING, cluster, false);

// We restart get both nodes out of compatibility mode, so the limit should be 2106.
restartNodeWithCompatibilityMode(cluster, 1, NONE);
verify(Step.NODE1_NONE_NODE2_UPGRADING, cluster, false);
restartNodeWithCompatibilityMode(cluster, 2, NONE);
verify(Step.NODE1_NONE_NODE2_NONE, cluster, false);
}
});
}
}

0 comments on commit 2fc2be5

Please sign in to comment.