Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add CQL config to allow disabling TTL #3400

Merged
merged 2 commits into from
Dec 20, 2022
Merged

Add CQL config to allow disabling TTL #3400

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
950f18a
Amazon Keyspace: Add hint on full-scan
li-boxuan Dec 17, 2022
fe0e964
Add CQL config to allow disabling TTL
li-boxuan Dec 18, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/configs/janusgraph-cfg.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -413,6 +413,7 @@ CQL storage backend options
| storage.cql.session-leak-threshold | The maximum number of live sessions that are allowed to coexist in a given VM until the warning starts to log for every new session. If the value is less than or equal to 0, the feature is disabled: no warning will be issued. See DataStax Java Driver option `advanced.session-leak.threshold` for more information. | Integer | (no default value) | MASKABLE |
| storage.cql.session-name | Default name for the Cassandra session | String | JanusGraph Session | MASKABLE |
| storage.cql.speculative-retry | The speculative retry policy. One of: NONE, ALWAYS, <X>percentile, <N>ms. | String | (no default value) | FIXED |
| storage.cql.ttl-enabled | Whether TTL should be enabled or not. Must be turned off if the storage does not support TTL. Amazon Keyspace, for example, does not support TTL by default unless otherwise enabled. | Boolean | true | LOCAL |
| storage.cql.use-external-locking | True to prevent JanusGraph from using its own locking mechanism. Setting this to true eliminates redundant checks when using an external locking mechanism outside of JanusGraph. Be aware that when use-external-locking is set to true, that failure to employ a locking algorithm which locks all columns that participate in a transaction upfront and unlocks them when the transaction ends, will result in a 'read uncommitted' transaction isolation level guarantee. If set to true without an appropriate external locking mechanism in place side effects such as dirty/non-repeatable/phantom reads should be expected. | Boolean | false | MASKABLE |
| storage.cql.write-consistency-level | The consistency level of write operations against Cassandra | String | QUORUM | MASKABLE |

Expand Down
15 changes: 12 additions & 3 deletions docs/storage-backend/cassandra.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -250,7 +250,7 @@ as per this [doc](https://docs.datastax.com/en/astra/docs/manage-application-tok
To learn more about different configuration options of DataStax driver, please refer to the DataStax driver configuration
[documentation](https://docs.datastax.com/en/developer/java-driver/latest/manual/core/configuration/).

## Deploying on Amazon Keyspaces (Experimental)
## Deploying on Amazon Keyspaces

> Amazon Keyspaces (for Apache Cassandra) is a scalable, highly available, and managed
> Apache Cassandra–compatible database service. Amazon Keyspaces is serverless, so you
Expand Down Expand Up @@ -343,6 +343,13 @@ storage.cql.ssl.truststore.password=<your-trust-store-password>
# Thus, the below config must be turned off
graph.assign-timestamp=false

# We strongly recommend you to turn on this config. It will
# prohibit all full-scan attempts. This is because Amazon keyspace
# diverges from Apache Cassandra and might result in incomplete
# results when a full-scan is executed. See issue #3390 for more
# details
query.force-index=true

# Amazon Keyspaces only supports LOCAL QUORUM consistency
storage.cql.only-use-local-consistency-for-system-operations=true
storage.cql.read-consistency-level=LOCAL_QUORUM
Expand All @@ -352,10 +359,12 @@ log.tx.key-consistent=true

# Amazon Keyspaces does not have metadata available to clients
# Thus, we need to tell JanusGraph that metadata are disabled,
# and provide a hint of which partitioner AWS is using.
# and provide a hint of which partitioner AWS is using. Valid
# partitioner-names are: Murmur3Partitioner, RandomPartitioner,
# and DefaultPartitioner
storage.cql.metadata-schema-enabled=false
storage.cql.metadata-token-map-enabled=false
storage.cql.partitioner-name=DefaultPartitioner
storage.cql.partitioner-name=Murmur3Partitioner
```

Now you should be able to open the graph via gremlin console
Expand Down
9 changes: 9 additions & 0 deletions janusgraph-cql/src/main/java/org/janusgraph/diskstorage/cql/CQLConfigOptions.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -207,6 +207,15 @@ public interface CQLConfigOptions {
ConfigOption.Type.MASKABLE,
Long.class);

ConfigOption<Boolean> TTL_ENABLED = new ConfigOption<>(
CQL_NS,
"ttl-enabled",
"Whether TTL should be enabled or not. Must be turned off if the storage does not support TTL. " +
"Amazon Keyspace, for example, does not support TTL by default unless otherwise enabled.",
ConfigOption.Type.LOCAL,
Boolean.class,
true);

ConfigOption<String> PARTITIONER_NAME = new ConfigOption<>(
CQL_NS,
"partitioner-name",
Expand Down
8 changes: 6 additions & 2 deletions ...aph-cql/src/main/java/org/janusgraph/diskstorage/cql/builder/CQLStoreFeaturesBuilder.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@
import static org.janusgraph.diskstorage.cql.CQLConfigOptions.ONLY_USE_LOCAL_CONSISTENCY_FOR_SYSTEM_OPERATIONS;
import static org.janusgraph.diskstorage.cql.CQLConfigOptions.PARTITIONER_NAME;
import static org.janusgraph.diskstorage.cql.CQLConfigOptions.READ_CONSISTENCY;
import static org.janusgraph.diskstorage.cql.CQLConfigOptions.TTL_ENABLED;
import static org.janusgraph.diskstorage.cql.CQLConfigOptions.USE_EXTERNAL_LOCKING;
import static org.janusgraph.diskstorage.cql.CQLConfigOptions.WRITE_CONSISTENCY;
import static org.janusgraph.graphdb.configuration.GraphDatabaseConfiguration.METRICS_PREFIX;
Expand Down Expand Up @@ -62,6 +63,10 @@ public CQLStoreFeaturesWrapper build(final CqlSession session, final Configurati
fb.optimisticLocking(true);
fb.multiQuery(false);

if (!configuration.get(TTL_ENABLED)) {
fb.cellTTL(false).storeTTL(false);
porunov marked this conversation as resolved.
Show resolved Hide resolved
}

String partitioner = null;
if (configuration.has(PARTITIONER_NAME)) {
partitioner = getShortPartitionerName(configuration.get(PARTITIONER_NAME));
Expand All @@ -79,8 +84,7 @@ public CQLStoreFeaturesWrapper build(final CqlSession session, final Configurati
"server, please check %s and %s options", PARTITIONER_NAME.getName(), METADATA_TOKEN_MAP_ENABLED.getName()));
}
switch (partitioner) {
case "DefaultPartitioner": // Amazon managed KeySpace uses com.amazonaws.cassandra.DefaultPartitioner
fb.timestamps(false);
case "DefaultPartitioner": // Amazon managed KeySpace supports com.amazonaws.cassandra.DefaultPartitioner
case "RandomPartitioner":
case "Murmur3Partitioner": {
fb.keyOrdered(false).orderedScan(false).unorderedScan(true);
Expand Down