Skip to content

Commit

Permalink
Add ScyllaDB driver support
Browse files Browse the repository at this point in the history
- Add ScyllaDB driver support via `janusgraph-scylla`
- `janusgraph-cql` contains tests for Cassandra3, Cassandra4, ScyllaDB. `janusgraph-scylla` contains tests for ScyllaDB only.
- Fix ScyllaDB tests (configs were not applied to ScyllaDB tests previously)

Fixes #1778
Related to #2451
Fixes #2505

Signed-off-by: Oleksandr Porunov <alexandr.porunov@gmail.com>
  • Loading branch information
porunov committed Feb 15, 2023
1 parent b674e2e commit 7179cbb
Show file tree
Hide file tree
Showing 21 changed files with 1,291 additions and 73 deletions.
20 changes: 9 additions & 11 deletions .github/workflows/ci-backend-cql-dummy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -74,11 +74,10 @@ jobs:
args: "-Pscylladb -Dtest=\"**/graphdb/cql/*\""
name: scylladb-graphdb
java: 8
# FIXME: this takes forever to run
# - module: cql
# args: "-Pscylladb -Dtest=\"**/hadoop/*\""
# name: scylladb-hadoop
# java: 8
- module: cql
args: "-Pscylladb -Dtest=\"**/hadoop/*\""
name: scylladb-hadoop
java: 8
- module: cql
args: "-Pscylladb -Dtest=\"**/core/cql/*\""
name: scylladb-core
Expand Down Expand Up @@ -138,12 +137,11 @@ jobs:
name: scylladb-graphdb
install-args: "-Pjava-11"
java: 11
# FIXME: this takes forever to run
# - module: cql
# args: "-Pscylladb -Dtest=\"**/hadoop/*\""
# name: scylladb-hadoop
# install-args: "-Pjava-11"
# java: 11
- module: cql
args: "-Pscylladb -Dtest=\"**/hadoop/*\""
name: scylladb-hadoop
install-args: "-Pjava-11"
java: 11
- module: cql
args: "-Pscylladb -Dtest=\"**/core/cql/*\""
name: scylladb-core
Expand Down
20 changes: 9 additions & 11 deletions .github/workflows/ci-backend-cql.yml
Original file line number Diff line number Diff line change
Expand Up @@ -107,11 +107,10 @@ jobs:
args: "-Pscylladb -Dtest=\"**/graphdb/cql/*\""
name: scylladb-graphdb
java: 8
# FIXME: this takes forever to run
# - module: cql
# args: "-Pscylladb -Dtest=\"**/hadoop/*\""
# name: scylladb-hadoop
# java: 8
- module: cql
args: "-Pscylladb -Dtest=\"**/hadoop/*\""
name: scylladb-hadoop
java: 8
- module: cql
args: "-Pscylladb -Dtest=\"**/core/cql/*\""
name: scylladb-core
Expand Down Expand Up @@ -171,12 +170,11 @@ jobs:
name: scylladb-graphdb
install-args: "-Pjava-11"
java: 11
# FIXME: this takes forever to run
# - module: cql
# args: "-Pscylladb -Dtest=\"**/hadoop/*\""
# name: scylladb-hadoop
# install-args: "-Pjava-11"
# java: 11
- module: cql
args: "-Pscylladb -Dtest=\"**/hadoop/*\""
name: scylladb-hadoop
install-args: "-Pjava-11"
java: 11
- module: cql
args: "-Pscylladb -Dtest=\"**/core/cql/*\""
name: scylladb-core
Expand Down
121 changes: 121 additions & 0 deletions .github/workflows/ci-backend-scylla-dummy.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,121 @@
# Copyright 2023 JanusGraph Authors
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

name: CI Backend ScyllaDB

on:
pull_request:
paths-ignore:
- 'docs/**'
- '.github/workflows/ci-docs.yml'
- '.github/ISSUE_TEMPLATE/**'
- 'requirements.txt'
- 'docs.Dockerfile'
- '*.md'
push:
paths-ignore:
- 'docs/**'
- '.github/workflows/ci-docs.yml'
- '.github/ISSUE_TEMPLATE/**'
- 'requirements.txt'
- 'docs.Dockerfile'
- '*.md'
branches-ignore:
- 'dependabot/**'

env:
BUILD_MAVEN_OPTS: "-DskipTests=true --batch-mode --also-make"
VERIFY_MAVEN_OPTS: "-Pcoverage"

jobs:
build-all:
runs-on: ubuntu-20.04
steps:
- uses: actions/checkout@v3
- uses: actions/cache@v3
with:
path: ~/.m2/repository
key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}
restore-keys: |
${{ runner.os }}-maven-
- uses: actions/setup-java@v3
with:
java-version: 8
distribution: zulu
- run: mvn clean install --projects janusgraph-all -Pjanusgraph-cache -Dmaven.javadoc.skip=true ${{ env.BUILD_MAVEN_OPTS }}
- run: mvn verify --projects janusgraph-all -Pjanusgraph-cache ${{ env.VERIFY_MAVEN_OPTS }}

tests:
runs-on: ubuntu-20.04
needs: build-all
strategy:
fail-fast: false
matrix:
include:
- module: scylla
args: "-Pscylla-murmur -Dtest=\"**/diskstorage/cql/*\""
name: murmur-diskstorage
java: 8
- module: scylla
args: "-Pscylla-murmur -Dtest=\"**/graphdb/cql/*\""
name: murmur-graphdb
java: 8
- module: scylla
args: "-Pscylla-murmur -Dtest=\"**/hadoop/*\""
name: murmur-hadoop
java: 8
- module: scylla
args: "-Pscylla-murmur -Dtest=\"**/core/cql/*\""
name: murmur-core
java: 8
- module: scylla
args: "-Pscylla-murmur-ssl -Dtest=\"**/diskstorage/cql/CQLStoreTest.java\""
name: murmur-ssl
java: 8
- module: scylla
args: "-Pscylla-murmur-client-auth -Dtest=\"**/diskstorage/cql/CQLStoreTest.java\""
name: murmur-client-auth
java: 8
- module: scylla
args: "-Pscylla-murmur -Dtest=\"**/diskstorage/cql/*\""
name: murmur-diskstorage
install-args: "-Pjava-11"
java: 11
- module: scylla
args: "-Pscylla-murmur -Dtest=\"**/graphdb/cql/*\""
name: murmur-graphdb
install-args: "-Pjava-11"
java: 11
- module: scylla
args: "-Pscylla-murmur -Dtest=\"**/hadoop/*\""
name: murmur-hadoop
install-args: "-Pjava-11"
java: 11
- module: scylla
args: "-Pscylla-murmur -Dtest=\"**/core/cql/*\""
name: murmur-core
install-args: "-Pjava-11"
java: 11
- module: scylla
args: "-Pscylla-murmur-ssl -Dtest=\"**/diskstorage/cql/CQLStoreTest.java\""
name: murmur-ssl
install-args: "-Pjava-11"
java: 11
- module: scylla
args: "-Pscylla-murmur-client-auth -Dtest=\"**/diskstorage/cql/CQLStoreTest.java\""
name: murmur-client-auth
install-args: "-Pjava-11"
java: 11
steps:
- run: 'echo "No build required"'
146 changes: 146 additions & 0 deletions .github/workflows/ci-backend-scylla.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,146 @@
# Copyright 2023 JanusGraph Authors
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

name: CI Backend ScyllaDB

on:
pull_request:
paths-ignore:
- 'docs/**'
- '.github/workflows/ci-docs.yml'
- '.github/ISSUE_TEMPLATE/**'
- 'requirements.txt'
- 'docs.Dockerfile'
- '*.md'
push:
paths-ignore:
- 'docs/**'
- '.github/workflows/ci-docs.yml'
- '.github/ISSUE_TEMPLATE/**'
- 'requirements.txt'
- 'docs.Dockerfile'
- '*.md'
branches-ignore:
- 'dependabot/**'

env:
BUILD_MAVEN_OPTS: "-DskipTests=true --batch-mode --also-make"
VERIFY_MAVEN_OPTS: "-Pcoverage"

jobs:
build-all:
runs-on: ubuntu-20.04
steps:
- uses: actions/checkout@v3
- uses: actions/cache@v3
with:
path: ~/.m2/repository
key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}
restore-keys: |
${{ runner.os }}-maven-
- uses: actions/setup-java@v3
with:
java-version: 8
distribution: zulu
- run: mvn clean install --projects janusgraph-all -Pjanusgraph-cache -Dmaven.javadoc.skip=true ${{ env.BUILD_MAVEN_OPTS }}
- run: mvn verify --projects janusgraph-all -Pjanusgraph-cache ${{ env.VERIFY_MAVEN_OPTS }}

tests:
runs-on: ubuntu-20.04
needs: build-all
strategy:
fail-fast: false
matrix:
include:
- module: scylla
args: "-Pscylla-murmur -Dtest=\"**/diskstorage/cql/*\""
name: murmur-diskstorage
java: 8
- module: scylla
args: "-Pscylla-murmur -Dtest=\"**/graphdb/cql/*\""
name: murmur-graphdb
java: 8
- module: scylla
args: "-Pscylla-murmur -Dtest=\"**/hadoop/*\""
name: murmur-hadoop
java: 8
- module: scylla
args: "-Pscylla-murmur -Dtest=\"**/core/cql/*\""
name: murmur-core
java: 8
# FIXME: this test is failing due to problems with ScyllaDB startup for Testcontainers. Issue: https://github.com/JanusGraph/janusgraph/issues/3595
# - module: scylla
# args: "-Pscylla-murmur-ssl -Dtest=\"**/diskstorage/cql/CQLStoreTest.java\""
# name: murmur-ssl
# java: 8
# FIXME: this test is failing due to problems with ScyllaDB startup for Testcontainers. Issue: https://github.com/JanusGraph/janusgraph/issues/3595
# - module: scylla
# args: "-Pscylla-murmur-client-auth -Dtest=\"**/diskstorage/cql/CQLStoreTest.java\""
# name: murmur-client-auth
# java: 8
- module: scylla
args: "-Pscylla-murmur -Dtest=\"**/diskstorage/cql/*\""
name: murmur-diskstorage
install-args: "-Pjava-11"
java: 11
- module: scylla
args: "-Pscylla-murmur -Dtest=\"**/graphdb/cql/*\""
name: murmur-graphdb
install-args: "-Pjava-11"
java: 11
- module: scylla
args: "-Pscylla-murmur -Dtest=\"**/hadoop/*\""
name: murmur-hadoop
install-args: "-Pjava-11"
java: 11
- module: scylla
args: "-Pscylla-murmur -Dtest=\"**/core/cql/*\""
name: murmur-core
install-args: "-Pjava-11"
java: 11
# FIXME: this test is failing due to problems with ScyllaDB startup for Testcontainers. Issue: https://github.com/JanusGraph/janusgraph/issues/3595
# - module: scylla
# args: "-Pscylla-murmur-ssl -Dtest=\"**/diskstorage/cql/CQLStoreTest.java\""
# name: murmur-ssl
# install-args: "-Pjava-11"
# java: 11
# FIXME: this test is failing due to problems with ScyllaDB startup for Testcontainers. Issue: https://github.com/JanusGraph/janusgraph/issues/3595
# - module: scylla
# args: "-Pscylla-murmur-client-auth -Dtest=\"**/diskstorage/cql/CQLStoreTest.java\""
# name: murmur-client-auth
# install-args: "-Pjava-11"
# java: 11
steps:
- uses: actions/checkout@v3
with:
fetch-depth: 2
- uses: actions/cache@v3
with:
path: ~/.m2/repository
key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}
restore-keys: |
${{ runner.os }}-maven-
- uses: actions/setup-java@v3
with:
java-version: ${{ matrix.java }}
distribution: zulu
- run: mvn clean install --projects janusgraph-${{ matrix.module }} ${{ env.BUILD_MAVEN_OPTS }} ${{ matrix.install-args }}
- run: mvn verify --projects janusgraph-${{ matrix.module }} ${{ env.VERIFY_MAVEN_OPTS }} ${{ matrix.install-args }} ${{ matrix.args }}
- uses: actions/upload-artifact@v3
with:
name: jacoco-reports
path: target/jacoco-combined.exec
- uses: codecov/codecov-action@v3
with:
name: codecov-cql-${{ matrix.name }}-java-${{ matrix.java }}
14 changes: 7 additions & 7 deletions TESTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -139,13 +139,13 @@ mvn clean install -pl janusgraph-cql -Pscylladb

System properties to configure CQL test executions:

| Property | Description | Default value |
| -------- | ----------- | ------------- |
| `cassandra.docker.image` | Docker image to pull and run. | `cassandra` |
| `cassandra.docker.version` | Docker image tag to pull and run | `3.11.10` |
| `cassandra.docker.partitioner` | Set the cassandra partitioner. Supported partitioner are `murmur`, or `byteordered`| `murmur` |
| `cassandra.docker.useSSL` | Activate SSL **Note: This property currently only works with the partitioner set to `murmur`.** | `false` |
| `cassandra.docker.useDefaultConfigFromImage` | If set to `false` default configs of the image are used. **Note: `cassandra.docker.partitioner` and `cassandra.docker.useSSL` are ignored.** | `false` |
| Property | Description | Default value |
|----------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------|---------------|
| `cassandra.docker.image` | Docker image to pull and run. | `cassandra` |
| `cassandra.docker.version` | Docker image tag to pull and run | `3.11.10` |
| `cassandra.docker.partitioner` | Set the cassandra partitioner. Supported partitioner are `murmur`, or `byteordered` | `murmur` |
| `cassandra.docker.useSSL` | Activate SSL **Note: This property currently only works with the partitioner set to `murmur`.** | `false` |
| `cassandra.docker.useDefaultConfigFromImage` | If set to `true` default configs of the image are used. **Note: `cassandra.docker.partitioner` and `cassandra.docker.useSSL` are ignored.** | `false` |

The following examples show possible configuration combinations.

Expand Down
Loading

0 comments on commit 7179cbb

Please sign in to comment.