Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

4084: adds Couchbase as JanusGraph backend #4086

Open
wants to merge 6 commits into
base: master
Choose a base branch
from
Open

4084: adds Couchbase as JanusGraph backend #4086

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
6ec1d8c
4084: adds Couchbase as JanusGraph backend
chedim Oct 31, 2023
bbd8f70
4084: Add Couchbase Backend
chedim Nov 8, 2023
1367f47
fixes automatically replaced folder name in docs
chedim Dec 8, 2023
c4e454c
Merge branch 'JanusGraph:master' into add-couchbase-backend
chedim Feb 12, 2024
9bbed63
Adds couchbase to janusgraph-all and janusgraph-dist builds
chedim Feb 23, 2024
0123c4c
Adds couchbase to janusgraph-all and janusgraph-dist builds
chedim Feb 23, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
33 changes: 32 additions & 1 deletion docs/configs/janusgraph-cfg.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -144,6 +144,20 @@ Configuration for Elasticsearch fixed circle processor which is used for BKD Geo
| index.[X].bkd-circle-processor.fixed.bounding-box-fallback | Allows to return bounding box for the circle which cannot be converted to proper shape with the specified error distance. In case `false` is set for this configuration an exception will be thrown whenever circle cannot be converted to another shape following error distance. | Boolean | true | MASKABLE |
| index.[X].bkd-circle-processor.fixed.error-distance | The difference between the resulting inscribed distance from center to side and the circle’s radius. Specified in meters. | Double | 10.0 | MASKABLE |

### index.[X].cb
Couchbase index configuration


| Name | Description | Datatype | Default Value | Mutability |
| ---- | ---- | ---- | ---- | ---- |
| index.[X].cb.cluster-connect-bucket | Bucket in the Couchbase cluster | String | default | LOCAL |
| index.[X].cb.cluster-connect-password | Password to the Couchbase cluster | String | password | LOCAL |
| index.[X].cb.cluster-connect-string | Connect string to the Couchbase cluster | String | couchbase://localhost | LOCAL |
| index.[X].cb.cluster-connect-username | Username to the Couchbase cluster | String | Administrator | LOCAL |
| index.[X].cb.cluster-default-fuziness | Default FTS fuziness | Integer | 2 | LOCAL |
| index.[X].cb.cluster-default-scope | Default Scope | String | _default | LOCAL |
| index.[X].cb.version | The version of the Couchbase cluster. | Integer | 703 | LOCAL |

### index.[X].elasticsearch
Elasticsearch index configuration

Expand Down Expand Up @@ -417,6 +431,23 @@ BerkeleyDB JE configuration options
| storage.berkeleyje.lock-mode | The BDB record lock mode used for read operations | String | LockMode.DEFAULT | MASKABLE |
| storage.berkeleyje.shared-cache | If true, the shared cache is used for all graph instances | Boolean | true | MASKABLE |

### storage.cb
Couchbase storage backend options


| Name | Description | Datatype | Default Value | Mutability |
| ---- | ---- | ---- | ---- | ---- |
| storage.cb.cluster-connect-bucket | Bucket in the Couchbase cluster | String | default | LOCAL |
| storage.cb.cluster-connect-password | Password to the Couchbase cluster | String | password | LOCAL |
| storage.cb.cluster-connect-string | Connect string to the Couchbase cluster | String | couchbase://localhost | LOCAL |
| storage.cb.cluster-connect-username | Username to the Couchbase cluster | String | Administrator | LOCAL |
| storage.cb.cluster-default-collection | Default Collection | String | _default | LOCAL |
| storage.cb.cluster-default-scope | Default Scope | String | _default | LOCAL |
| storage.cb.cluster-parallelism | Number of parallel threads in which multi-slice queries are executed | String | 1 | LOCAL |
| storage.cb.get-range-mode | The mode of executing CB getRange, either `iterator` or `list` | String | list | LOCAL |
| storage.cb.isolation-level | Options are serializable, read_committed_no_write, read_committed_with_write | String | serializable | LOCAL |
| storage.cb.version | The version of the Couchbase cluster. | Integer | 703 | LOCAL |

### storage.cql
CQL storage backend options

Expand Down Expand Up @@ -583,7 +614,7 @@ HBase storage options
| storage.hbase.short-cf-names | Whether to shorten the names of JanusGraph's column families to one-character mnemonics to conserve storage space | Boolean | true | FIXED |
| storage.hbase.skip-schema-check | Assume that JanusGraph's HBase table and column families already exist. When this is true, JanusGraph will not check for the existence of its table/CFs, nor will it attempt to create them under any circumstances. This is useful when running JanusGraph without HBase admin privileges. | Boolean | false | MASKABLE |
| storage.hbase.snapshot-name | The name of an existing HBase snapshot to be used by HBaseSnapshotInputFormat | String | janusgraph-snapshot | LOCAL |
| storage.hbase.snapshot-restore-dir | The temporary directory to be used by HBaseSnapshotInputFormat to restore a snapshot. This directory should be on the same File System as the HBase root dir. | String | /tmp | LOCAL |
| storage.hbase.snapshot-restore-dir | The temporary directory to be used by HBaseSnapshotInputFormat to restore a snapshot. This directory should be on the same File System as the HBase root dir. | String | /var/folders/9r/nv_glvys3kgcw706nvfkvgf40000gr/T/ | LOCAL |
chedim marked this conversation as resolved.
Show resolved Hide resolved
| storage.hbase.table | The name of the table JanusGraph will use. When storage.hbase.skip-schema-check is false, JanusGraph will automatically create this table if it does not already exist. If this configuration option is not provided but graph.graphname is, the table will be set to that value. | String | janusgraph | LOCAL |

### storage.lock
Expand Down
5 changes: 5 additions & 0 deletions janusgraph-all/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,11 @@
<artifactId>janusgraph-mixed-index-utils</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>org.janusgraph</groupId>
<artifactId>janusgraph-couchbase</artifactId>
<version>${project.version}</version>
</dependency>

<!-- Jackson 2.x -->
<dependency>
Expand Down
2 changes: 1 addition & 1 deletion janusgraph-core/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -129,7 +129,7 @@
<configuration>
<archive>
<manifestEntries>
<Gremlin-Plugin-Dependencies>org.janusgraph:janusgraph-cql:${project.version};org.janusgraph:janusgraph-hadoop:${project.version}</Gremlin-Plugin-Dependencies>
<Gremlin-Plugin-Dependencies>org.janusgraph:janusgraph-cql:${project.version};org.janusgraph:janusgraph-hadoop:${project.version};org.janusgraph:janusgraph-couchbase:${project.version}</Gremlin-Plugin-Dependencies>
</manifestEntries>
</archive>
</configuration>
Expand Down
2 changes: 2 additions & 0 deletions janusgraph-core/src/main/java/org/janusgraph/core/util/ReflectiveConfigOptionLoader.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -122,6 +122,8 @@ public void loadStandard(Class<?> caller) {
*/
List<String> classnames = Collections.unmodifiableList(Arrays.asList(
"org.janusgraph.diskstorage.hbase.HBaseStoreManager",
"org.janusgraph.diskstorage.couchbase.CouchbaseConfigOptions",
"org.janusgraph.diskstorage.couchbase.CouchbaseIndexConfigOptions",
"org.janusgraph.diskstorage.cql.CQLConfigOptions",
"org.janusgraph.diskstorage.es.ElasticSearchIndex",
"org.janusgraph.diskstorage.solr.SolrIndex",
Expand Down
282 changes: 282 additions & 0 deletions janusgraph-couchbase/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,282 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.janusgraph</groupId>
<artifactId>janusgraph</artifactId>
<version>1.1.0-SNAPSHOT</version>
<relativePath>../pom.xml</relativePath>
</parent>

<artifactId>janusgraph-couchbase</artifactId>
<name>JanusGraph-Couchbase: Couchbase Backend for JanusGraph</name>

<properties>
<maven.compiler.plugin.version>3.8.1</maven.compiler.plugin.version>
<maven.compiler.source>1.8</maven.compiler.source>
<maven.compiler.target>1.8</maven.compiler.target>
<jdk.version>1.8</jdk.version>
<couchbase.core-io.version>2.3.4</couchbase.core-io.version>
<couchbase.java-client.version>3.3.4</couchbase.java-client.version>
<lucene-parser.version>8.11.2</lucene-parser.version>
<jackson-databind.version>2.10.3</jackson-databind.version>
<dependency.plugin.version>3.2.0</dependency.plugin.version>
<test.skip.default>false</test.skip.default>
<test.skip.tp>true</test.skip.tp>
<testcontainers.version>1.15.3</testcontainers.version>
</properties>

<developers>
<developer>
<name>Jagadesh Munta</name>
<email>jagadesh.munta@couchbase.com</email>
</developer>
<developer>
<name>Denis Souza Rosa</name>
<email>denis.rosa@couchbase.com</email>
</developer>
<developer>
<name>Dmitrii Chechetkin</name>
<email>dmitrii.chechetkin@couchbase.com</email>
</developer>
</developers>

<licenses>
<license>
<name>The Apache Software License, Version 2.0</name>
<url>http://www.apache.org/licenses/LICENSE-2.0.txt</url>
</license>
</licenses>

<dependencies>
<dependency>
<groupId>org.janusgraph</groupId>
<artifactId>janusgraph-core</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>org.janusgraph</groupId>
<artifactId>janusgraph-server</artifactId>
<version>${project.version}</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>ch.qos.logback</groupId>
<artifactId>logback-classic</artifactId>
<version>0.9.24</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.janusgraph</groupId>
<artifactId>janusgraph-backend-testutils</artifactId>
<version>${project.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.janusgraph</groupId>
<artifactId>janusgraph-test</artifactId>
<version>${project.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.couchbase.client</groupId>
<artifactId>core-io</artifactId>
<version>${couchbase.core-io.version}</version>
<exclusions>
<exclusion>
<groupId>org.reactivestreams</groupId>
<artifactId>reactive-streams</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.reactivestreams</groupId>
<artifactId>reactive-streams</artifactId>
<version>1.0.3</version>
</dependency>
<dependency>
<groupId>com.couchbase.client</groupId>
<artifactId>java-client</artifactId>
<version>${couchbase.java-client.version}</version>
</dependency>
<dependency>
<groupId>org.apache.lucene</groupId>
<artifactId>lucene-queryparser</artifactId>
<version>${lucene-parser.version}</version>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
<version>${jackson-databind.version}</version>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>4.5.6</version>
</dependency>
<dependency>
<groupId>io.reactivex</groupId>
<artifactId>rxjava</artifactId>
<version>1.3.8</version>
</dependency>
<dependency>
<groupId>org.testcontainers</groupId>
<artifactId>testcontainers</artifactId>
<version>${testcontainers.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.testcontainers</groupId>
<artifactId>junit-jupiter</artifactId>
<version>${testcontainers.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-inline</artifactId>
<version>3.12.4</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.powermock</groupId>
<artifactId>powermock-core</artifactId>
<version>2.0.9</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.powermock</groupId>
<artifactId>powermock-api-mockito2</artifactId>
<version>2.0.9</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.powermock</groupId>
<artifactId>powermock-module-junit4</artifactId>
<version>2.0.9</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.powermock</groupId>
<artifactId>powermock-reflect</artifactId>
<version>2.0.9</version>
<scope>test</scope>
</dependency>
</dependencies>

<build>
<directory>${basedir}/target</directory>
<testResources>
<testResource>
<directory>${basedir}/src/test/resources</directory>
</testResource>
</testResources>
<plugins>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<version>${maven.compiler.plugin.version}</version>
<configuration>
<source>${jdk.version}</source>
<target>${jdk.version}</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<version>${dependency.plugin.version}</version>
<executions>
<execution>
<id>copy-dependencies</id>
<phase>prepare-package</phase>
<goals>
<goal>copy-dependencies</goal>
</goals>
<configuration>
<outputDirectory>${project.build.directory}/lib</outputDirectory>
<includeScope>compile</includeScope>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.22.1</version>
<configuration>
<parallel>none</parallel>
<runOrder>alphabetical</runOrder>
<failIfNoTests>false</failIfNoTests>
<excludes>
<exclude>**/*PerformanceTest.java</exclude>
<exclude>**/*ConcurrentTest.java</exclude>
<exclude>**/*Groovy*Test.java</exclude>
<exclude>**/*ComputerTest.java</exclude>
<exclude>**/*ProcessTest.java</exclude>
<exclude>**/*ProcessPerformanceTest.java</exclude>
<exclude>**/*StructureTest.java</exclude>
</excludes>
<skip>${test.skip.default}</skip>
<!-- Use log4j.properties from module test resources -->
<systemProperties>
<property>
<name>log4j.configuration</name>
<value>file:${project.build.directory}/test-classes/log4j.properties</value>
</property>
</systemProperties>
</configuration>
<executions>
<execution>
<id>tinkerpop-test</id>
<goals>
<goal>test</goal>
</goals>
<phase>test</phase>
<configuration combine.self="override">
<reuseForks>false</reuseForks>
<forkCount>1</forkCount>
<parallel>none</parallel>
<threadCount>1</threadCount>
<perCoreThreadCount>false</perCoreThreadCount>
<includes>
<include>**/*Groovy*Test.java</include>
<include>**/*ComputerTest.java</include>
<include>**/*ProcessTest.java</include>
<include>**/*ProcessPerformanceTest.java</include>
<include>**/*StructureTest.java</include>
</includes>
<runOrder>alphabetical</runOrder>
<skipTests>${test.skip.tp}</skipTests>
<systemPropertyVariables>
<build.dir>${project.build.directory}</build.dir>
<log4j.configuration>file:${project.build.directory}/test-classes/log4j.properties</log4j.configuration>
<is.testing>true</is.testing>
</systemPropertyVariables>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<source>${jdk.version}</source>
<target>${jdk.version}</target>
</configuration>
</plugin>
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<executions>
<execution>
<phase>package</phase>
<goals><goal>single</goal></goals>
<configuration>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>