Skip to content

Commit

Permalink
Add support for graph binary
Browse files Browse the repository at this point in the history
Fixes #1564

Signed-off-by: Jan Jansen <jan.jansen@gdata.de>
  • Loading branch information
farodin91 committed Apr 16, 2020
1 parent db8bcb1 commit 1cb4b6e
Show file tree
Hide file tree
Showing 24 changed files with 869 additions and 165 deletions.
61 changes: 61 additions & 0 deletions docs/changelog.md
Expand Up @@ -24,9 +24,70 @@ use the latest versions of the software.
| 0.3.z | 2 | 1.2.z, 2.0.z, 2.1.z, 2.2.z, 3.0.z, 3.11.z | 1.0.z, 1.1.z, 1.2.z, 1.3.z, 1.4.z | 1.0.0, 1.1.0, 1.1.2, 1.2.0, 1.3.0, 1.4.0 | 1.5-1.7.z, 2.3-2.4.z, 5.y, 6.y | 5.2-5.5.z, 6.2-6.6.z, 7.y | 3.3.z | 2.2.z | 2.11.z |
| 0.4.z | 2 | 2.1.z, 2.2.z, 3.0.z, 3.11.z | 1.2.z, 1.3.z, 1.4.z, 2.1.z | N/A | 5.y, 6.y | 7.y | 3.4.z | 2.2.z | 2.11.z |
| 0.5.z | 2 | 2.1.z, 2.2.z, 3.0.z, 3.11.z | 1.2.z, 1.3.z, 1.4.z, 2.1.z | 1.3.0, 1.4.0, 1.5.z, 1.6.z, 1.7.z, 1.8.z, 1.9.z, 1.10.z, 1.11.z, 1.14.z | 6.y, 7.y | 7.y | 3.4.z | 2.2.z | 2.11.z |
| 0.6.z | 2 | 2.1.z, 2.2.z, 3.0.z, 3.11.z | 1.2.z, 1.3.z, 1.4.z, 2.1.z | 1.3.0, 1.4.0, 1.5.z, 1.6.z, 1.7.z, 1.8.z, 1.9.z, 1.10.z, 1.11.z, 1.14.z | 6.y, 7.y | 7.y | 3.4.z | 2.2.z | 2.11.z |

## Release Notes

### Version 0.6.0 (Release Date: X, 2020)

```xml tab='Maven'
<dependency>
<groupId>org.janusgraph</groupId>
<artifactId>janusgraph-core</artifactId>
<version>0.6.0</version>
</dependency>
```

```groovy tab='Gradle'
compile "org.janusgraph:janusgraph-core:0.6.0"
```

**Tested Compatibility:**

* Apache Cassandra 2.2.10, 3.0.14, 3.11.0
* Apache HBase 1.2.6, 1.3.1, 1.4.10, 2.1.5
* Google Bigtable 1.3.0, 1.4.0, 1.5.0, 1.6.0, 1.7.0, 1.8.0, 1.9.0, 1.10.0, 1.11.0, 1.14.0
* Oracle BerkeleyJE 7.5.11
* Elasticsearch 6.0.1, 6.6.0, 7.6.1
* Apache Lucene 7.0.0
* Apache Solr 7.0.0
* Apache TinkerPop 3.4.6
* Java 1.8

For more information on features and bug fixes in 0.6.0, see the GitHub milestone:

- <https://github.com/JanusGraph/janusgraph/milestone/17?closed=1>

#### Upgrade Instructions

##### Serialization of JanusGraph predicates has changed

The serialization of JanusGraph predicates has changed in this version for both
GraphSON and Gryo. It is therefore necessary to update both the client and
the server to this version in parallel as the server will not be able to
deserialize a JanusGraph predicate that was serialized by a client prior
to version 0.6.0 once it was updated to version 0.6.0.

##### GraphBinary is now supported

[GraphBinary](http://tinkerpop.apache.org/docs/current/dev/io/#graphbinary) is a
new binary serialization format from TinkerPop that supersedes Gryo and it will
eventually also replace GraphSON. GraphBinary is language independent and has a
low serialization overhead which results in an improved performance.

If you want to use GraphBinary, you have to add following to the `gremlin-server.yaml`
after the keyword `serializers`. This will add the support on the server site.

```yaml
- { className: org.apache.tinkerpop.gremlin.driver.ser.GraphBinaryMessageSerializerV1,
config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
- { className: org.apache.tinkerpop.gremlin.driver.ser.GraphBinaryMessageSerializerV1,
config: { serializeResultToString: true }}
```
!!! note
The java driver is the only driver that currently supports GraphBinary,
see [Connecting to JanusGraph using Java](connecting/java.md).

### Version 0.5.1 (Release Date: March 25, 2020)

```xml tab='Maven'
Expand Down
100 changes: 54 additions & 46 deletions docs/connecting/java.md
Expand Up @@ -7,78 +7,86 @@ information on how to embed JanusGraph, see the [JanusGraph Examples
projects](https://github.com/JanusGraph/janusgraph/tree/master/janusgraph-examples).

This section only covers how applications can connect to JanusGraph
Server. Refer to [Gremlin Query Language](../basics/gremlin.md) for an introduction to Gremlin and
Server using the [GraphBinary](http://tinkerpop.apache.org/docs/current/dev/io/#graphbinary) serialization. Refer to [Gremlin Query Language](../basics/gremlin.md) for an introduction to Gremlin and
pointers to further resources.

## Getting Started with JanusGraph and Gremlin-Java

To get started with JanusGraph in Java:

1. Create an application with Maven:
```bash
mvn archetype:generate -DgroupId=com.mycompany.project
-DartifactId=gremlin-example
-DarchetypeArtifactId=maven-archetype-quickstart
-DinteractiveMode=false
```

```bash
mvn archetype:generate -DgroupId=com.mycompany.project
-DartifactId=gremlin-example
-DarchetypeArtifactId=maven-archetype-quickstart
-DinteractiveMode=false
```

2. Add dependencies on `janusgraph-driver` and `gremlin-driver` to the dependency manager:

```xml tab='Maven'
<dependency>
<groupId>org.janusgraph</groupId>
<artifactId>janusgraph-driver</artifactId>
<version>{{ latest_version }}</version>
</dependency>
<dependency>
<groupId>org.apache.tinkerpop</groupId>
<artifactId>gremlin-driver</artifactId>
<version>{{ tinkerpop_version }}</version>
</dependency>
```

```groovy tab='Gradle'
compile "org.janusgraph:janusgraph-driver:{{ latest_version }}"
compile "org.apache.tinkerpop:gremlin-driver:{{ tinkerpop_version }}"
```
```xml tab='Maven'
<dependency>
<groupId>org.janusgraph</groupId>
<artifactId>janusgraph-driver</artifactId>
<version>{{ latest_version }}</version>
</dependency>
<dependency>
<groupId>org.apache.tinkerpop</groupId>
<artifactId>gremlin-driver</artifactId>
<version>{{ tinkerpop_version }}</version>
</dependency>
```

```groovy tab='Gradle'
compile "org.janusgraph:janusgraph-driver:{{ latest_version }}"
compile "org.apache.tinkerpop:gremlin-driver:{{ tinkerpop_version }}"
```

3. Add two configuration files, `conf/remote-graph.properties` and
`conf/remote-objects.yaml`:

```properties tab='conf/remote-graph.properties'
gremlin.remote.remoteConnectionClass=org.apache.tinkerpop.gremlin.driver.remote.DriverRemoteConnection
gremlin.remote.driver.clusterFile=conf/remote-objects.yaml
gremlin.remote.driver.sourceName=g
```
```properties tab='conf/remote-graph.properties'
gremlin.remote.remoteConnectionClass=org.apache.tinkerpop.gremlin.driver.remote.DriverRemoteConnection
gremlin.remote.driver.clusterFile=conf/remote-objects.yaml
gremlin.remote.driver.sourceName=g
```

```yaml tab='conf/remote-objects.yaml'
hosts: [localhost]
port: 8182
serializer: {
className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV1d0,
config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
```
```yaml tab='conf/remote-objects.yaml'
hosts: [localhost]
port: 8182
serializer: {
className: org.apache.tinkerpop.gremlin.driver.ser.GraphBinaryMessageSerializerV1,
config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
```

4. Create a `GraphTraversalSource` which is the basis for all Gremlin traversals:
```java

```java
Graph graph = EmptyGraph.instance();
GraphTraversalSource g = graph.traversal().withRemote("conf/remote-graph.properties");
// Reuse 'g' across the application
// and close it on shut-down to close open connections with g.close()
```
```

5. Execute a simple traversal:
```java
Object herculesAge = g.V().has("name", "hercules").values("age").next();
System.out.println("Hercules is " + herculesAge + " years old.");
```
`next()` is a terminal step that submits the traversal to the Gremlin Server and returns a single result.

```java
Object herculesAge = g.V().has("name", "hercules").values("age").next();
System.out.println("Hercules is " + herculesAge + " years old.");
```

`next()` is a terminal step that submits the traversal to the Gremlin Server and returns a single result.

## JanusGraph Specific Types and Predicates

JanusGraph specific types and [predicates](../index-backend/search-predicates.md) can be
used directly from a Java application through the dependency `janusgraph-driver`.


## JanusGraph-Core vs JanusGraph-Driver
## Consideration for Accessing the Management API

The described connection uses [GraphBinary](http://tinkerpop.apache.org/docs/current/dev/io/#graphbinary) and the `janusgraph-driver` which doesn't allow accessing the internal JanusGraph components such as `ManagementSystem`. To access the `ManagementSystem`, you have to update the package and the serialization.

1. If you just want to use Gremlin to communicate with JanusGraph, consider to use the maven package `janusgraph-driver`.
2. If you require to access to internal JanusGraph component such as ManagementSystem, consider to use the maven package `janusgraph-core`.
* The maven package `janusgraph-driver` needed to be replaced with the maven package `janusgraph-core`.
* Serialization class in the file `conf/remote-objects.yaml` have to be updated by replacing `className: org.apache.tinkerpop.gremlin.driver.ser.GraphBinaryMessageSerializerV1` with `className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV1d0,`.
Expand Up @@ -26,6 +26,8 @@ scriptEngines: {
org.apache.tinkerpop.gremlin.jsr223.ImportGremlinPlugin: {classImports: [java.lang.Math], methodImports: [java.lang.Math#*]},
org.apache.tinkerpop.gremlin.jsr223.ScriptFileGremlinPlugin: {files: [scripts/empty-sample.groovy]}}}}
serializers:
- { className: org.apache.tinkerpop.gremlin.driver.ser.GraphBinaryMessageSerializerV1, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
- { className: org.apache.tinkerpop.gremlin.driver.ser.GraphBinaryMessageSerializerV1, config: { serializeResultToString: true }}
- { className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV3d0, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
- { className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV3d0, config: { serializeResultToString: true }}
- { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV3d0, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
Expand Down
Expand Up @@ -26,6 +26,8 @@ scriptEngines: {
org.apache.tinkerpop.gremlin.jsr223.ImportGremlinPlugin: {classImports: [java.lang.Math], methodImports: [java.lang.Math#*]},
org.apache.tinkerpop.gremlin.jsr223.ScriptFileGremlinPlugin: {files: [scripts/empty-sample.groovy]}}}}
serializers:
- { className: org.apache.tinkerpop.gremlin.driver.ser.GraphBinaryMessageSerializerV1, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
- { className: org.apache.tinkerpop.gremlin.driver.ser.GraphBinaryMessageSerializerV1, config: { serializeResultToString: true }}
- { className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV3d0, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
- { className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV3d0, config: { serializeResultToString: true }}
- { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV3d0, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
Expand Down
Expand Up @@ -28,6 +28,8 @@ scriptEngines: {
org.apache.tinkerpop.gremlin.jsr223.ImportGremlinPlugin: {classImports: [java.lang.Math], methodImports: [java.lang.Math#*]},
org.apache.tinkerpop.gremlin.jsr223.ScriptFileGremlinPlugin: {files: []}}}}
serializers:
- { className: org.apache.tinkerpop.gremlin.driver.ser.GraphBinaryMessageSerializerV1, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
- { className: org.apache.tinkerpop.gremlin.driver.ser.GraphBinaryMessageSerializerV1, config: { serializeResultToString: true }}
- { className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV3d0, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
- { className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV3d0, config: { serializeResultToString: true }}
- { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV3d0, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
Expand Down
Expand Up @@ -27,6 +27,8 @@ scriptEngines: {
org.apache.tinkerpop.gremlin.jsr223.ImportGremlinPlugin: {classImports: [java.lang.Math], methodImports: [java.lang.Math#*]},
org.apache.tinkerpop.gremlin.jsr223.ScriptFileGremlinPlugin: {files: [scripts/empty-sample.groovy]}}}}
serializers:
- { className: org.apache.tinkerpop.gremlin.driver.ser.GraphBinaryMessageSerializerV1, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
- { className: org.apache.tinkerpop.gremlin.driver.ser.GraphBinaryMessageSerializerV1, config: { serializeResultToString: true }}
- { className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV3d0, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
- { className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV3d0, config: { serializeResultToString: true }}
- { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV3d0, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
Expand Down
Expand Up @@ -27,6 +27,8 @@ scriptEngines: {
org.apache.tinkerpop.gremlin.jsr223.ImportGremlinPlugin: {classImports: [java.lang.Math], methodImports: [java.lang.Math#*]},
org.apache.tinkerpop.gremlin.jsr223.ScriptFileGremlinPlugin: {files: [scripts/empty-sample.groovy]}}}}
serializers:
- { className: org.apache.tinkerpop.gremlin.driver.ser.GraphBinaryMessageSerializerV1, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
- { className: org.apache.tinkerpop.gremlin.driver.ser.GraphBinaryMessageSerializerV1, config: { serializeResultToString: true }}
- { className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV3d0, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
- { className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV3d0, config: { serializeResultToString: true }}
- { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV3d0, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
Expand Down
17 changes: 17 additions & 0 deletions janusgraph-dist/src/assembly/static/conf/remote-graph-binary.yaml
@@ -0,0 +1,17 @@
# Copyright 2020 JanusGraph Authors
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

hosts: [localhost]
port: 8182
serializer: { className: org.apache.tinkerpop.gremlin.driver.ser.GraphBinaryMessageSerializerV1, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
Expand Up @@ -16,7 +16,7 @@

import com.google.common.base.Preconditions;
import org.janusgraph.graphdb.query.JanusGraphPredicate;
import org.apache.tinkerpop.gremlin.process.traversal.P;
import org.janusgraph.graphdb.tinkerpop.io.JanusGraphP;

/**
* Comparison relations for geographic shapes.
Expand Down Expand Up @@ -156,16 +156,16 @@ public boolean isQNF() {

//////////////// statics

public static <V> P<V> geoIntersect(final V value) {
return new P(Geo.INTERSECT, value);
public static <V> JanusGraphP geoIntersect(final V value) {
return new JanusGraphP(Geo.INTERSECT, value);
}
public static <V> P<V> geoDisjoint(final V value) {
return new P(Geo.DISJOINT, value);
public static <V> JanusGraphP geoDisjoint(final V value) {
return new JanusGraphP(Geo.DISJOINT, value);
}
public static <V> P<V> geoWithin(final V value) {
return new P(Geo.WITHIN, value);
public static <V> JanusGraphP geoWithin(final V value) {
return new JanusGraphP(Geo.WITHIN, value);
}
public static <V> P<V> geoContains(final V value) {
return new P(Geo.CONTAINS, value);
public static <V> JanusGraphP geoContains(final V value) {
return new JanusGraphP(Geo.CONTAINS, value);
}
}
Expand Up @@ -22,8 +22,8 @@

import org.apache.commons.lang.StringUtils;
import org.apache.commons.text.similarity.LevenshteinDistance;
import org.apache.tinkerpop.gremlin.process.traversal.P;
import org.janusgraph.graphdb.query.JanusGraphPredicate;
import org.janusgraph.graphdb.tinkerpop.io.JanusGraphP;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

Expand Down Expand Up @@ -322,25 +322,25 @@ public boolean isQNF() {
public final static Set<Text> HAS_CONTAINS = Collections
.unmodifiableSet(EnumSet.of(CONTAINS, CONTAINS_PREFIX, CONTAINS_REGEX, CONTAINS_FUZZY));

public static <V> P<V> textContains(final V value) {
return new P(Text.CONTAINS, value);
public static <V> JanusGraphP textContains(final V value) {
return new JanusGraphP(Text.CONTAINS, value);
}
public static <V> P<V> textContainsPrefix(final V value) {
return new P(Text.CONTAINS_PREFIX, value);
public static <V> JanusGraphP textContainsPrefix(final V value) {
return new JanusGraphP(Text.CONTAINS_PREFIX, value);
}
public static <V> P<V> textContainsRegex(final V value) {
return new P(Text.CONTAINS_REGEX, value);
public static <V> JanusGraphP textContainsRegex(final V value) {
return new JanusGraphP(Text.CONTAINS_REGEX, value);
}
public static <V> P<V> textPrefix(final V value) {
return new P(Text.PREFIX, value);
public static <V> JanusGraphP textPrefix(final V value) {
return new JanusGraphP(Text.PREFIX, value);
}
public static <V> P<V> textRegex(final V value) {
return new P(Text.REGEX, value);
public static <V> JanusGraphP textRegex(final V value) {
return new JanusGraphP(Text.REGEX, value);
}
public static <V> P<V> textContainsFuzzy(final V value) {
return new P(Text.CONTAINS_FUZZY, value);
public static <V> JanusGraphP textContainsFuzzy(final V value) {
return new JanusGraphP(Text.CONTAINS_FUZZY, value);
}
public static <V> P<V> textFuzzy(final V value) {
return new P(Text.FUZZY, value);
public static <V> JanusGraphP textFuzzy(final V value) {
return new JanusGraphP(Text.FUZZY, value);
}
}
Expand Up @@ -14,12 +14,16 @@

package org.janusgraph.graphdb.tinkerpop;

import org.apache.tinkerpop.gremlin.process.traversal.P;
import org.apache.tinkerpop.gremlin.structure.io.AbstractIoRegistry;
import org.apache.tinkerpop.gremlin.structure.io.binary.GraphBinaryIo;
import org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONIo;
import org.apache.tinkerpop.gremlin.structure.io.gryo.GryoIo;
import org.janusgraph.core.attribute.Geoshape;
import org.janusgraph.graphdb.tinkerpop.io.JanusGraphP;
import org.janusgraph.graphdb.relations.RelationIdentifier;
import org.janusgraph.graphdb.tinkerpop.io.binary.GeoshapeGraphBinarySerializer;
import org.janusgraph.graphdb.tinkerpop.io.binary.JanusGraphPBinarySerializer;
import org.janusgraph.graphdb.tinkerpop.io.binary.RelationIdentifierGraphBinarySerializer;
import org.janusgraph.graphdb.tinkerpop.io.graphson.JanusGraphSONModuleV2d0;

/**
Expand All @@ -32,9 +36,12 @@ public class JanusGraphIoRegistry extends AbstractIoRegistry {

private JanusGraphIoRegistry() {
register(GraphSONIo.class, null, JanusGraphSONModuleV2d0.getInstance());
register(GraphBinaryIo.class, RelationIdentifier.class, new RelationIdentifierGraphBinarySerializer());
register(GraphBinaryIo.class, Geoshape.class, new GeoshapeGraphBinarySerializer());
register(GraphBinaryIo.class, JanusGraphP.class, new JanusGraphPBinarySerializer());
register(GryoIo.class, RelationIdentifier.class, null);
register(GryoIo.class, Geoshape.class, new Geoshape.GeoShapeGryoSerializer());
register(GryoIo.class, P.class, new JanusGraphPSerializer());
register(GryoIo.class, JanusGraphP.class, new JanusGraphPSerializer());
}

public static JanusGraphIoRegistry instance() {
Expand Down

0 comments on commit 1cb4b6e

Please sign in to comment.