Skip to content

Commit

Permalink
[GIE Doc] Add examples to collect gremlin results in a streaming way (#…
Browse files Browse the repository at this point in the history
…2842)

<!--
Thanks for your contribution! please review
https://github.com/alibaba/GraphScope/blob/main/CONTRIBUTING.md before
opening an issue.
-->

## What do these changes do?
The official python gremlin client lacks guidelines about how to use
streaming apis, here give a java example instead. The python example
will be added later.

<!-- Please give a short brief about these changes. -->

## Related issue number

<!-- Are there any issues opened that will be resolved by merging this
change? -->

Fixes
  • Loading branch information
shirly121 committed Jun 9, 2023
1 parent 5fd6cc1 commit a0b36ee
Showing 1 changed file with 44 additions and 0 deletions.
44 changes: 44 additions & 0 deletions docs/interactive_engine/tinkerpop_eco.md
Expand Up @@ -33,10 +33,54 @@ A simpler option is to use the `gremlin` object for submitting Gremlin queries t
Gremlin-Python and will automatically acquire the endpoint.
````

In large-scale data processing scenarios, streaming the returned data is often necessary to avoid Out of Memory (OOM) issues caused by handling a large volume of data, which provides benefits such as memory efficiency, continuous processing, incremental analysis, reduced latency, scalability, and resource optimization. It enables you to handle and analyze vast amounts of data effectively while mitigating the risk of memory-related issues. Here is an example to guide you how to collect results in a streaming way by python sdk.
```Python
from queue import Queue
from gremlin_python.driver.client import Client

graph_url = # the GIE Frontend service endpoint you've obtained
client = Client(graph_url, "g")

ret = []
q = client.submit('g.V()')
while True:
try:
ret.extend(q.next())
except StopIteration:
break

print(ret)
```
Furthermore, here are some parameters that can be used to configure the streaming size on the server-side.
```bash
# interactive_engine/compiler/src/main/resources/conf/gremlin-server.yaml
...
# total num of streaming batch size returned by compiler service
resultIterationBatchSize: 64
...

```

## Connecting Gremlin within Java
See [Gremlin-Java](https://tinkerpop.apache.org/docs/current/reference/#gremlin-java) for connecting Gremlin
within the Java language.

Here is an example to guide you how to collect results in a streaming way by java sdk.
```java
Cluster cluster = Cluster.build()
.addContactPoint("localhost") // use your host ip
.port(8182) // use your port
.create();
Client client = cluster.connect();
ResultSet resultSet = client.submit("g.V()"); // use your query
Iterator<Result> results = resultSet.iterator();
while(results.hasNext()) {
display(results.next()); // display each result in your way
}
client.close();
cluster.close();
```

## Gremlin Console
1. Download Gremlin console and unpack to your local directory.
```bash
Expand Down

0 comments on commit a0b36ee

Please sign in to comment.