diff --git a/docs/docs/ops/storages.md b/docs/docs/ops/storages.md index a719b7c6..59d20e51 100644 --- a/docs/docs/ops/storages.md +++ b/docs/docs/ops/storages.md @@ -386,19 +386,7 @@ You can find end-to-end examples fitting into any of supported property graphs i ### Neo4j -If you don't have a Neo4j database, you can start a Neo4j database using our docker compose config: - -```bash -docker compose -f <(curl -L https://raw.githubusercontent.com/cocoindex-io/cocoindex/refs/heads/main/dev/neo4j.yaml) up -d -``` - -:::warning - -The docker compose config above will start a Neo4j Enterprise instance under the [Evaluation License](https://neo4j.com/terms/enterprise_us/), -with 30 days trial period. -Please read and agree the license before starting the instance. - -::: +#### Spec The `Neo4j` target spec takes the following fields: @@ -417,17 +405,32 @@ Neo4j also provides a declaration spec `Neo4jDeclaration`, to configure indexing * `primary_key_fields` (required) * `vector_indexes` (optional) -### Kuzu +#### Neo4j dev instance -CocoIndex supports talking to Kuzu through its [API server](https://github.com/kuzudb/api-server). -You can bring up a Kuzu API server locally by running: +If you don't have a Neo4j database, you can start a Neo4j database using our docker compose config: ```bash -KUZU_DB_DIR=$HOME/.kuzudb -KUZU_PORT=8123 -docker run -d --name kuzu -p ${KUZU_PORT}:8000 -v ${KUZU_DB_DIR}:/database kuzudb/api-server:latest +docker compose -f <(curl -L https://raw.githubusercontent.com/cocoindex-io/cocoindex/refs/heads/main/dev/neo4j.yaml) up -d ``` +If will bring up a Neo4j instance, which can be accessed by username `neo4j` and password `cocoindex`. +You can access the Neo4j browser at [http://localhost:7474](http://localhost:7474). + +:::warning + +The docker compose config above will start a Neo4j Enterprise instance under the [Evaluation License](https://neo4j.com/terms/enterprise_us/), +with 30 days trial period. +Please read and agree the license before starting the instance. + +::: + + +### Kuzu + +#### Spec + +CocoIndex supports talking to Kuzu through its [API server](https://github.com/kuzudb/api-server). + The `Kuzu` target spec takes the following fields: * `connection` (type: [auth reference](../core/flow_def#auth-registry) to `KuzuConnectionSpec`): The connection to the Kuzu database. `KuzuConnectionSpec` has the following fields: @@ -440,3 +443,25 @@ Kuzu also provides a declaration spec `KuzuDeclaration`, to configure indexing o * Fields for [nodes to declare](#declare-extra-node-labels), including * `nodes_label` (required) * `primary_key_fields` (required) + +#### Kuzu dev instance + +If you don't have a Kuzu instance yet, you can bring up a Kuzu API server locally by running: + +```bash +KUZU_DB_DIR=$HOME/.kuzudb +KUZU_PORT=8123 +docker run -d --name kuzu -p ${KUZU_PORT}:8000 -v ${KUZU_DB_DIR}:/database kuzudb/api-server:latest +``` + +To explore the graph you built with Kuzu, you can use the [Kuzu Explorer](https://github.com/kuzudb/explorer). +Currently Kuzu API server and the explorer cannot be up at the same time. So you need to stop the API server before running the explorer. + +To start the instance of the explorer, run: + +```bash +KUZU_EXPLORER_PORT=8124 +docker run -d --name kuzu-explorer -p ${KUZU_EXPLORER_PORT}:8000 -v ${KUZU_DB_DIR}:/database -e MODE=READ_ONLY kuzudb/explorer:latest +``` + +You can then access the explorer at [http://localhost:8124](http://localhost:8124). diff --git a/examples/docs_to_knowledge_graph/README.md b/examples/docs_to_knowledge_graph/README.md index e297db36..affea9c1 100644 --- a/examples/docs_to_knowledge_graph/README.md +++ b/examples/docs_to_knowledge_graph/README.md @@ -12,10 +12,10 @@ Please drop [Cocoindex on Github](https://github.com/cocoindex-io/cocoindex) a s ![example-explanation](https://github.com/user-attachments/assets/07ddbd60-106f-427f-b7cc-16b73b142d27) - ## Prerequisite * [Install Postgres](https://cocoindex.io/docs/getting_started/installation#-install-postgres) if you don't have one. -* [Install Neo4j](https://cocoindex.io/docs/ops/storages#neo4j) if you don't have one. +* Install [Neo4j](https://cocoindex.io/docs/ops/storages#neo4j-dev-instance) or [Kuzu](https://cocoindex.io/docs/ops/storages#kuzu-dev-instance) if you don't have one. + * The example uses Neo4j by default for now. If you want to use Kuzu, find out the "SELECT ONE GRAPH DATABASE TO USE" section and switch the active branch. * [Configure your OpenAI API key](https://cocoindex.io/docs/ai/llm#openai). ## Documentation @@ -45,21 +45,18 @@ cocoindex update main.py ### Browse the knowledge graph -After the knowledge graph is build, you can explore the knowledge graph you built in Neo4j Browser. +After the knowledge graph is built, you can explore the knowledge graph. -For the dev enviroment, you can connect neo4j browser using credentials: -- username: `neo4j` -- password: `cocoindex` -which is pre-configured in the our docker compose [config.yaml](https://raw.githubusercontent.com/cocoindex-io/cocoindex/refs/heads/main/dev/neo4j.yaml). +* If you're using Neo4j, you can open the explorer at [http://localhost:7474](http://localhost:7474), with username `neo4j` and password `cocoindex`. +* If you're using Kuzu, you can start a Kuzu explorer locally. See [Kuzu dev instance](https://cocoindex.io/docs/ops/storages#kuzu-dev-instance) for more details. -You can open it at [http://localhost:7474](http://localhost:7474), and run the following Cypher query to get all relationships: +You can run the following Cypher query to get all relationships: ```cypher MATCH p=()-->() RETURN p ``` -neo4j-for-coco-docs - +neo4j-for-coco-docs ## CocoInsight I used CocoInsight (Free beta now) to troubleshoot the index generation and understand the data lineage of the pipeline. diff --git a/examples/docs_to_knowledge_graph/main.py b/examples/docs_to_knowledge_graph/main.py index 6bf2fd97..b73274d5 100644 --- a/examples/docs_to_knowledge_graph/main.py +++ b/examples/docs_to_knowledge_graph/main.py @@ -5,27 +5,6 @@ import dataclasses import cocoindex - -@dataclasses.dataclass -class DocumentSummary: - """Describe a summary of a document.""" - - title: str - summary: str - - -@dataclasses.dataclass -class Relationship: - """ - Describe a relationship between two entities. - Subject and object should be Core CocoIndex concepts only, should be nouns. For example, `CocoIndex`, `Incremental Processing`, `ETL`, `Data` etc. - """ - - subject: str - predicate: str - object: str - - neo4j_conn_spec = cocoindex.add_auth_entry( "Neo4jConnection", cocoindex.storages.Neo4jConnection( @@ -41,19 +20,43 @@ class Relationship: ), ) -# Use Neo4j as the graph database +# SELECT ONE GRAPH DATABASE TO USE +# This example can use either Neo4j or Kuzu as the graph database. +# Please make sure only one branch is live and others are commented out. + +# Use Neo4j GraphDbSpec = cocoindex.storages.Neo4j GraphDbConnection = cocoindex.storages.Neo4jConnection GraphDbDeclaration = cocoindex.storages.Neo4jDeclaration conn_spec = neo4j_conn_spec -# Use Kuzu as the graph database +# Use Kuzu # GraphDbSpec = cocoindex.storages.Kuzu # GraphDbConnection = cocoindex.storages.KuzuConnection # GraphDbDeclaration = cocoindex.storages.KuzuDeclaration # conn_spec = kuzu_conn_spec +@dataclasses.dataclass +class DocumentSummary: + """Describe a summary of a document.""" + + title: str + summary: str + + +@dataclasses.dataclass +class Relationship: + """ + Describe a relationship between two entities. + Subject and object should be Core CocoIndex concepts only, should be nouns. For example, `CocoIndex`, `Incremental Processing`, `ETL`, `Data` etc. + """ + + subject: str + predicate: str + object: str + + @cocoindex.flow_def(name="DocsToKG") def docs_to_kg_flow( flow_builder: cocoindex.FlowBuilder, data_scope: cocoindex.DataScope diff --git a/examples/product_recommendation/README.md b/examples/product_recommendation/README.md index 98e9be55..c97dd298 100644 --- a/examples/product_recommendation/README.md +++ b/examples/product_recommendation/README.md @@ -9,7 +9,8 @@ Please drop [CocoIndex on Github](https://github.com/cocoindex-io/cocoindex) a s ## Prerequisite * [Install Postgres](https://cocoindex.io/docs/getting_started/installation#-install-postgres) if you don't have one. -* [Install Neo4j](https://cocoindex.io/docs/ops/storages#neo4j) if you don't have one. +* Install [Neo4j](https://cocoindex.io/docs/ops/storages#neo4j-dev-instance) or [Kuzu](https://cocoindex.io/docs/ops/storages#kuzu-dev-instance) if you don't have one. + * The example uses Neo4j by default for now. If you want to use Kuzu, find out the "SELECT ONE GRAPH DATABASE TO USE" section and switch the active branch. * [Configure your OpenAI API key](https://cocoindex.io/docs/ai/llm#openai). ## Documentation @@ -39,18 +40,17 @@ cocoindex update main.py ### Browse the knowledge graph -After the knowledge graph is built, you can explore the knowledge graph you built in Neo4j Browser. +After the knowledge graph is built, you can explore the knowledge graph. -For the dev enviroment, you can connect neo4j browser using credentials: -- username: `neo4j` -- password: `cocoindex` -which is pre-configured in the our docker compose [config.yaml](https://raw.githubusercontent.com/cocoindex-io/cocoindex/refs/heads/main/dev/neo4j.yaml). +* If you're using Neo4j, you can open the explorer at [http://localhost:7474](http://localhost:7474), with username `neo4j` and password `cocoindex`. +* If you're using Kuzu, you can start a Kuzu explorer locally. See [Kuzu dev instance](https://cocoindex.io/docs/ops/storages#kuzu-dev-instance) for more details. -You can open it at [http://localhost:7474](http://localhost:7474), and run the following Cypher query to get all relationships: +You can run the following Cypher query to get all relationships: ```cypher MATCH p=()-->() RETURN p ``` + ![Neo4j Browser Screenshot](img/neo4j.png) ## CocoInsight diff --git a/examples/product_recommendation/main.py b/examples/product_recommendation/main.py index 0cc3e231..7c0c328f 100644 --- a/examples/product_recommendation/main.py +++ b/examples/product_recommendation/main.py @@ -7,6 +7,38 @@ import cocoindex from jinja2 import Template +neo4j_conn_spec = cocoindex.add_auth_entry( + "Neo4jConnection", + cocoindex.storages.Neo4jConnection( + uri="bolt://localhost:7687", + user="neo4j", + password="cocoindex", + ), +) +kuzu_conn_spec = cocoindex.add_auth_entry( + "KuzuConnection", + cocoindex.storages.KuzuConnection( + api_server_url="http://localhost:8123", + ), +) + +# SELECT ONE GRAPH DATABASE TO USE +# This example can use either Neo4j or Kuzu as the graph database. +# Please make sure only one branch is live and others are commented out. + +# Use Neo4j +GraphDbSpec = cocoindex.storages.Neo4j +GraphDbConnection = cocoindex.storages.Neo4jConnection +GraphDbDeclaration = cocoindex.storages.Neo4jDeclaration +conn_spec = neo4j_conn_spec + +# Use Kuzu +# GraphDbSpec = cocoindex.storages.Kuzu +# GraphDbConnection = cocoindex.storages.KuzuConnection +# GraphDbDeclaration = cocoindex.storages.KuzuDeclaration +# conn_spec = kuzu_conn_spec + + # Template for rendering product information as markdown to provide information to LLMs PRODUCT_TEMPLATE = """ # {{ title }} @@ -77,34 +109,6 @@ def extract_product_info(product: cocoindex.Json, filename: str) -> ProductInfo: ) -neo4j_conn_spec = cocoindex.add_auth_entry( - "Neo4jConnection", - cocoindex.storages.Neo4jConnection( - uri="bolt://localhost:7687", - user="neo4j", - password="cocoindex", - ), -) -kuzu_conn_spec = cocoindex.add_auth_entry( - "KuzuConnection", - cocoindex.storages.KuzuConnection( - api_server_url="http://localhost:8123", - ), -) - -# Use Neo4j as the graph database -GraphDbSpec = cocoindex.storages.Neo4j -GraphDbConnection = cocoindex.storages.Neo4jConnection -GraphDbDeclaration = cocoindex.storages.Neo4jDeclaration -conn_spec = neo4j_conn_spec - -# Use Kuzu as the graph database -# GraphDbSpec = cocoindex.storages.Kuzu -# GraphDbConnection = cocoindex.storages.KuzuConnection -# GraphDbDeclaration = cocoindex.storages.KuzuDeclaration -# conn_spec = kuzu_conn_spec - - @cocoindex.flow_def(name="StoreProduct") def store_product_flow( flow_builder: cocoindex.FlowBuilder, data_scope: cocoindex.DataScope