POST /connect
Neo4j database connection on frontend is done with this API.
API Parameters :
-
uri= Neo4j uri, -
userName= Neo4j db username, -
password= Neo4j db password, -
database= Neo4j database name
Response :
{
"status": "Success",
"data": {
"db_vector_dimension": 384,
"application_dimension": 384,
"message": "Connection Successful",
"gds_status": true,
"write_access": true,
"elapsed_api_time": "5.48"
}POST /upload
The upload endpoint is designed to handle the uploading of large files by breaking them into smaller chunks. This method ensures that large files can be uploaded efficiently without overloading the server.
API Parameters
-
file=The file to be uploaded, received in chunks, -
chunkNumber=The current chunk number being uploaded, -
totalChunks=The total number of chunks the file is divided into (each chunk of 1Mb size), -
originalname=The original name of the file, -
model=The model associated with the file, -
uri=Neo4j uri, -
userName= Neo4j db username, -
password= Neo4j db password, -
database= Neo4j database name
Response :
{
"status": "Success",
"message": "File uploaded and chunks merged successfully."
}POST /schema
User can set schema for graph generation (i.e. Nodes and relationship labels) in settings panel or get existing db schema through this API.
API Parameters :
-
uri=Neo4j uri, -
userName= Neo4j db username, -
password= Neo4j db password, -
database= Neo4j database name
Response :
{
"status": "Success",
"data": [
{
"labels": [
"Access_token",
"Activity",
"Ai chatbot",
"Book",
"Metric",
"Mode",
"Mountain"
],
"relationshipTypes": [
"ACCELERATE",
"ACCEPTS",
"CONVERT",
"CORRELATE",
"ESTABLISHED",
"EXAMPLE_OF"
]
}
]
}POST /populate_graph_schema
The API is used to populate a graph schema based on the provided input text, model, and schema description flag.
API Parameters :
-
input_text=The input text used to populate the graph schema. -
model=The model to be used for populating the graph schema. -
is_schema_description_checked=A flag indicating whether the schema description should be considered.
Response :
{
"status": "Success",
"data": [
{
"labels": [
"Technology",
"Company",
"Person",
"Location",
"Organization",
"Concept"
],
"relationshipTypes": [
"LOCATED_AT",
"SUBSIDARY_OF",
"BORN_IN",
"LAST_MESSAGE",
"ATTENDED",
"PARTNERED_WITH"
]
}
]
}POST /url/scan
Create Document node for other sources - s3 bucket, gcs bucket, wikipedia, youtube url and web pages.
API Parameters :
-
uri=Neo4j uri, -
userName= Neo4j db username, -
password= Neo4j db password, -
database= Neo4j database name -
model= LLM model, -
source_url= <s3 bucket url or youtube url> , -
aws_access_key_id= AWS access key, -
aws_secret_access_key= AWS secret key, -
wiki_query= Wikipedia query sources, -
gcs_project_id= GCS project id, -
gcs_bucket_name= GCS bucket name, -
gcs_bucket_folder= GCS bucket folder, -
source_type= s3 bucket/ gcs bucket/ youtube/Wikipedia as source type -
gcs_project_id=Form(None), -
access_token=Form(None)
Response :
{
"status": "Success",
"success_count": 2,
"failed_count": 0,
"message": "Source Node created successfully for source type: Wikipedia and source: Albert Einstein, neo4j",
"file_name": [
{
"fileName": "Albert Einstein",
"fileSize": 8074,
"url": "https://en.wikipedia.org/wiki/Albert_Einstein",
"status": "Success"
}
]
}POST /extract :
This API is responsible for -
-
Reading the content of source provided in the form of langchain Document object from respective langchain loaders
-
Dividing the document into multiple chunks, and make below relations -
-
PART_OF - relation from Document node to all chunk nodes
-
FIRST_CHUNK - relation from document node to first chunk node
-
NEXT_CHUNK - relation from a chunk pointing to next chunk of the document.
-
HAS_ENTITY - relation between chunk node and entities extracted from LLM.
-
-
Extracting nodes and relations in the form of GraphDocument from respective LLM.
-
Update embedding of chunks and create vector index.
-
Update K-Nearest Neighbors graph for similar chunks.
Implementation :
-
For multiple sources of content -
-
Local file - User can upload pdf file from their device.
-
s3 bucket - User passes the bucket url and all the pdf files inside folders and subfolders will be listed.
-
GCS bucket - User passes gcs project id, gcs bucket name and folder name, do google authentication to access all the pdf files under that folder and its subfolders and if folder name is not passed by user, all the pdf files under the bucket and its subfolders will be listed if user have read access of the bucket.
-
Web Sources -
-
Wikipedia - Wikipedia 1st page content is rendered url passed by user.
-
Youtube - Youtube video transcript is processed and if no transcript is available then respective error is thrown.
-
Web urls - Text Content from any web url is processed for generating graph.
-
-
-
Langchain’s LLMGraphTransformer library is used to get nodes and relations in the form of GraphDocument from LLMs. User and System prompts, LLM chain, graphDocument schema are defined in the library itself.
-
SentenceTransformer embeddingds are used by default, also embeddings are made configurable to use either OpenAIEmbeddings or VertexAIEmbeddings.
-
Vector index is created in databse on embeddingds created for chunks.
API Parameters :
-
uri=Neo4j uri, -
userName= Neo4j db username, -
password= Neo4j db password, -
database= Neo4j database name -
model= LLM model, -
file_name= File uploaded from device -
source_url= <s3 bucket url or youtube url> , -
aws_access_key_id= AWS access key, -
aws_secret_access_key= AWS secret key, -
wiki_query= Wikipedia query sources, -
gcs_project_id=GCS project id, -
gcs_bucket_name= GCS bucket name, -
gcs_bucket_folder= GCS bucket folder, -
gcs_blob_filename= GCS file name, -
source_type= local file/ s3 bucket/ gcs bucket/ youtube/ Wikipedia as source, allowedNodes=Node labels passed from settings panel, -
allowedRelationship=Relationship labels passed from settings panel, -
language=Language in which wikipedia content will be extracted
Response :
{
"status": "Success",
"data": {
"fileName": <PDF File Name/ Wikipedia Query string/ Youtube video title>,
"nodeCount": <No. Nodes extracted from LLM>,
"relationshipCount": <No. of relations extracted from LLM>,
"processingTime": <Total time taken by application to give response>,
"status": "Completed",
"model": <LLM Model choosen by User>
}
}GET /sources_list
List all sources (Document nodes) present in Neo4j graph database.
API Parameters :
-
uri=Neo4j uri, -
userName= Neo4j db username, -
password= Neo4j db password, -
database= Neo4j database name
Response :
{
"status": "Success",
"data": [
{
"fileName": "About Amazon.pdf",
"fileSize": 163931,
"errorMessage": "",
"fileSource": "local file",
"nodeCount": 62,
"model": "OpenAI GPT 4",
"fileType": "pdf",
"processingTime": 122.71,
"relationshipCount": 187,
"status": "Completed",
"updatedAt": {
"_DateTime__date": {
"_Date__ordinal": 738993,
"_Date__year": 2024,
"_Date__month": 4,
"_Date__day": 17
},
"_DateTime__time": {
"_Time__ticks": 28640715768000,
"_Time__hour": 7,
"_Time__minute": 57,
"_Time__second": 20,
"_Time__nanosecond": 715768000,
"_Time__tzinfo": null
}
}
}
]
}POST /post_processing :
This API is called at the end of processing of whole document to get create k-nearest neighbor relations between similar chunks of document based on KNN_MIN_SCORE which is 0.8 by default and to drop and create a full text index on db labels.
API Parameters :
-
uri=Neo4j uri, -
userName= Neo4j db username, -
password= Neo4j db password, -
database= Neo4j database name -
tasks= List of tasks to perform
Response :
{
"status":"Success",
"message":"All tasks completed successfully"
}POST /chat_bot
The API responsible for a chatbot system designed to leverage multiple AI models and a Neo4j graph database, providing answers to user queries. It interacts with AI models from OpenAI and Google’s Vertex AI and utilizes embedding models to enhance the retrieval of relevant information.
Components :
-
Embedding Models - Includes OpenAI Embeddings, VertexAI Embeddings, and SentenceTransformer Embeddings(Default) to support vector-based query operations.
-
AI Models - OpenAI GPT 3.5, GPT 4o, GPT 40 mini, gemini_1.5_flash can be configured for the chatbot backend to generate responses and process natural language.
-
Graph Database (Neo4jGraph) - Manages interactions with the Neo4j database, retrieving, and storing conversation histories.
-
Response Generation - Utilizes Vector Embeddings from the Neo4j database, chat history, and the knowledge base of the LLM used.
-
Chat Modes - Vector , Graph, Vector + Graph, Fulltext, Vector + Graph+Fulltext, Entity Search + Vector, Global search Vector
API Parameters :
-
uri= Neo4j uri -
userName= Neo4j database username -
password= Neo4j database password -
model= LLM model -
question= User query for the chatbot -
session_id= Session ID used to maintain the history of chats during the user’s connection -
mode= chat mode to use -
document_names= the names of documents to be filtered works for vector mode and vector+Graph mode
Response :
{
"status": "Success",
"data": {
"session_id": "0cbd04a8-abc3-4776-b393-6a9a2cea36b3",
"message": "response generated by the chat",
"info": {
"sources": [
"About Amazon.pdf"
],
"model": "gpt-4o-2024-08-06",
"nodedetails": {
"chunkdetails": [
{
"id": "73bc9c9170bcd807d2fa87d87a0eeb3d82f95160",
"score": 1.0
},
{
"id": "de5486776978353c9f8ac530bcff33eeecbdbbad",
"score": 0.9425
}
],
"entitydetails": [],
"communitydetails": []
},
"total_tokens": 4575,
"response_time": 17.19,
"mode": "graph_vector_fulltext",
"entities": {
"entityids": [
"4:98e5e9bb-8095-440d-9462-03985fed2fa2:307",
"4:98e5e9bb-8095-440d-9462-03985fed2fa2:1877",
],
"relationshipids": [
"5:98e5e9bb-8095-440d-9462-03985fed2fa2:8072566611095062357",
"5:98e5e9bb-8095-440d-9462-03985fed2fa2:8072566508015847224"
]
},
"metric_details": {
"question": "tell me about amazon ",
"contexts": "context sent to LLM"
"answer": "response generated by the LLM"
}
},
"user": "chatbot"
}
}/chunk_entities
This API is used to get the entities and relations associated with a particular chunk and chunk metadata.
API Parameters :
-
uri=Neo4j uri, -
userName= Neo4j db username, -
password= Neo4j db password, -
database= Neo4j database name -
nodedetails= Node element id’s to get information(chunks,entities,communities) -
entities= entities received from the retriver for graph based modes
Response :
{
"status": "Success",
"data": {
"nodes": [
{
"element_id": "4:98e5e9bb-8095-440d-9462-03985fed2fa2:307",
"labels": [
"Company"
],
"properties": {
"id": "Amazon",
"description": "Initially an online bookstore, Amazon has transformed into a $48 billion retail giant, offering products in over forty categories, from books and electronics to groceries. Today, it operates as a logistics platform, a search engine, an Internet advertising platform, an e-commerce platform, and an IT platform."
}
}
],
"relationships": [
{
"element_id": "5:98e5e9bb-8095-440d-9462-03985fed2fa2:6917952339617775946",
"type": "OFFERS",
"start_node_element_id": "4:98e5e9bb-8095-440d-9462-03985fed2fa2:307",
"end_node_element_id": "4:98e5e9bb-8095-440d-9462-03985fed2fa2:330"
}
],
"chunk_data": [
{
"element_id": "4:98e5e9bb-8095-440d-9462-03985fed2fa2:14",
"id": "d1e92be81a0872d621242cee9fed69d14b0cd68d",
"position": 13,
"text": " 6 eBay, operating as the biggest online auction house and focusing as a service provider, employs cost leadership strategy by solely operating e-commerce as an intermediary without holding any inventories or physical infrastructures. It also applies a differentiation strategy by providing a ....",
"content_offset": 9886,
"fileName": "About Amazon.pdf",
"page_number": 7,
"length": 1024,
"fileSource": "local file",
"embedding": null
}
],
"community_data": [
{
"element_id": "4:98e5e9bb-8095-440d-9462-03985fed2fa2:1026",
"summary": "Google, led by CEO Sundar Pichai, is actively involved in various business and product initiatives.",
"id": "0-311",
"level": 0,
"weight": 7,
"embedding": null,
"community_rank": 1
}
]
},
"message": "Total elapsed API time 3.75"
}POST /graph_query
This API is used to view graph for a particular file.
API Parameters :
-
uri=Neo4j uri, -
userName= Neo4j db username, -
password= Neo4j db password, -
query_type= Neo4j database name -
document_names= File name for which user wants to view graph
Response :
{
"status": "Success",
"data": {
"nodes": [
{
"element_id": "4:98e5e9bb-8095-440d-9462-03985fed2fa2:9972",
"labels": [
"Person"
],
"properties": {
"id": "Jeff"
}
},
{
"element_id": "4:98e5e9bb-8095-440d-9462-03985fed2fa2:9973",
"labels": [
"Team"
],
"properties": {
"id": "Miami"
}
}
],
"relationships": [
{
"element_id": "5:98e5e9bb-8095-440d-9462-03985fed2fa2:1153200780560312052",
"type": "PLAYER",
"start_node_element_id": "4:98e5e9bb-8095-440d-9462-03985fed2fa2:9972",
"end_node_element_id": "4:98e5e9bb-8095-440d-9462-03985fed2fa2:9973"
}
]
}
}POST /get_neighbours
This API is used to retrive the neighbor nodes of the given element id of the node.
API Parameters :
-
uri=Neo4j uri, -
userName= Neo4j db username, -
password= Neo4j db password, -
database= Neo4j database name, -
elementId= Element id of the node to retrive its neighbours
Response :
{
"status": "Success",
"data": {
"nodes": [
{
"summary": null,
"element_id": "4:98e5e9bb-8095-440d-9462-03985fed2fa2:3",
"id": "73bc9c9170bcd807d2fa87d87a0eeb3d82f95160",
"position": 2,
"text": null,
"content_offset": 186,
"labels": [
"Chunk"
],
"page_number": 2,
"fileName": "About Amazon.pdf",
"length": 904,
"properties": {
"id": "73bc9c9170bcd807d2fa87d87a0eeb3d82f95160"
},
"embedding": null
}
],
"relationships": [
{
"element_id": "5:98e5e9bb-8095-440d-9462-03985fed2fa2:1175445000301838339",
"end_node_element_id": "4:98e5e9bb-8095-440d-9462-03985fed2fa2:18",
"start_node_element_id": "4:98e5e9bb-8095-440d-9462-03985fed2fa2:3",
"type": "HAS_ENTITY"
},
]
},
"message": "Total elapsed API time 0.24"
}POST /clear_chat_bot
This API is used to clear the chat history which is saved in Neo4j DB.
API Parameters :
-
uri=Neo4j uri, -
userName= Neo4j db username, -
password= Neo4j db password, -
database= Neo4j database name, -
session_id= User session id for QA chat
Response :
{
"status": "Success",
"data": {
"session_id": "99c1a808-377f-448f-9ea6-4b4a8de46b14",
"message": "The chat History is cleared",
"user": "chatbot"
}
}GET /update_extract_status
The API provides a continuous update on the extraction status of a specified file. It uses Server-Sent Events (SSE) to stream updates to the client.
API Parameters :
-
file_name=The name of the file whose extraction status is being tracked, -
uri=Neo4j uri, -
userName= Neo4j db username, -
password= Neo4j db password, -
database= Neo4j database name
Response :
{
"fileName": "testFile.pdf",
"status": "Processing",
"processingTime": 0,
"nodeCount": 0,
"relationshipCount": 0,
"model": "OpenAI GPT 3.5",
"total_chunks": 3,
"fileSize": 92373,
"processed_chunk": 0
}POST /delete_document_and_entities
Overview :
Deleteion of nodes and relations for multiple files is done through this API. User can choose multiple documents to be deleted, also user have option to delete only 'Document' and 'Chunk' nodes and keep the entities extracted from that document.
API Parameters :
-
uri=Neo4j uri, -
userName= Neo4j db username, -
password= Neo4j db password, -
database= Neo4j database name, -
filenames= List of files to be deleted, -
source_types= Document sources(Wikipedia, youtube, etc.), -
deleteEntities= Boolean value to check entities deletion is requested or not
Response :
{
"status": "Success",
"message": "Deleted 1 documents with 68 entities from database"
}/cancelled_job
This API is responsible for cancelling an in process job.
API Parameters :
-
uri=Neo4j uri, -
userName= Neo4j db username, -
password= Neo4j db password, -
database= Neo4j database name, -
filenames= Name of the file whose processing need to be stopped, -
source_types= Source of the file
Response :
{
"message":"Cancelled the processing job successfully"
}POST /get_unconnected_nodes_list
The API retrieves a list of nodes in the graph database that are not connected to any other nodes.
API Parameters :
-
uri=Neo4j uri, -
userName= Neo4j db username, -
password= Neo4j db password, -
database= Neo4j database name
Response :
{ "status": "Success",
"data": [
"e":
{
"id": "Leela Chess Zero",
"elementId": "4:abf6f691-928d-4b1c-80fc-2914ae517b4c:336",
"labels": ["Technology"],
"embedding": null
},
"documents": ["AlphaZero - Wikipedia.pdf"],
"chunkConnections": 7
]
}POST /delete_unconnected_nodes
The API is used to delete unconnected entities from database.
API Parameters :
-
uri=Neo4j uri, -
userName= Neo4j db username, -
password= Neo4j db password, -
database= Neo4j database name, -
unconnected_entities_list=selected entities list to delete of unconnected entities.
Response :
{
"status": "Success",
"message: "Unconnected entities delete successfully"
}POST /get_duplicate_nodes
The API is used to fetch duplicate entities from database.
API Parameters :
-
uri=Neo4j uri, -
userName= Neo4j db username, -
password= Neo4j db password, -
database= Neo4j database name,
Response :
{
"status": "Success",
"data": [
{
"e": {
"id": "13 September 2024",
"elementId": "4:b104b2e7-e2ed-4902-b78b-7ad1518ca04f:14007",
"communities": [
2969,
383,
81
],
"labels": [
"__Entity__",
"Date"
],
"embedding": null
},
"similar": [
{
"id": "20 September 2024",
"elementId": "4:b104b2e7-e2ed-4902-b78b-7ad1518ca04f:14153",
"description": null,
"labels": [
"__Entity__",
"Date"
]
}
],
"documents": [],
"chunkConnections": 0
}
],
"message": {
"total": 1
}
}POST /merge_duplicate_nodes
The API is used to merge duplicate entities from database selected by user.
API Parameters :
-
uri=Neo4j uri, -
userName= Neo4j db username, -
password= Neo4j db password, -
database= Neo4j database name, -
duplicate_nodes_list= selected entities list to merge of with similar entities.
Response :
{
"status": "Success",
"data": [
{
"totalMerged": 2
}
],
"message": "Duplicate entities merged successfully"
}POST /drop_create_vector_index
The API is used to drop and create the vector index when vector index dimesion are different.
API Parameters :
-
uri=Neo4j uri, -
userName= Neo4j db username, -
password= Neo4j db password, -
database= Neo4j database name, -
isVectorIndexExist= True or False based on whether vector index exist in database,
Response :
{
"status": "Success",
"message": "Drop and Re-Create vector index succesfully"
}POST /retry_processing
This API is used to Ready to Reprocess cancelled, completed or failed file sources. Users have 3 options to Ready to Reprocess files:
-
Start from begnning - In this condition file will be processed from the begnning i.e. 1st chunk again.
-
Delete entities and start from begnning - If the file source is already processed and have any existing nodes and relations then those will be deleted and file will be reprocessed from the 1st chunk.
-
Start from last processed postion - Cancelled or failed files will be processed from the last successfully processed chunk position. This option is not available for completed files.
Ones the status is set to 'Ready to Reprocess', user can again click on Generate graph to process the file for knowledge graph creation.
API Parameters :
-
uri=Neo4j uri, -
userName= Neo4j db username, -
password= Neo4j db password, -
database= Neo4j database name, -
file_name= Name of the file which user want to Ready to Reprocess. -
retry_condition= One of the above 3 conditions which is selected for reprocessing.
Response :
{
"status": "Success",
"message": "Status set to Ready to Reprocess for filename : $filename"
}POST /metric
The API responsible for a evaluating chatbot responses on the basis of different metrics such as faithfulness and answer relevancy. This utilises RAGAS library to calculate these metrics.
API Parameters :
-
question= User query for the chatbot -
context= context retrieved by retrieval mode used for answer generation -
answer= answer generated by chatbot -
model= LLM model -
mode= Retrieval mode used for answer generationRetrieval mode used for answer generation
Response :
{
"status": "Success",
"data": {
"graph+vector+fulltext": {
"faithfulness": 1.0,
"answer_relevancy": 0.9699
}
}
}POST /additional_metrics
The API responsible for a evaluating chatbot responses on the basis of different metrics such as context entity recall, semantic score, rouge score. This reuqire additional ground truth to be supplied by user. This utilises RAGAS library to calculate these metrics.
API Parameters :
-
question= User query for the chatbot -
context= context retrieved by retrieval mode used for answer generation -
answer= answer generated by chatbot -
reference= ground truth/ expected answer provided by user -
model= LLM model -
mode= Retrieval mode used for answer generationRetrieval mode used for answer generation
Response :
{
"status": "Success",
"data": {
"graph_vector_fulltext": {
"rouge_score": 1.0,
"semantic_score": 0.9842,
"context_entity_recall_score": 0.5
}
}
}POST /fetch_chunktext
The API responsible for a fetching text associated with a particular chunk and chunk metadata.
API Parameters :
-
uri=Neo4j uri, -
userName= Neo4j db username, -
password= Neo4j db password, -
database= Neo4j database name -
document_name= Name of document for which chunks needs to be fetched. -
page no= page number for multipage
Response :
{
"status": "Success",
"data": {
"pageitems": [
{
"text": "By T. Albert Illustrated by: maaillustrations.com Science has never been so much fun. Here is all that a child needs to know about water, rain, hail, sleet and water cycle. When Professor Mois Ture teaches- little readers read, learn and ask for more….. Published by Monkey Pen Ltd Dear Supporter, Thank you for downloading our childrens books. Monkey Pens Vision is to provide thousands of free childrens books to young readers around the globe. Please share our books with your friends and family to support our mission. Thank you Please make a donation on Patreon to support Monkey Pens Free Book Project: Hi, I am Professor Mois Ture and I will be telling you about water. You can call it RAIN. You can call it SNOW. You can call it SLEET. You can call it HAIL. But it’s WATER all the same. Did you ever wonder how",
"position": 1,
"pagenumber": 1
},
{
"text": " it HAIL. But it’s WATER all the same. Did you ever wonder how old water is or where it comes from? The answers may surprise you. The next time you see a pond or even a glass of water, think about how old that water might be. Do you really want to know ? I thought you did. Did you brush your teeth this morning? Well, some of the water that you used could have fallen from the sky yesterday, or a week, or month ago. It’s pretty new. But, some part of that water is very old and was around during the time of the dinosaurs, or even longer. Or maybe it’s a little newer; like from the time when the Pharaohs were building pyramids. You see there is only a limited amount of water and it gets recycled. Yep! It keeps going round and round. We call it the “Water Cycle.” Yes – You",
"position": 2,
"pagenumber": 2
}
],
"total_pages": 1
},
"message": "Total elapsed API time 0.48"
}POST /backend_connection_configuation
The API responsible for create the connection obj from Neo4j DB based on environment variable and return the status for show/hide login dialog on UI
Response :
{
"status": "Success",
"data": true,
"message": "Backend connection successful"
}{
"status": "Failed",
"error": "Could not connect to Neo4j database. Please ensure that the username and password are correct",
"message": "Unable to connect backend DB"
}
POST /schema_visualization
User can visualize schema of the db through this API.
API Parameters :
-
uri=Neo4j uri, -
userName= Neo4j db username, -
password= Neo4j db password, -
database= Neo4j database name
Response :
{
"status": "Success",
"data": {
"nodes": [
{
"element_id": "-5374",
"labels": [
"Entity"
],
"properties": {
"name": "Entity",
"indexes": [
"id,description"
],
"constraints": []
}
},
],
"relationships": [
{
"element_id": "-44223",
"end_node_element_id": "-5411",
"start_node_element_id": "-5342",
"properties": {
"name": "OWNED"
},
"type": "OWNED"
},
]
},
"message": "Total elapsed API time 3.51"
}