Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

knowledge graph example #934

Merged
merged 43 commits into from
Apr 8, 2021
Merged

knowledge graph example #934

merged 43 commits into from
Apr 8, 2021

Conversation

julian-risch
Copy link
Member

@julian-risch julian-risch commented Mar 31, 2021 •
edited
Loading

This PR adds:

  • GraphDBKnowledgeGraph, which stores a knowledge graph in the form of triples and executes SPARQL queries
    • and its base class BaseKnowledgeGraph
  • Text2SparqlRetriever, which uses a pre-trained model to translate text queries to SPARQL queries
    • and its base class BaseGraphRetriever
  • Tutorial10_Knowledge_Graph, which shows how to do question answering on a knowledge graph with the help of a pre-trained model and a knowledge graph extracted from https://harrypotter.fandom.com/

closes #917 #918 #919

Limitations:
Unfortunately, there seems to be no easy way to run the tutorial in google colab notebooks because GraphDB cannot be installed automatically. GraphDB requires filling out a form to download installer files: https://www.ontotext.com/products/graphdb/graphdb-free/

The number of beams in beam search num_beams is not parameterized but fixed to 5.
The Text2SparqlRetriever is fixed to BartForConditionalGeneration models and could be generalized so that it works with any seq2seq model.

oryx1729 and others added 30 commits April 7, 2021 16:37
@oryx1729 @julian-risch
Add knowledge graph module
79394f7
@oryx1729 @julian-risch
Fix type hint
0a41328
@oryx1729 @julian-risch
Add graph retriver module
a997f4a
@Timoeller @julian-risch
Change type annotations, change return format
5625c5c
@julian-risch
Add graph retriever that executes questions as sparql queries
d265267
@julian-risch
Linking only those entities that are in the knowledge graph
0cef826
@julian-risch
Added logging and using relations extracted from Knowledge graph for …
17668e6 
…linking
@julian-risch
Preventing entity linking from linking the same token to multiple ent…
4c171b5 
…ities
@julian-risch
Pruning triples that have no variables for select and count queries
8aea754
@oryx1729 @julian-risch
Support knowledge graphs with Pipelines
0643f97
@Timoeller @julian-risch
Add text2sparql
0e3b4ba
@julian-risch
Entity linking and relation linking consider more special cases now b…
cad63af 
…ased on evaluation on labelled data
@julian-risch
Separating example code from KGQA implementation
581ae95
@Timoeller @julian-risch
Add eval on combined extarctive and kg questions
8b9f270
@oryx1729 @julian-risch
Remove references to hp-test
ca78354
@Timoeller @julian-risch
Add fields sparql_query and long_answer_list to metadata
342bbb8
@julian-risch
Removing modular Question2SPARQL approach
3e57d8c
@julian-risch
Removing additional classes used for modular kgqa approach
b491985
@julian-risch
preparing lcquad data
5ee18a0
@julian-risch
change graph db
2f80bdb
@julian-risch
Translating namespaces in knowledge graph queries
ed1430b
@julian-risch
Creating graphdb index and loading triples from .ttl file
1f7e08a
@julian-risch
Fetching graph config files, triples and model from S3
5707644
@julian-risch
Fix incompatibility issues with BaseGraphRetriever and BaseComponent
20844af
@julian-risch
Removing unused utility functions
30a11ba
@julian-risch
Adding doc strings and tutorial header
765b787
@julian-risch
Adding sparqlwrapper dependency
0e8bcbb
@julian-risch
Moving tutorial header
b94c7ab
@julian-risch
Sorting tutorials by number within name of notebook
8c1e302
@github-actions @julian-risch
Add latest docstring and tutorial changes
7b66006
Timoeller
Timoeller approved these changes Apr 7, 2021
View reviewed changes
Copy link
Contributor

@Timoeller Timoeller left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good, once the proposed changes are in (or dismissed) I am fine from the Text2Sparql side. Lets get @oryx1729 feedback for the KG part.

haystack/graph_retriever/text_to_sparql.py Show resolved Hide resolved
haystack/graph_retriever/text_to_sparql.py Outdated Show resolved Hide resolved
haystack/graph_retriever/text_to_sparql.py Outdated Show resolved Hide resolved
tutorials/Tutorial10_Knowledge_Graph.py Outdated Show resolved Hide resolved
tutorials/Tutorial10_Knowledge_Graph.py Outdated Show resolved Hide resolved
tutorials/Tutorial10_Knowledge_Graph.py Show resolved Hide resolved
@Timoeller Timoeller requested a review from oryx1729 April 7, 2021 17:37
oryx1729
oryx1729 reviewed Apr 8, 2021
View reviewed changes
Copy link
Contributor

@oryx1729 oryx1729 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

@julian-risch julian-risch changed the title WIP: knowledge graph example knowledge graph example Apr 8, 2021
@julian-risch julian-risch marked this pull request as ready for review April 8, 2021 11:49
@julian-risch julian-risch merged commit d38c07e into master Apr 8, 2021
@julian-risch julian-risch deleted the knowledge-graph-example branch April 8, 2021 12:05
This was referenced Apr 8, 2021
Closed
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Timoeller Timoeller Timoeller approved these changes

@oryx1729 oryx1729 oryx1729 approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Implement Knowledge Graph database
3 participants
@julian-risch @Timoeller @oryx1729