SKA: Structured Knowledge Augmentation for OgbujiPT

[uogbuji]

The understanding has been emerging, that knowledge graphs are a very high potential tool for improving LLM-based applications¹. One of the key areas where this technology is expected to help is in-context completion, i.e. providing the main information an LLM needs in order to generate a response. The most prevalent, current technique for in-context completion is in Retrieval Augmented Generation (RAG), where documents, or more usually fragments of documents relevant to the user query are gathered and included in context for the final LLM prompt. This technique is very useful, but suffers from some key limitations which have been widely discussed. At Oori, we believe that further augmenting LLM prompted with structured knowledge, possibly in combination with the usual RAG document fragments can give even stronger results. It allows the long-standing work in knowledge engineering to be applied to generative AI. Note: for this discussion I'm considering property graphs as a form of knowledge graph. We'd want to target all these approaches.

Once RAG emerged as a technique, many vendors and open-source projects emerged to make it easy to use it. A good example is privateGPT, which quickly earned tens of thousands of GitHub stars by providing a simple system where users could merely drop PDFs, word docs, HTML downloads, and such into a directory and "chat your documents". Even OgbujiPT implemented RAG among its earliest demos. I think one of the reasons knowledge-engineering enhancements to RAG have not yet become as popular is that there is no such easy project for applying them.

We should provide facilities for SKA, with or without accompanying RAG in OgbujiPT, in order to help speed up adoption of this technique which can really help[ people get even more value out of LLM use.

Footnotes

1. The value of knowledge graphs to generative AI has especially been noted by professionals such as
   @chimezie and I, who've been working in AI long before the Deep Learning Neural Network expansion (and Transformer revolution) of the 2010s. ↩

[chimezie]

It would be nice to provide a "chat my embedded vector DB" feature / library

[uogbuji]

I think "chat my embedded vector DB" is actually what we already have, RAG-wise, if I'm understanding you. What we don't have is "chat my embedded graph DB".

[uogbuji]

One of the trickier aspects of SKA would be the knowledge engineering to create high-quality graphs in the first place. Here are some ready sources of knowledge graph (KG) or property graph (PG) data that newcomers to structured data can leverage towards SKA.

• library.link—structured data converted from library catalogs (my own former project). Books, of course, but also electronic, audio, video, etc. resources. Here's a sample library landing page and a sample item: Dune, directed by Denis Villeneuve
• The Google Knowledge Graph—Google's general-purpose KG for smarter web search. Requires an API key.
• Wikidata—Wikimedia Foundation's (i.e. Wikipedia & more) official KG
• Local web structured data in RDF-A, JSON-LD or Microdata, usually following schema.org, but also Open Graph protocol, etc. No centralized credibility control in this case, but useful for web analysis, etc.
• Of course many of the entries in awesome-public-datasets are available in, or can be quickly converted to KG or PG

[uogbuji]

While more about the LLM interaction pattern than the style of provided context, the paper Graph of Thoughts: Solving Elaborate Problems with Large Language Models does illustrate how argument components organized into graph patterns enables more powerful reasoning in the LLM, and it references various knowledge engineering work.

[uogbuji]

I'm using the Backdrop Build program to inspire discipline, and perhaps some collab around the development of this SKA project. One of the things I've had to do is come up with a pragmatic user definition, at least to guide initial development. Here's what I have so far for that:

Ideal users will be: Developers in startups working on disruptive applications of AI, or product teams in SMBs looking to gain growth advantages. Focus industries include health care/life sciences, educational tech, public policy NGOs, publishing and media, where there are  available bodies of structured, engineered knowledge. For many such users, naive attempts to use AI technology give poor results. They may encounter hallucinations, or perhaps the rough material they try to provide to guide the LLM will be ignored, or wrongly applied. Such users seek solutions which are inexpensive, can be developed collaboratively, but adapted for proprietary usage, with a high degree of compatibility with various AI tools and UI modalities.

TODO: 1) Industry focus might be too broad. Maybe identify even narrower focus areas 2) Possibly address PII in HC/LS ("In industries such as health care, there are applications which do not involve processing PII."?)