ChatObsidian

ChatObsidian is an Question and Answer AI integration for a folder of markdown notes.
Intended for Obsidian notes vault, but you could use it for any local folder of markdown files.
This app is for private use only. you can't upload files, and you should not run it in public. maybe on a closed local network, but even then maybe add basic auth or something.

This is very much a work in progress project, but some parts work ok already. You'll probably need to reindex after syncs of the app repo:

• index/query with chromadb and gptsimpleindex works correctly. Tested with up to 10000 notes in Obsidian vault. will likely remove gptsimple soon, as chroma works well now.

• notes are indexed locally, so they're not all sent to OpenAI.

• Semantic Search is local only, so does not require OpenAI key... indexing doesn't use it either, so any string prob works.

• ...but notes related to each Query from Query page are sent to OpenAI API for final synthesis. If you think this is a privacy issue, then don't use it. OpenAI data policy

• EXPERIMENTAL: embedding model selection. only 3 model available on ui. need to add to enum list in model_settings.py

---

run locally with

pipenv shell \
pipenv install \
export OPENAI_API_KEY=YOUR_API_KEY \
streamlit run ChatObsidian.py

TODO

• vector database integration (chroma)
• gpt3.5-turbo support (cost reduction)
• local embeddings model support (langchain/huggingface) (cost reduction, privacy)
• open sources in obsidian via uri
• multi collection support (collection per folder. saves embedding model used)
• just semantic search of files (no_text config)
• Local embedding model switching (multilingual model)
• add custom embedding models from ui. (maybe save to tinydb collection)
• refresh files, instead of delete and reindex
• local LLM support (free, privacy) Explore Alpaca 7B, when it becomes available in LangChain
• select and configure LLM on sidebar (model and prompthelper)
• filetypes support support to folder config. checkbox for simpledirectoryreader supported filetypes. save to collection settings.
• image support
• docker support (folder mapping would suck a bit. maybe map to a default './data/obsidianvault')