Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add local language models support #42

Open
kabachuha opened this issue Jun 1, 2023 · 1 comment · May be fixed by #43
Open

Add local language models support #42

kabachuha opened this issue Jun 1, 2023 · 1 comment · May be fixed by #43
Labels
enhancement New feature or request

Comments

@kabachuha
Copy link

kabachuha commented Jun 1, 2023

Hi, @vasilecampeanu !

While GPT-* models are awesome, there's an issue of privacy arising from using an external third party API, especially when the information may be sensitive. More than that, they don't allow as much customization for one's needs as the local models. In my mind, adding support of local models is one of the top priorities of such a plugin.

Fortunately, thanks for people like oobabooga making a wrapper for many possible local models like LLaMa, Vicuna, MPT, Galactica, etc it's pretty straightforward to implement. Just replace the call for OpenAI's API to oobabooga API's endpoint. For example, how I did this rizerphe/obsidian-companion#13 (may require some tweaks)

supports streaming mode as well

@kabachuha kabachuha linked a pull request Jun 3, 2023 that will close this issue
@vasilecampeanu vasilecampeanu added the enhancement New feature or request label Jun 3, 2023
@vasilecampeanu
Copy link
Owner

vasilecampeanu commented Jun 3, 2023

Hello, I'm sorry for the late response. Thank you for the pull request and for bringing up the issue. I intend to add support for local models, but I'm quite busy at the moment and haven't found the time to delve into it yet. Currently, my primary focus is to release the plugin to the community list.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
Status: Short Term
Development

Successfully merging a pull request may close this issue.

2 participants