Support for Claude 3?

[Neurrone]

Hey, I know this could be a big ask depending on how the add-on's code was structured.

Would you consider adding support for the Claude 3 API as an alternative backend? It could allow for convenient experimentation and comparisons between different APIs.

I'm unfortunately unable to use OpenAI because every credit card I've tried gets declined and the customer support chat is not being manned by a human, probably just a gpt since I've not gotten any response.

[AAClause]

Hello,
Have you tried with OpenRouter?
Unfortunately, Claude is unavailable in my country, so I'm unable to integrate their API into the add-on.

[Neurrone]

Thanks, I didn't know something like that existed.

I've funded my account there with $5 but even after generating an API key and entering it in settings, nvda+g still says I need to enter an API key.

[Neurrone]

Also, does the add-on support customizing which model to use the API with? E.g, using a specific model for image recognition and another one for chats

[AAClause]

Thanks, I didn't know something like that existed.
I've funded my account there with $5 but even after generating an API key and entering it in settings, nvda+g still says I need to enter an API key.

Please make sure you are selecting the correct service provider. It is likely that you are using OpenAI. Within the list of models, you’ll notice the Open Router models are mentioned subsequent to those from OpenAI.

Here are examples of what you'll find:

gpt-3.5-turbo [provider: OpenAI. context window: 16385. max output tokens: 4096]
....
Airoboros 70B [provider: OpenRouter. ID: jondurbin/airoboros-l2-70b. context window: 4096]
...

[AAClause]

Also, does the add-on support customizing which model to use the API with? E.g, using a specific model for image recognition and another one for chats

The add-on currently remembers the last model used for chats. However, it does not provide the option to choose the default model/provider for image recognition. This is a great suggestion for improvement! :)

[Neurrone]

How do I choose a model to use? In the OpenAI tab of the NVDA preferences dialog, I don't see any model options except the one under text to speech (tts-1 and tts-1-hd).

[AAClause]

In the main dialog (NVDA+g) there is a list box to select the desired model. Once you've triggered NVDA+g, press the Tab key to focus on this list.

[Neurrone]

I cant launch the dialog as it says i have to add an API key Get Outlook for iOS<https://aka.ms/o0ukef>

…

________________________________ From: André-Abush Clause ***@***.***> Sent: Sunday, March 10, 2024 2:32:42 AM To: aaclause/nvda-OpenAI ***@***.***> Cc: Dickson Tan ***@***.***>; Author ***@***.***> Subject: Re: [aaclause/nvda-OpenAI] Support for Claude 3? (Issue #69) In the main dialog (NVDA+g) there is a list box to select the desired model. Once you've triggered NVDA+g, press the Tab key to focus on this list. — Reply to this email directly, view it on GitHub<#69 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AD5VPGR55KZI322VSSLQJVDYXNIUVAVCNFSM6AAAAABEOE77NWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSOBWHE2DGNBVHE>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[AAClause]

Hmm, strange. If you have provided an API key for Open Router the dialog should open. Does the API key register correctly?

[Neurrone]

Does the API key register correctly?

It doesn't error when I exit the dialog and apply my settings, but is still not allowing me to open the main dialog with NVDA+g.

I left the organization name and key blank as well, since I didn't have one.

[AAClause]

You must supply at least one API key for a provider (currently Open AI, Open Router, and/or Mistral AI), to gain access to the main dialog interface. Therefore, you should generate an API key on the Open Router website and configure it within the add-on settings under the "OpenRouter API keys" dialog. The organization field is not required.

[Neurrone]

I've done that. Precisely, the steps I've done are:

1. Generated an OpenRouter API key. I've used curl to call the completions endpoint to verify that the key works.
2. In the settings -> OpenAI dialog, entered this key under OpenRouter API keys. I've left the organization name and key fields blank
3. Hit apply and ok to save.
4. Pressing nvda+g fails with the error message that no API keys have been added for any provider, even though I've already added the OpenRouter API key.

[AAClause]

OK, I just reproduced your manipulations on a fresh setup and it seems there is a bug. Sorry.
Please also fill in a fake API key for the Open AI provider. The dialogue should open afterwards.
I will fix this today. Thanks.

[Neurrone]

No problem, thanks a lot - that fixed it for me.

[mzanm]

Hello. Would you be willing to accept a PR to support Claude models? I think I could try to add it, I've found Claude to transcribe Arabic text way more accurately than gpt-4V, so I'd be interested in working on it if possible. Thanks.

[AAClause]

@mzanm - Sure, feel free to implement this! :) However, I recommend waiting for the PR #62 merge into master branch. Thanks!

[AAClause]

Hi @Neurrone, @mzanm, and all!
After some thought and feedback, I've decided to continue in a dedicated app instead of adding everything to NVDA.
There will remain an NVDA add-on but minimalist, which will communicate with the dedicated app. Useful for some features (e.g.: capturing objects).
The app will run in the background and could be invoked anytime.
Adding more features directly to NVDA is becoming too complex and unsuitable.
This is a new project based on this add-on. I hope to offer a first draft by the end of the month.

[rperez030]

I think that makes a lot of sense. It could even be made cross platform or integrate with other screen readers and applications more easily.

[Neurrone]

That's awesome, thanks a lot for the continued work. Look forward to trying it when its ready.

[AAClause]

Thanks for your interest. The repository has just been created with the first commit: https://github.com/aaclause/basiliskLLM