Play with LLMs in Java
TheoKanning/openai-java is a comprehensive Java client for the OpenAI API. It is focused on the "real" OpenAI, but it can be easily extended to support other LLM providers that are compatible with the OpenAI API.
This project provides a wrapper around the OpenAiService class that makes it easy to work with any OpenAI-compatible LLM provider.
It was tested against OpenAI and Anyscale
See the Main class for an example of how to use the wrapper.
It does a couple of simple interactions against OpenAI as well as Anyscale. It then goes into an interactive chat session in the terminal (poor man's ChatGPT).
The Chat class supports the OpenAI API for function calling.
It works with OpenAI, but unfortunately fails at the moment with Anyscale (the function is never called).
Anyscale supports function calling for the following models:
- mistralai/Mistral-7B-Instruct-v0.1
- mistralai/Mixtral-8x7B-Instruct-v0.1
See https://docs.endpoints.anyscale.com/guides/function-calling for more details.
I verified that function calling works on Anyscale when using their Python sample code.
In general, when working with other LLM providers it's important to understand the differences between them and the OpenAI API. The API compatability might be partial, so you'll need to check the documentation.
Here is a migration guide for the Anyscale API: https://docs.anyscale.com/endpoints/model-serving/openai-migration-guide