Skip to content

OpenAI endpoint and CPU Tier

Choose a tag to compare

View all tags
@liampetti liampetti released this 27 Jun 06:30
v2.2.0
b2367fd

Big Release! Here are just a few of the changes:

  • Web UI becomes central configuration point, simple docker compose up and then launch the browser to get up and running with the setup wizard.
  • CPU Tier is back, lots of testing and the best model configuration has been chosen (at least for my current setup).
  • OpenAI API can be used instead of the in-built LLM model. Offload your requests to an already running LLM (Note: you will lose the forced JSON GBNF ruling on this setup, so higher chance of bad tool calls etc.)
  • Too many other fixes and prompt tweaks to mention...
Assets 2
Loading
0 Join discussion