Aurelian Desktop

Aurelian Desktop is a cross platform application that powers local inference of LLMs. It's inspired by gpt4all hoping to improve on inference performance and UI.

It uses tauri and SvelteKit for the desktop UI.

Rustformers/llm powers inference through a binding to ggml.

Roadmap

• Chat Interface through tauri commands / events.
• Model downloads through GUI
  • Register deep-link to allow downloads from HF card.
  • Something like this: aurelian://model_name=TheBloke/<model_name> would initiate a download model popup.
• Automatic Settings Optimization
  • If GPU but all layers can't be fit, add as many layers as possible.
  • If it can all fit in the GPU, completely offload through candle or burn.
    • Burn allows for torch backend allowing hf models to be loaded directly.
    • This would require a change in llm, the inf backend for the app.

Development

   npm i
   npm run tauri dev

The desktop should launch watching for changes in the src/ and src-tauri/.

First build will take decades but following reloads should be pretty quick.

Releases

Only way to use in through a build. For now, follow steps in Development