Skip to content

[FEAT]: improve onboarding for NPU chips #4989

Description

@flatlines

What would you like to see?

Hello there,

I am running a qualcomm snapdragon chip and have just installed and run AnythingLLm for the first time. I am a long time localLM user. A few observations:

  1. the programme auto-detects the Qualcomm chipset and suggests loading the NPU compatible models during installation, this is great
  2. the programme appears to launch with all models loaded, but it actually doesn't download the models at all. It isn't obvious to a first time user that the models are not loaded, because there's no visual indicator in the GUI.
  3. there is some documentation here that explains where and how to download the models (https://docs.anythingllm.com/manual-qnn-model-download) - this documentation links to the anythingllm directory on a second page, rather than in-page
  4. within the programme, the GUI list drop-down shows all 4 available NPU models, even though I only had one downloaded... while it was still in zip format
  5. once I had extracted the one model I had manually downloaded, the GUI still suggested 4 might be available on my machine

Some suggestions:

  • Have the GUI reflect the reality of what is happening on the machine, rather than a pre-set drop-down list associated with a perfect installation
  • If no models are installed, prompt the user to access the docs for manual download from within the GUI
  • offer users the option of what models to install during installation - for example, my machine seems to have auto-installed your ollama models folder, even though I never selected or asked for this

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions