Easy and flexible way to use online LLMs

[magillos]

I wanted be able to connect Aurscan to online LLMs easily and switch them (to run a few test with different models). LiteLLM allows that but it's an overkill, and requires editing config and restarting. It also has history of being compromised, which is makes it a bit ironic to use it for security scans.

So here comes LLamification.

It will let you to use Aurscan with an online provider (Openrouter, Nvidia NIM and others):
-install python-pyqt6 and python-aiohttp (available in official repos)
-set your e. variable with AURSCAN_OPENAI_URL="http://127.0.0.1:4001/v1/chat/completions" (default port, can be changed),
-clone the repo
-run app with python -m llamification -g, set up your provider and and pick the model, start proxy. That's it.

You will be able to change providers and models on the fly.
python -m llamification will run it in headless mode with the last used settings.

PKGBUILD and AUR package may arrive later.

(The app usage goes beyond Aurscan, of course. It should work whenever you need to disguise your online LLM as Ollama).

[magillos]

I uploaded to AUR: https://aur.archlinux.org/packages/llamification-git

And I had to:

scanning llamification-git (2 files) ...

[  OK  ] llamification-git  confidence 95%
         Standard VCS package building a Python GUI app from a GitHub repo; no suspicious network fetches, obfuscation, or credential access.
         [info] PKGBUILD: Source points to a GitHub repo matching the package name and description; no typo-squatting or personal domain.
             > source=("${pkgname}::git+https://github.com/magillos/LLamification.git")
         [info] PKGBUILD: Standard Python wheel build; no npm/bun/pip install of unrelated packages.
             > python -m build --wheel --no-isolation
         [info] PKGBUILD: Standard Python install into pkgdir; no writes outside srcdir/pkgdir.
             > python -m installer --destdir="${pkgdir}" dist/*.whl
         [info] PKGBUILD: Installs a .desktop file and icon; no systemd units, cron, or hooks.
             > install -Dm644 com.github.magillos.llamification.desktop ...
         [info] .SRCINFO: Version matches a git commit hash; no suspicious version strings.
             > pkgver = 8dca5b1
         ↳ tokens: ~2,321 in / ~334 out · cost n/a

scanner usage: 1 call(s) · tokens: ~2,321 in / ~334 out · cost n/a

[manticore-projects]

Great! Added a hint to the README https://github.com/manticore-projects/aurscan#backend-fallback-chain