Repository navigation
Podlight 0.3.0
Podlight 0.3.0
New
- GGUF models. Models that exist only as GGUF now run, with llama.cpp. Pick the quantisation when you add the model (Podlight suggests Q4_K_M); how much memory a token needs is read from the file's header without downloading the model. llama.cpp's engine is smaller, so new pods are often ready sooner (85 seconds in the measurement).
- Claude Code and Codex. Connection → Configuration files for experts now has Claude Code and Codex, each with the install command and a ready terminal command or shell function (
claude-podlight,codex-podlight) that leaves your normal setup alone. Podlight passes through the Anthropic Messages API and the OpenAI Responses API they need, on both engines. Anthropic does not officially support other models in Claude Code; Podlight says so next to the command. - Overview. Podlight opens with an overview: what is running and what it costs, the next steps to a connected agent, spending against your limits, your models with Start, and the address to connect.
- Location per model. Where a model's pods may run (anywhere, EU, EU and EEA, Secure Cloud) is chosen when you add it, at the top of the card step. Settings hold the default for new models.
- Start where you look. A stopped model shows a large Start button right under its status, and adding a model ends with "Add and Start…".
Fixed
- Changing the context while adding or editing a model now changes the card selection with it: the suggestion is followed unless you picked a card yourself, a card that needs more of itself for the longer context moves up, and a card that no longer fits gives way, with a note. Editing no longer jumps back to the saved card. The memory the model needs for the chosen context is shown above the list.
- The model's detail shows the card under its catalog name and the price as "card + storage", so it matches the list.
- Speed estimates for B300, A100 40 GB, RTX 3070 and more cards that had none.
Measured
On RunPod, both engines served chat with tool calls, the Anthropic Messages API (with tool use, streaming and token counting) and the Responses API (with a function call) through Podlight: llama.cpp with a GGUF model, ready after 85 seconds, and vLLM, ready after 308 seconds (measurement). A full Claude Code or Codex session against a large model was not part of it.
Deutsch
- GGUF-Modelle laufen jetzt, mit llama.cpp. Die Quantisierung wählst du beim Anlegen (Vorschlag Q4_K_M); den Speicher pro Token liest Podlight aus dem Dateikopf, ohne das Modell zu laden.
- Claude Code und Codex: unter „Verbindung“ → „Konfigurationsdateien für Fortgeschrittene“ mit Installationsbefehl und fertigem Terminalbefehl bzw. Funktion (
claude-podlight,codex-podlight). Podlight reicht die Anthropic-Messages-API und die Responses-API durch. Anthropic unterstützt andere Modelle in Claude Code nicht offiziell; Podlight sagt das dazu. - Übersicht als Startbildschirm: was läuft und was es kostet, die nächsten Schritte, Ausgaben, Modelle mit „Starten“, die Adresse für Agenten.
- Standort pro Modell, gewählt beim Anlegen oben im Kartenschritt. Die Einstellungen sind der Standard für neue Modelle.
- „Starten“ dort, wo man hinschaut: großer Knopf unter dem Status eines gestoppten Modells, und „Hinzufügen und starten …“ am Ende des Anlegens.
- Behoben: Ändert sich der Kontext, folgt die Kartenauswahl; beim Bearbeiten springt sie nicht mehr zurück. Über der Liste steht, wie viel Kartenspeicher der Kontext braucht. Karte und Preis im Detail stimmen mit der Liste überein.