Skip to content

v0.5.0 — LiteLLM now records usage; failing subscriptions are saved and retryable

Choose a tag to compare

@sagarsuperuser sagarsuperuser released this 12 Oct 19:37
· 368 commits to main since this release
84408e5

One day after 0.4.0, with one fix that matters to anyone using LiteLLM, two requests from a user, and the first of the audit fixes.

If you use the LiteLLM integration, read this first

Before this release, Velox recorded nothing from a real LiteLLM proxy. A proxy logs the caller as end_user and puts token counts at the top level of each record. Velox read only user and usage, so it refused every record inside a 200 response. It now reads both shapes.

  • Usage sent before you upgrade was not stored and cannot be recovered from Velox.
  • The guide was checked against LiteLLM 1.83.9 with a real proxy: LiteLLM integration.

Behaviour changes

  • POST /v1/billing/run skips a subscription whose billing is failing until its wait has passed. Use the new retry endpoint to try one now.
  • velox_billing_due_subscriptions and velox_billing_oldest_due_age_seconds no longer count failing subscriptions. The new velox_billing_failing_subscriptions does. Update alerts that relied on the old meaning.
  • Run migrations first, as for any upgrade. This release adds five columns to subscriptions.

What changed

Billing

  • A subscription whose billing fails now keeps a saved record: how many attempts failed, when, and when the next attempt is due. It waits 15 minutes, then 1 hour, 4 hours, 12 hours, then a day at a time, and is set aside after eight failures.
  • POST /v1/subscriptions/{id}/retry-billing clears the failure and bills at once. The retry is written to the audit log.
  • GET /v1/subscriptions?billing=failing lists the failing ones.
  • A lost database connection or a shutdown mid-run does not count as a failure.
  • A panic while billing one subscription no longer stops every later billing run.

Credits

  • GET /v1/public/cost-dashboard/{token} now returns a credits object: the balance, what is left of the prepaid commit and of promotional credit, and an estimate of what remains after this period's usage so far. A customer-facing page can show a balance without a secret key. Requested in #86.

Invoices

  • The buyer's tax ID no longer prints on top of the tax line in the invoice PDF. It appears once, under the buyer's name. Reported in #164.

Running it

  • Release images are built for ARM as well as Intel/AMD, so docker pull works on Apple Silicon Macs and ARM servers.
  • Dashboard build tooling updated; 28 of 35 dependency advisories cleared.

Docs

  • Nine sentences that claimed more than had been shown are corrected, including the README line about several businesses in one install.
  • Thanks to @KeuqS for correcting two claims on the features page (#990).

Every change, with its reasoning: CHANGELOG.