RunR 0.5.7

RunR is an open-source, self-hosted serverless platform on Kubernetes and Knative: functions, containers, sandboxes, WebAssembly and durable workflows, with an API, a web console, the runr CLI, SDKs, a Terraform provider and a GitHub Action. You install it on infrastructure you operate, connected or air-gapped. It is developed by BRF Teknoloji and released under the MIT License.

0.5.7 is the first release published in this repository. It contains 0.5.0, the first open-source release, and the patch releases 0.5.1 to 0.5.7 (0.5.5 is a security release); the notes of each follow below, and CHANGELOG.md has the complete history. Downloads, the Terraform provider registry and the documentation are on runr.sh.

Install

• Kubernetes (Helm): the chart comes from BRF's public Helm repository, without credentials:

  helm repo add brf-tech https://helm.brf.sh
  helm upgrade --install runr brf-tech/runr --namespace runr --create-namespace --set domain=runr.example.com

  See the README; helm show values brf-tech/runr lists every value. A pinned install adds --version 0.5.7 --set api.image.tag=0.5.7 --set ui.image.tag=0.5.7.
• Single host (Docker Compose, also air-gapped): compose/README.md.
• CLI: curl -fsSL https://runr.sh/install.sh | sh on Linux and macOS. The archives of every platform are at runr.sh/dl (https://runr.sh/dl/cli/v0.5.7/) and attached below; check them against checksums.txt (docs/cli.md#install).
• Terraform / OpenTofu: source = "runr.sh/brf-tech/runr". runr.sh serves the provider registry protocol; terraform init checks the archive against SHA256SUMS, signed with the RunR release key (key ID 2588846B241BC095) (docs/terraform.md#install).
• GitHub Action: uses: brf-tech/runr/actions/deploy@v0.5.7; it downloads the CLI from runr.sh (docs/github-action.md).
• Images: every image is published under ghcr.io/brf-tech/runr and pulls anonymously.

The documentation is at runr.sh/docs and in the documentation index here.

Upgrading

Upgrade in place with helm upgrade of the running release; never uninstall and install again. Back up the database first and read docs/upgrade.md.

• From 0.4.x: read the upgrade notes of 0.5.0 in CHANGELOG.md (roles and permissions, secrets encrypted at rest with APP_KEY), and those of 0.5.1, 0.5.2 and 0.5.4 below.
• Sign-in sessions (0.5.5): a sign-in session now ends 30 days after the sign-in (auth.loginTokenTtlDays). Sessions older than that end with the upgrade and their users sign in again; API tokens keep the expiry they were created with. See the 0.5.5 notes below.
• From 0.5.0: the upgrade moves every stored image reference from ghcr.io/runrsh to ghcr.io/brf-tech/runr.
• Terraform: change the provider source to runr.sh/brf-tech/runr and move the state once: terraform state replace-provider registry.terraform.io/brf-tech/runr runr.sh/brf-tech/runr (from 0.5.1 to 0.5.6), or registry.terraform.io/runrsh/runr as the old address (0.5.0).

Highlights of 0.5.0

• Functions, containers and WebAssembly: pluggable runtimes (a runtime.yaml manifest plus an image, in http, exec or persistent mode) with C, Rust, Bash and Q examples; WebAssembly functions on the runwasi Spin shim; uploaded images and .wasm modules (docs/runtimes.md, docs/wasm.md, docs/images.md).
• Deploy: git push deploys with GitHub App, GitLab and Forgejo integration and pull request previews; the runr CLI with runr dev, the Terraform provider and the GitHub Action (docs/git-deploy.md, docs/cli.md, docs/terraform.md, docs/github-action.md).
• Data and workflows: bindings for KV (Redis ACL users), object storage (Garage or an external S3 endpoint) and Postgres (CloudNativePG); durable workflows with Python and TypeScript SDKs (docs/bindings.md, docs/workflows.md).
• AI: GPU workers and GPU CaaS, one-click Hugging Face models online, offline or air-gapped, a built-in MCP server and hosted MCP servers, and the Sandbox API on Kata or gVisor with warm pools (docs/ai.md, docs/huggingface.md, docs/mcp.md, docs/sandbox-api.md).
• Security: roles and permissions (built-in and custom roles, groups, project access, SSO group mapping, scoped tokens), secrets encrypted at rest with rotation, an append-only audit log, a software supply chain (SBOM, vulnerability scan, cosign signatures, admission policy) and data residency (docs/permissions.md, docs/security.md, docs/supply-chain.md).
• Operations: OpenTelemetry tracing with Grafana Tempo, replay, uptime checks, a status page and alerts; white-label branding and self-service signup; billing calculation per tenant, with invoicing and payment left to your own billing system (docs/observability.md, docs/white-label.md, docs/billing.md).

Changes in 0.5.7 (2026-10-17)

Changed

• The CLI downloads from runr.sh. curl -fsSL https://runr.sh/install.sh | sh installs the runr CLI on Linux and macOS (amd64, arm64): it reads the latest release from https://runr.sh/dl/cli/latest.txt (RUNR_VERSION pins one), checks the archive's SHA-256 against the release's checksums.txt and installs into /usr/local/bin when that is writable, else into ~/.local/bin (RUNR_INSTALL_DIR sets another directory). The archives of every platform, Windows included, are under https://runr.sh/dl/cli/<tag>/. runr version --check-update (and update_check: true) reads the latest release from that latest.txt (docs/cli.md).
• Terraform provider address runr.sh/brf-tech/runr. runr.sh serves Terraform's provider registry protocol: with source = "runr.sh/brf-tech/runr", terraform init downloads the archive for the platform and checks it against the release's SHA256SUMS, signed with the RunR release key (key ID 2588846B241BC095). A configuration that used brf-tech/runr changes source and moves its state once with terraform state replace-provider registry.terraform.io/brf-tech/runr runr.sh/brf-tech/runr (docs/terraform.md).
• The GitHub Action downloads the CLI from runr.sh. cli-base-url defaults to https://runr.sh/dl/cli and version: latest is the tag in <cli-base-url>/latest.txt; the github-token input is gone (a workflow that still sets it gets an "unexpected input" warning) (docs/github-action.md).
• Documentation on runr.sh. The documentation is published at https://runr.sh/docs/ and the security policy at https://runr.sh/security/.
• The Helm chart comes from BRF's public Helm repository. The chart runr is installed without credentials: helm repo add brf-tech https://helm.brf.sh, then helm upgrade --install runr brf-tech/runr --set domain=…; the install, upgrade, key rotation and white-label examples use the chart brf-tech/runr instead of ./helm of a repository checkout (README, docs/upgrade.md).

Changes in 0.5.6 (2026-10-17)

Changed

• The CLI's own API token expires. runr login --with-password created its runr-cli@<host> API token without an expiry. The token now expires 90 days after the login; --token-ttl sets another lifetime (30d, 720h; 0 or never keeps a token without expiry). runr login and runr whoami show when it expires (token_expires_at in their JSON output, and in the context of the config file); once it has expired, a command the API refuses says that the token of the context expired and to log in again, and runr logout no longer tries to revoke it. A new runr login into a context revokes the token an earlier runr login --with-password created for it, as runr logout does: logging in again replaces a token from an earlier release with an expiring one. --expires of runr tokens create and --token-ttl refuse a fractional day count such as 1.5d instead of reading it as 1d (docs/cli.md).

Fixed

• ci.sh ml-runtimes --push pushed untested images. The push built the PyTorch and TensorFlow images in a separate builder and pushed them right away; the smoke test, and on a GPU host its GPU check, ran for local builds only. --push now builds every image into the local Docker, smoke-tests it like a local build (with the GPU check where docker run --gpus all gives a GPU, required with ML_SMOKE_GPU=1) and pushes only once every image passed: docker push of the image the smoke test ran, its image ID checked again before the push. --push --no-smoke is refused (docs/runtimes.md).
• A deleted user's tokens stayed in the database. An administrator deleting a user (DELETE /api/users/{id}) left the user's sign-in sessions and API tokens behind; no request could use them without the user, but the rows stayed. Deleting a user — by an administrator or, as before, by the user (DELETE /api/account) — now deletes every sign-in session and API token of the user in the same transaction as the account, then the user's MCP server sessions. In the multi-tenant edition, a deleted account's copies in the organizations it belonged to go as well, with its groups and role assignments there (only the memberships went before); a member removed from an organization loses the API tokens they created in it (these work in that organization only; sign-in sessions and other organizations' tokens stay), and deleting an organization deletes the tokens created in it. The daily php artisan tokens:prune also removes the tokens of users that no longer exist and, in the multi-tenant edition, the tokens created in an organization that no longer exists or whose user is no longer a member (a platform administrator's token stays), and the organizations' copies and role rows of users that no longer exist; an organization whose schema is missing is skipped. The rows earlier deletions left go after the upgrade without any action (docs/security.md).

Changes in 0.5.5 (2026-10-17)

Security

• Signing out did not end the session. From 0.3.0 to 0.5.4, POST /api/auth/logout answered "Logged out." and deleted the session's token, but the API's session tracking (client address, user agent and device of a token), which runs after every request, saved the deleted token again: the token stayed valid. The same kept the sign-in session alive that runr login --with-password ends once it has created its API token. The session tracking now updates the token's row and never re-creates it, and writes only when the address, user agent or device changed. Revoking sessions, deleting API tokens and disabling or deleting the account deleted their tokens as intended; tests now cover each of these paths.
• Sign-in sessions expire. A sign-in session — a password sign-in of the console or the CLI, an SSO sign-in, a self-service signup, an accepted invitation — ends 30 days after the sign-in (RUNR_LOGIN_TOKEN_TTL_DAYS, Helm value auth.loginTokenTtlDays; 0 keeps sessions until they are signed out). They had no expiry before. The lifetime counts from the sign-in for the sessions that exist at the upgrade as well: those older than 30 days end with the upgrade, among them the sessions that were signed out but stayed valid, and their users sign in again. Operators need to do nothing. A session signed out during the 30 days before the upgrade stays valid until it is 30 days old; its user can end it at once under Account → Active Sessions (Revoke). API tokens keep the expiry they were created with.
• Changing the password ends the user's other sign-in sessions (their browsers are signed out); the session that changed it and the API tokens stay.
• php artisan tokens:prune, scheduled daily, removes the sign-in sessions that ended and API tokens 30 days after they expired. API tokens cannot be named invitation-accept (nor api, as before): these names mark sign-in sessions, and the API token list no longer shows the session of an accepted invitation, which Account → Active Sessions lists (docs/security.md).

Changes in 0.5.4 (2026-10-17)

Upgrade notes

• The last region workloads on RunR's earlier image names move too. region:reconcile (step image-registry) left the node installers of Kata (with Firecracker's devmapper setup) and gVisor and the region's queue Redis on ghcr.io/runrsh / ghcr.io/brf-tech until their next install. Their pod templates now move to ghcr.io/brf-tech/runr as well, and their running pods keep running: the installer DaemonSets get the new images together with the update strategy OnDelete (a new pod would re-run the install on its node; kata-deploy's cleanup and reinstall restart the container runtime); the queue Redis, whose streams, consumer groups and KV bindings' data live in memory only, keeps its pod — the Deployment is paused while its ReplicaSet takes the new image, then takes it as well and is resumed without starting a pod. A pod created later (a new node, a deleted or evicted pod) starts from ghcr.io/brf-tech/runr. A gVisor install, a Kata install that re-runs kata-deploy and Kata's Firecracker setup apply RollingUpdate again; Kata's rollout status counts the pinned installer pods by their config hash, and its in-place drop-in update keeps them pinned (docs/upgrade.md).
• hf-serve 1.1.1. The defaults move to ghcr.io/brf-tech/runr/hf-serve:1.1.1 and …:1.1.1-cuda (ai.images.hfServe, ai.images.hfServeCuda). Installations that set these values, for example to a registry mirror of an air-gapped region, update them and mirror the new images.
• Components on dedicated workers restart once. region:reconcile (step dedicated-worker-toleration) gives the registry, Prometheus, Loki, Tempo and the data services the toleration of the dedicated-worker taint where a pod of them runs on a dedicated worker; that pod restarts once, on the same node (below, Fixed). Components on the other nodes keep running as they are. php artisan region:reconcile --dry-run lists them.
• exec-tensorflow:2.21-gpu is published again under the same tag (below, Fixed). A function on tensorflow 2.21-gpu runs on the new image from its next deploy: Knative resolves the tag to the new digest for the new revision. Installations that mirror RunR's images copy the tag again.

Changed

• hf-serve starts a cached model without the Hub. In online mode hf-serve asked the Hub at every start, also when the pinned commit was already on the model volume, so a start from zero depended on the Hub's reachability and latency. After a load of a pinned commit it now records the load next to the cached repository (per engine, task and pipeline arguments), and a later start with the same settings loads the cached snapshot directly; when that does not load, it loads from the Hub as before. A revision that is a branch or tag is still resolved by the Hub at each start (docs/huggingface.md).
• The GPU case of ci.sh test-ai leaves Docker's default runtime alone. The Kind worker with the GPU gets the driver's files that nvidia-container-cli list names, mounted read-only (Kind's privileged nodes see the host's /dev/nvidia* already), so a host with the NVIDIA driver and Container Toolkit runs the case as it is configured. Where the NVIDIA runtime is Docker's default runtime and accepts device requests as volume mounts, the worker gets the GPU that way, as before (docs/ai.md).
• The GPU case of ci.sh test-ai needs far less disk. Its check image is the new GPU CaaS example (below) instead of CUDA PyTorch (3.3 GB compressed), it goes into the Kind GPU worker only instead of every node, and the host's copy is removed once the worker has it. kind load writes its temporary archives to RUNR_E2E_TMPDIR (default /var/tmp) instead of /tmp, which is a RAM-backed tmpfs on some hosts. Before the cluster is created the suite adds up what the run writes, from the built images' sizes and per Kind node, plus RUNR_E2E_GPU_RESERVE_GB (default 12) for the rest; it does not start when Docker's root has less free and prints both figures. This replaces the fixed RUNR_E2E_GPU_MIN_FREE_GB. The case now also asserts that the container ran the kernel on the GPU and returned the CPU's result (docs/ai.md).
• The GPU CaaS example runs a CUDA kernel in a 43 MB image. examples/ai/gpu-caas was CUDA PyTorch with a small inference: 3.3 GB compressed, about 9 GB unpacked. It is now Python's slim image with a standard-library server that runs a SAXPY kernel through the CUDA driver API (the driver compiles the kernel's PTX for its GPU, so no CUDA toolkit is in the image) and compares the result with the CPU's: 43 MB compressed, 134 MB unpacked, for NVIDIA GPUs from compute capability 5.0 on. The answer keeps cuda, device and device_count; the PyTorch fields give way to compute_capability, memory_mb, driver_cuda_version, computed_on, checksum, expected_checksum and result_ok (examples/ai/gpu-caas).
• ci.sh ml-runtimes smoke-tests the -gpu images on the GPU. On a host where docker run --gpus all gives containers a GPU, the handler of a -gpu image's smoke test runs with it, over HTTP and as a scheduled run: in both, the framework has to see the GPU and run a mean and a matrix product on it. Without a GPU the CUDA build is checked as before and the output says the GPU check was skipped; ML_SMOKE_GPU=1 requires it, ML_SMOKE_GPU=0 leaves it out (docs/runtimes.md).
• Pinned helm and kubectl in the API image. The image took kubectl from the latest stable Kubernetes release at build time, so two builds of one tag could differ. helm v3.16.0 and kubectl v1.35.9 are now pinned and checked against their published SHA-256 for amd64 and arm64, and the downloads retry on network errors.
• ci.sh test-ai keeps etcd in memory. The suite sets KIND_ETCD_IN_MEMORY=1 unless it is set: etcd keeps its data on the Kind control-plane's tmpfs. On a GPU host whose disk other jobs kept busy, the region setup's Knative Operator install failed with etcdserver: request timed out; with etcd in memory the same run got past it. KIND_ETCD_IN_MEMORY=0 puts etcd on the disk again (docs/ai.md).
• Organization quotas overcommit scale-to-zero workloads. In the multi-tenant edition an organization's CPU and memory quota on a region splits into the same two budgets as the region capacity, with the region's overcommit factors. Always-on projects and model services, the Kata pod overhead, queue consumers, sandboxes and warm pool pods count in full against the quota; one pod of every scale-to-zero project, model service and git deploy preview counts against quota × the factor (CPU 3, memory 1.5 by default) minus what the always-on part holds. A factor of 1 on a region keeps the quotas there hard. The quotas on a region still add up to at most its capacity, stored quotas stay as they are, and billing is unchanged. A refusal (422) names the budget that is used up (budget: always_on or faas) and what it has free (docs/architecture.md).

Fixed

• The region registry stopped after a worker selection. Zot installed before a region's workers were selected had no toleration of the dedicated-worker taint. Its node-bound volume (local-path, k3s's default class) keeps it on its node, and when that node was selected as a dedicated worker (runr.dev/dedicated=projects:NoSchedule), the next Zot pod — after the restart of region:reconcile's registry-sync-config, the roll of image-registry, registry:rolling-restart or a Helm upgrade — stayed Pending ("untolerated taint" and "volume node affinity conflict"): builds failed and the registry pre-warm Job gave up after 600 s. Prometheus, Loki, Tempo and the data services (KV Redis, Garage, the CloudNativePG clusters) had the same exposure. All of them now tolerate the taint and prefer the nodes that are not dedicated workers. The worker selection adds the toleration to such a component whose pod runs on a selected node, and region:reconcile (new first step dedicated-worker-toleration) to those on any dedicated worker; a pod Pending on the taint is replaced, as a StatefulSet does not replace a pod that never became ready. The registry restart of registry-sync-config carries the toleration as well (docs/architecture.md).
• The registry and Knative after deselecting workers. A registry installed while dedicated workers were selected, its node mirror and Knative's system deployments select the workers by the label runr.dev/dedicated=projects. Once the last worker was deselected their next pods found no node, and a registry whose volume sat on a deselected node could not start there again. The worker selection and region:reconcile (step dedicated-worker-toleration) now move such a registry to any node (the toleration kept, the other nodes preferred, a Pending pod replaced), apply the node mirror for every node once no worker is left — projects then run on any node and pull from the registry there — and drop Knative's nodeSelector (docs/architecture.md).
• RunR's own pods on a node selected as a dedicated worker. When RunR runs in a cluster it also uses as a region, the node holding PostgreSQL's volume or the API storage could be selected as a dedicated worker; its taint then kept the next pods (a restart, an upgrade) from starting there. Every pod of the chart — the Deployments and the migration and bootstrap hook Jobs — now tolerates runr.dev/dedicated=projects:NoSchedule and prefers the nodes that are not dedicated workers (placement.tolerateDedicatedWorkers, on by default). New values placement.tolerations, placement.affinity and placement.nodeSelector apply to every pod; the chart adds its toleration, preference and the API storage colocation to them (docs/architecture.md).
• The registry pre-warm Job's log. While the registry did not answer, every try printed wget's "can't connect to remote host: Connection refused". The Job now prints one line per wait instead: waiting for registry http://… to answer: next try in 4 s (2 s waited).
• Executor images in the Kind cluster of ci.sh test. The preload of the default executor images matched brf-tech/… and loaded none since RunR's images moved to ghcr.io/brf-tech/runr/. It loads the default general runtimes from there again; an ML runtime (several GB) is pulled by the node that runs it.
• The TensorFlow -gpu runtime ran on the CPU. In exec-tensorflow:2.21-gpu TensorFlow registered no GPU and ran every function on the CPU, without an error: it opens the CUDA libraries of NVIDIA's pip packages by soname through its RUNPATH, which lacks the cuSOLVER directory, and one library it cannot open makes it skip all GPUs. The image now puts every site-packages/nvidia/*/lib directory into the dynamic loader's cache, whatever the entrypoint and a function's own LD_LIBRARY_PATH, and its build fails unless every library there is in the cache.
• diffusers model services and dtype. hf-serve passed the dtype to diffusers as torch_dtype, which diffusers 0.41 deprecates (removed in 1.0); with a dtype in RUNR_PIPELINE_KWARGS the load failed, because diffusers refuses both arguments together, and a dtype name such as "float16" fell back to float32. hf-serve 1.1.1 passes dtype to both engines, takes torch_dtype of the pipeline arguments as dtype, and gives diffusers the torch dtype a name stands for.
• pip's dependency conflict in hf-serve's GPU image. The PyTorch base image ships spin 0.18, a developer CLI that requires click below 8.4, and hf-serve's server brings click 8.5: the build printed pip's dependency-conflict error and pip check failed in the image. hf-serve 1.1.1-cuda leaves spin out, which nothing in the image uses, and its build fails unless pip check passes.
• GPU nodes of ci.sh test-ai. Every Kind node counted as a GPU node, because all of them are privileged and see /dev/nvidia0: on a GPU host each would have got the NVIDIA Container Toolkit, the GPU label and a --gpu-node argument. Only Kind nodes where nvidia-smi -L lists a GPU count now — the worker with the driver.
• Resource bars on overcommitted regions. The projects page added up the projects' reservations in the browser against the region capacity and capped the bar at 100 %, so a region holding more scale-to-zero reservations than its capacity looked full while it still admitted projects. "Total Resources" now shows the always-on and the FaaS overcommit budget per resource as the API admits against them, summed over the regions and per region, with the bars of Settings → Regions → (region) → Workers. In an organization a platform region shows the organization's quota there, split the same way (capacity with capacity_scope: "quota" in GET /api/settings), and no longer shows as unreachable.
• An organization's quota usage seen by platform administrators. Tenants → (tenant) → Quotas (GET /api/tenants/{id}/regions) counted the projects of the organization the administrator was working in, not those of the organization opened. It now counts in that organization's own schema and shows its quota's always-on and FaaS overcommit budgets (capacity per region) with the bars of the projects page.
• Quota refusals for a resource without a quota. A 422 of the organization quota (and a sandbox's quota_region) reported 9223372036854775807 as what was left of a resource with quota 0. It now gives remaining_cpu_m / remaining_memory_mb only for the resources with a quota and names only the resource that is short.
• The disk check of ci.sh test-ai counts the whole run. It counted hf-serve and the GPU image per Kind node plus a 12 GiB reserve, and let a GPU run start with 23 GiB free under Docker's root; that run was down to 5 GiB free by its first AI case, with the other AI cases, GPU support and the GPU case still ahead. The check now also counts each Kind node's own data (1 GiB: Kubernetes' images, copied into every node), the Compose stack (3 GiB) and, unless KIND_PRELOAD=0, the preloaded images on every node and the host (1.4 GiB each): about 35 GiB with the default Kind cluster, 25 GiB with KIND_WORKERS=1 KIND_PRELOAD=0. Its refusal names both options (docs/ai.md).
• A Kind e2e run stopped from outside cleans up. ci.sh test, its suites (test-ai, test-runtimes, …) and test-helm-upgrade remove their Kind cluster, Compose project, Kind lock and Cloud Provider KIND also when they are stopped: on TERM or HUP at once, also in the middle of a step, and on INT (Ctrl-C) once the running command has stopped. A run whose output pipe was stopped together with it ended at the cleanup's first line and left all of them behind; INT to the run while a command ran could let it go on with its next step; a stop during docker compose up left the Compose stack; and a run stopped while it waited for the Kind lock deleted the cluster of the same name that the lock holder used. The cleanup touches only what the run created, and when the run's output is gone it writes to ${TMPDIR:-/tmp}/<cluster>-cleanup.log (docs/development.md).
• A short hiccup of the Kubernetes API server no longer fails the Knative Operator install. The operator's Helm install failed on its first error, and with it the region setup: on a cluster whose etcd could not keep up with a busy disk, Helm got etcdserver: request timed out once and the region had no Knative. The install is now tried again on transient errors of the API server — etcd timeouts and leader changes, 5xx and 429 answers, timeouts and dropped connections on the way to it — five tries in all, 2, 4, 8 and 16 s apart, each retry in the operator's install log. Any other failure, Helm's own --wait running out included, fails the install at once, as before.

Changes in 0.5.3 (2026-10-16)

Fixed

• Tempo crash loop on storage that ignores fsGroup. With a storage driver that does not apply the pod's fsGroup, Tempo's volume stayed owned by root and Tempo restarted with "mkdir /var/tempo/traces: permission denied". Tempo gets an init container fix-permissions that hands /var/tempo to Tempo's user, as Loki's does; region:reconcile (step tracing) adds it to installed backends. A Tempo pod that crash-loops with the earlier template kept its StatefulSet from rolling: the step replaces it, and the new pod starts with the init container.
• Tracing requests on small regions. On a single-node region, Knative's own pods stayed Pending on "Insufficient cpu" once the tracing backend was installed. The collector and Tempo request 50m CPU each (100m before), in line with the other region add-ons; the limits are unchanged. region:reconcile (step tracing) moves installed backends to these requests, and the region capacity subtracts them like every other platform pod.
• Registry pre-warm while the registry restarts. The registry-prewarm Job started before image-registry rolled the registry, so every image failed with "Connection refused", and the failed Job kept later reconciles from starting another one for 24 hours. The step now runs last in region:reconcile; the Job waits for the registry to answer before it starts and before each further try of an image (three tries, backing off), a failed pod is retried twice, and a Job that failed is deleted and started again by the next reconcile.
• runr regions showed 0 projects to a tenant. A platform region assigned to a tenant answered without project_count in the tenant's view (GET /api/knative-regions). It now counts the tenant's own projects there.

Changes in 0.5.2 (2026-10-16)

Upgrade notes

• Image namespace ghcr.io/brf-tech/runr. Stored image references move automatically, from 0.5.1 (ghcr.io/brf-tech/<name>) and from earlier releases (ghcr.io/runrsh/<name>): Helm values (runr.image), the database (migration move_images_to_runr_namespace, tenant migration 0055, images:move-registry in the bootstrap job) and the region workloads (region:reconcile). A registry of your own that holds copies of RunR's images under their paths needs them under the new paths: copy them from ghcr.io/brf-tech/runr (docs/upgrade.md).
• ML runtime versions. PyTorch 2.2 / 2.3 and TensorFlow 2.15 / 2.16 and their -gpu versions, listed by the local and demo seeders only and never published as images, leave the catalog (migration retire_unpublished_ml_runtime_versions). Projects that name one keep running as deployed and take the runtime's default version at their next deploy. executor:sync lists the published versions: PyTorch 2.14 / 2.14-gpu, TensorFlow 2.21 / 2.21-gpu (docs/runtimes.md).
• Tracing backend on existing regions. region:reconcile (step tracing) installs the OpenTelemetry Collector and Grafana Tempo where tracing is enabled and the backend is missing — the region's own tracing mode, or for a region without one (Knative Serving installed by 0.4.x) the platform's observability.tracing.defaultMode while installWithServing is on — and moves installed backends to collector chart 0.176.0 (collector 0.162.0) and Tempo chart 3.2.0 (Tempo 3.1.0); the collector and Tempo pods roll once. Tempo chart 3.2.0 applies tempoRetentionHours (default 72 hours) to the deletion of old traces, which chart 3.1.0 left at Tempo's 14 days. To keep tracing off on regions without their own setting, set observability.tracing.installWithServing=false in the upgrade (docs/upgrade.md).

Added

• PyTorch and TensorFlow runtime images. ghcr.io/brf-tech/runr/exec-pytorch:2.14 (torch 2.14.1, torchvision 0.29.1) and …:2.14-gpu (the same torch for CUDA 12.6), ghcr.io/brf-tech/runr/exec-tensorflow:2.21 (TensorFlow 2.21.0, CPU build) and …:2.21-gpu (CUDA 12): Python 3.12 with Keras (TensorFlow), NumPy, SciPy, pandas, Matplotlib, scikit-learn and Pillow, on the Python runtime's handler server, for linux/amd64. runtime/ci.sh ml-runtimes [--push] [--list] builds them; a build fails unless pip check passes and the framework and its CUDA build match the tag, and a local build answers a tensor-op handler over HTTP and as a scheduled run. executor:sync reads a X.Y-gpu tag as the GPU version of X.Y. The node image pre-pull and the region registry pre-warm leave the ML runtime images out: the nodes their functions run on pull them (docs/runtimes.md).
• Billing events, newest first. GET /api/billing/events?order=desc lists the newest events first; before=<event id> bounds the list as after= does, and meta.next_before continues a newest-first list (meta.next_after an oldest-first one) (docs/billing.md). The console's billing settings read their 20 newest events with one request.
• Registry pre-warm. region:reconcile has the internal registry of each region fetch the base images of the release's function builds before a first build needs them (step registry-prewarm, a Job in the registry namespace) (docs/runtimes.md).
• Any sandbox command on a terminal. POST /api/sandboxes/{id}/exec/ws takes tty, cols and rows: the command runs on a terminal of that size, so shells, REPLs, editors and pagers work as in a terminal window. {"type":"resize","cols","rows"} frames change the size, stdin_close sends Ctrl+D, TERM defaults to xterm-256color, and the start frame and the answer say whether the command has a terminal. Without tty the exec WebSocket works as before (docs/sandbox-api.md).
• runr sandbox exec -it. -t runs the command on a terminal over the exec WebSocket; with -it on a terminal runr's terminal is raw while the command runs (every key, Ctrl+C included, goes to the command), the size follows the window (SIGWINCH on Linux and macOS, the console size on Windows), the terminal is restored on every way out and runr exits with the command's exit code (docs/cli.md).
• Exec WebSocket in the sandbox SDKs. Python Sandbox.exec_ws() / AsyncSandbox.exec_ws() and Node sandbox.execWs() / runCodeWs() run a command over the exec WebSocket: input goes to the command while it runs (send_stdin / sendStdin, close_stdin / closeStdin), the output arrives as events or through wait(), kill() stops the command, and with a terminal (tty, cols, rows) resize() changes its size. The Python SDK depends on websockets 13 or newer; the Node SDK uses the runtime's WebSocket (Node.js 22+, Bun, Deno) or its own client on Node.js 18 and 20 and keeps no dependencies (sdk/sandbox/python, sdk/sandbox/node).
• runr metrics live reads what a deployment uses right now — CPU, memory and requests per second from the region's Prometheus, with the averages of the last hour (--quick without them) — for the live deployment or --deployment, once or every --watch interval, -o json (docs/cli.md).
• Region lifecycle in the CLI. runr regions creates, checks, changes and removes regions (create, check, get, set, rm, status), saves a region's kubeconfig to a file only its owner can read (kubeconfig; --stdout prints it) and chooses the worker nodes projects run on (nodes set). It installs, adopts and removes Knative (knative install|uninstall|detect) and manages the queue Redis, the image registry (also regions created with one, and checks of their storage), the move to another cluster (migration), function-domain TLS, Skupper sites, the FaaS overcommit factors (capacity), demo projects and WebAssembly support (wasm [status]|install|uninstall). Kubeconfigs, passwords and keys come from files or stdin and never reach the trace; removals ask for confirmation unless --yes, and --wait follows background installs (docs/cli.md).
• Mirror fallback for Docker Hub's pull limit. When Docker Hub refuses a copy, ci.sh mirror-reqs and mirror-bases copy the image from mirror.gcr.io or public.ecr.aws/docker, by the digest Docker Hub reports for it, so the copy is the same image (REQS_UPSTREAMS sets the sources and their order). mirror-reqs takes image names to copy only those and exits 1 when a copy fails.

Changed

• Build deadline. Function build Jobs (Kaniko and WebAssembly toolchain builds) run up to BUILD_DEADLINE_SECONDS, 900 s by default; the build checks and the stale-build check follow it. A build that runs out of time while the region registry still fetches its base image is built once more, and its build_error says so meanwhile.
• Sandbox start. A starting sandbox pod is read every 150 ms (SANDBOX_START_POLL_MS, 100–200) for its first 30 s and once a second after that, so a sandbox is reported running at most one interval after its pod runs (docs/sandbox-api.md).
• Warm pools. A pod a pool refill starts can be claimed as soon as it runs; the next per-minute refill no longer has to mark it warm first (docs/sandbox-api.md).
• Model docblocks. A migrate runs ide-helper:models --write --reset only with APP_ENV=local and IDE_HELPER_POST_MIGRATE=true, so migrations in test containers leave the models alone (docs/development.md).
• Tracing images from the reqs mirror. The region collector and Tempo run ghcr.io/brf-tech/runr/reqs/opentelemetry-collector-contrib:0.162.0 and ghcr.io/brf-tech/runr/reqs/tempo:3.1.0, RunR's copies of the Docker Hub images; the Helm values observability.tracing.collectorImage / tempoImage point them at a registry of your own. collectorChartVersion / tempoChartVersion are empty by default: the versions RunR pins.
• Build base images from the reqs mirror. The WebAssembly build images, the Spin shim build, hf-serve's GPU image and the GPU images of the pytorch and tensorflow runtimes build on ghcr.io/brf-tech/runr/reqs copies of their Docker Hub bases (TinyGo, Node.js, Python, Rust, PyTorch, CUDA). ci/check-dockerfiles.sh (CI job check:build-scripts) fails on a Docker Hub base image.
• Image builds pull their base images. Every image build of runtime/ci.sh (local and --push, the release tars of build-deploy) and of compose/build.sh / compose/deploy.sh runs with --pull, so an older copy of a base image in the build host's Docker is never built on.
• Runtime versions from the image registry. executor:sync lists the versions of the built-in runtimes from the tags of their images under ghcr.io/brf-tech/runr (the registry's OCI distribution API, read anonymously) instead of Docker Hub's runrsh/exec-* repositories, so every version published in RunR's image namespace reaches the catalog, PyTorch and TensorFlow included. versionDiscovery.repository names the whole image repository (ghcr.io/brf-tech/runr/exec-python); one without a registry host is on Docker Hub (docs/runtimes.md).
• Image namespace ghcr.io/brf-tech/runr. RunR publishes every image under ghcr.io/brf-tech/runr, so the organization's other packages never share a name with RunR's: reqs/<image>, exec-<runtime>, hf-serve, image-upload, containerd-shim-spin keep their names there, runr-wrapper, runr-agent, runr-queue-consumer and runr-supply-chain become wrapper, agent, queue-consumer and supply-chain, and runr/api and runr/ui stay where they are. runtime/ci.sh pushes to REGISTRY=ghcr.io/brf-tech/runr by default, runr dev pulls from it, and the system images policy passes ghcr.io/brf-tech/runr/**.

Security

• ML runtimes. pip-audit finds no known vulnerability in the packages of the four ML runtime images. The PyTorch 2.3 package set they replace had 20 advisories in torch (among them CVE-2025-32434) and the TensorFlow 2.16 set one in protobuf that TensorFlow 2.16's protobuf < 5 pin cannot take the fix of; pip and setuptools are upgraded to 26.2.1 and 84.0.0. The PyTorch weights example (examples/ai/hf-python/weights) takes transformers 5.20.

Fixed

• Registry leftovers. Deleting a runtime deletes its images (runr-runtimes/<id>) from the registries of the regions it was built in. Builds no longer open an upload before they are built, so a failed build leaves no empty repository, and a daily job hands build repositories without a tag to Zot's garbage collection, which removes them.
• Sandbox API e2e. The Kata guest-size check creates its sandboxes with cpu and memory, so they have the size whose vCPUs it checks.
• Upgrade hook while PostgreSQL restarts. An upgrade that restarts the control plane's PostgreSQL (a new image, or its volume moving to another node) failed in the post-upgrade runr-bootstrap hook, which started before the database accepted connections. The hook now waits up to 4 minutes for the database, and its deadline is 10 minutes.
• WebAssembly build images on a docker-container builder. A buildx builder with the docker-container driver (ci.sh wasm --push, CI) gives every build step OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=unix:///dev/otel-grpc.sock, and Spin 4.1 exited with "Failed to initialize telemetry: invalid URI". The build steps of runtime/wasm-* run Spin with OTEL_SDK_DISABLED=true (a build argument: the images do not carry it).
• --push exited 1 after a successful push. ci.sh build, queue-consumer, wrapper, build-agent, test-image and supply-chain ended with a test that failed when pushing; they exit 0 once the push is done.

Changes in 0.5.1 (2026-10-16)

Upgrade notes

• Region capacity admission. Scale-to-zero projects and model services no longer hold the region capacity: each counts one pod against a FaaS overcommit budget of capacity × 3 (CPU) and × 1.5 (memory), set per region. Every existing region gets these defaults; a factor of 1 keeps every reservation against the capacity. Sandbox and model service pods are no longer subtracted as platform pods, so a region's capacity rises by what they request at its next status check (docs/architecture.md).
• Image namespace ghcr.io/brf-tech. RunR's images are published under ghcr.io/brf-tech — the same names, tags and digests as under ghcr.io/runrsh up to 0.5.0 — and the upgrade moves every reference an existing installation holds (exact prefix only; other registries stay): image values of the Helm release, also the ones --reuse-values keeps (chart helper runr.image); stored references in the central and every tenant schema (migration move_images_to_brf_tech_registry, php artisan images:move-registry in the bootstrap job); and the workloads in the regions (region:reconcile). Knative Services get a new revision that takes the traffic once it is ready; the other workloads roll once. The node installers of Kata and gVisor and the region's in-memory queue Redis keep running as they are, a restart would cost more than the move (docs/upgrade.md).
• Terraform. The provider's address is brf-tech/runr: change source and run terraform state replace-provider registry.terraform.io/runrsh/runr registry.terraform.io/brf-tech/runr once.
• Model images. The defaults move to TEI 1.9.4 (ai.images.teiCpu …:cpu-1.9.4, ai.images.teiGpu …:{arch}1.9.4) and hf-serve 1.1.0 (ghcr.io/brf-tech/hf-serve:1.1.0, …:1.1.0-cuda). Installations that set these values, for example to a registry mirror of an air-gapped region, update them and mirror the new images: TEI 1.5 can no longer download models from the Hugging Face Hub.

Added

• Why a model service does not start. While a model service deploys, RunR reads its pods and shows Kubernetes' reason next to the status — CrashLoopBackOff with the engine's last output, OOMKilled with the memory limit, ImagePullBackOff / ErrImagePull with the image and the registry's answer, Unschedulable with the scheduler's message — in the console, the API (status_reason, details in status_message) and the MCP tools (docs/huggingface.md).
• Model image pre-pull. Setting up a region's model storage pulls the hf-serve image onto the nodes projects run on, and creating or deploying a model service pulls its engine image onto the nodes it can run on (GPU nodes for GPU services), so a first start no longer waits for a half-gigabyte image (docs/huggingface.md).
• TEI on Blackwell GPUs. Compute capability 10.0, 12.0 and 12.1 get TEI's 100-, 120- and 121- images; any other GPU gets cuda-, which picks its kernels when it starts. TEI services answer with their served name in /v1/embeddings.

Changed

• Region capacity: always-on and FaaS overcommit budgets. Always-on projects (min_scale ≥ 1 or scale_to_zero: false) reserve (size + queue-proxy + Kata overhead) × max(1, min_scale) against the region capacity without overcommit, together with the Kata pod overhead of scale-to-zero projects, queue consumers, always-on model services, sandboxes and warm pool pods. Scale-to-zero projects and model services count one pod (size + queue-proxy) against capacity × the region's overcommit factor, minus the always-on reservations. Refusals name the budget and what is still free.
• Overcommit settings. GET / PUT /api/knative-regions/{id}/capacity (regions.view / regions.manage) shows both budgets and sets faas_cpu_overcommit and faas_memory_overcommit (1–10); Settings → Regions → (region) → Workers shows and sets them. The region resource carries the factors and a capacity object; used_cpu_m / used_memory_mb now include the sandboxes and may exceed the total on an overcommitted region. Capacity alerts measure the fuller budget. Audit event region.capacity.update.
• Sandbox API. A sandbox that needs a new pod, and every warm pool pod, needs room on the region's always-on budget; otherwise the create is refused with quota_capacity.
• Repository. The repository is github.com/brf-tech/runr: Go module paths github.com/brf-tech/runr/…, the GitHub Action brf-tech/runr/actions/deploy, CLI release downloads and update checks, the Terraform provider brf-tech/runr (mirror repository brf-tech/terraform-provider-runr), chart and SDK metadata.
• Images. Images under ghcr.io/brf-tech: the API and UI, the runtime and WebAssembly build images, the wrapper, region agent, queue consumer, image-upload, supply-chain, hf-serve and Spin shim images and the reqs mirror; the CLI's runr dev and every region component use them.
• Region reconcile. region:reconcile has two more steps: admission-policy brings the supply-chain admission policy of a region to this release's image globs, image-registry moves region workloads to RunR's image namespace; traefik-image also moves a Traefik of another version from ghcr.io/runrsh, and wasm-sandbox re-runs the WebAssembly installer for its shim image.
• Model sizes count the weights an engine loads. A Hub repository that holds one model in several formats (safetensors, PyTorch, TensorFlow, Flax, ONNX, OpenVINO) counts one of them: a 268 MB model gets 1536 MiB on CPU instead of 3328 MiB. Hub models already in the library are sized again the next time a one-click deploy reuses them (docs/huggingface.md).
• Model service threads follow the CPU limit. Model service containers get OMP_NUM_THREADS / MKL_NUM_THREADS = the CPU limit rounded up, and hf-serve reads its cgroup quota when they are missing: a 1-CPU hf-serve service no longer runs one PyTorch thread per node CPU.
• hf-serve 1.1.0 restarts on a failed model load. It writes the reason to the termination log and exits, so the pod retries with back-off and the service status shows why, instead of staying unready.

Security

• Dependency updates for published advisories: the console's code editor takes monaco-editor 0.57 (its bundled DOMPurify 3.4.15) and the console builds with Tailwind CSS 4 (it looks as before; braces, micromatch, chokidar, fast-glob and postcss-selector-parser leave the build tooling), the workflow SDK tests with Vitest 4 (development needs npm 11), every Go module requires go1.26.9 (GO-2026-6603 … 6617 in net/http, HTTP/2, net/textproto and crypto/tls) and the CLI takes golang.org/x/sys 0.48. npm audit reports one low advisory: DOMPurify's IN_PLACE mode in the copy bundled with Monaco, which does not use that mode. With Tailwind CSS 4 the console needs Safari 16.4, Chrome 111, Firefox 128 or newer.
• hf-serve 1.1.0 (the transformers / diffusers engine and the mirror and upload helpers): PyTorch 2.5.1 → 2.14.1 (19 advisories, among them CVE-2025-32434), transformers 4.49 → 5.20 (24), Starlette 0.52 → 1.8 with FastAPI 0.143 (5), huggingface_hub 1.33, pip 26.2 and setuptools 84 in the image (8); the GPU image is built on PyTorch 2.14.1 with CUDA 12.6 and diffusers 0.41. pip-audit finds no known vulnerability in the new image; hf-serve's tests run on pytest 9.1.

Fixed

• Kata overhead of the deployed RuntimeClass. Capacity counted a project's Kata pod overhead from the region's current sandbox mapping; it now follows the RuntimeClass the project (or model service) was last deployed with, so a mapping change counts from each workload's next deploy.
• Model services and sandboxes counted twice. The capacity refresh subtracted model service and sandbox pods as platform pods while admission counted them as well (a region showed 3800m used of 3405m). They are counted once now, from their records.
• TEI embedding services crash-looped with "relative URL without a base": the Hugging Face Hub answers downloads with a relative redirect that TEI 1.5 cannot follow. RunR runs TEI 1.9.4.