v0.108.3

Fixed

• A service or job booting its image's own kernel (kernel: image) no longer needs
  decompressors on the host. gzip, xz and zstd kernels boot without them, fixing Debian
  kernel boots on nodes without xz-utils. lz4 and bzip2 kernels are rejected with an error
  naming the format.

Full Changelog: v0.108.2...v0.108.3

v0.108.2

Fixed

• A fleet job larger than its reservation no longer starves on its node. While it waits
  for the rest of its size, the node turns down new reservations instead of letting smaller
  jobs take the room it waits for.

Full Changelog: v0.108.1...v0.108.2

v0.108.1

Fixed

• vk-hub's built-in CI tools build to the same bytes on every node. Each build's git
  binaries used to differ, so nodes building the same definition reported different checksums
  for them.

Full Changelog: v0.108.0...v0.108.1

v0.108.0

Added

• Registry storage areas can grant fleet nodes access. vk-hub namespaces add --grant
  takes nodes, every node, or node <id>, one node, beside the job rules, on areas whose
  path has no placeholders; * still means every job only. Nodes get no access unless granted.
  The registry's access tab lists these rules by node.
• Removing a fleet node revokes its registry access. Registries managed by the hub refuse
  the removed node's tokens as soon as they hear of the removal, also after they restart.
• vk-hub issues fleet nodes their own registry access. A node granted access to the
  hub-managed registry it uses gets a token for it when it connects, renewed before it expires
  and replaced when the registry or the node's grants change; one no longer granted anything
  is told to drop it.
• Fleet nodes keep the registry access the hub issues them. A restarted node, and any
  vk run on it, has it while it is valid even with the hub unreachable; it goes when the hub
  withdraws it, the node joins again or is removed, or the hub names another registry.
• Fleet nodes reach the hub-managed registry with the access the hub issues them. A node
  with no registry credential of its own uses it for virtkit/ bundles, vk registry pull,
  the Docker relay, caches and build locks, and for a placed job's image pulls once the job's
  token has expired; access that has expired is reported rather than replaced by anonymous
  requests. A job's pipeline images and caches still need the job's own token, and fail once it
  has expired.
• vk build on a fleet node pulls private base images from the hub-managed registry. Base
  images on that registry are pulled with the node's access, or with a placed job's token for
  the job's builds, and a build cache naming that registry without a credential of its own uses
  it too; images on other registries are still pulled anonymously.
• vk run on a fleet node builds from private base images on the hub-managed registry.
  Compose service builds and -f boots pull such bases with the node's access, as vk build
  does.
• Jobs from a fleet node's own gitlab-runner can reach the hub-managed registry. A storage
  area's --grant takes guests for them, within what the node itself is granted; such jobs
  then reach the registry at registry.vk with that access only, never the node's own. Jobs the
  hub places keep their own token alone. Nothing is granted unless an area says so.
• vk-hub comes with the CI tools nodes need. The hub holds a reproducible definition of a
  static git (with git submodule) and git-lfs of its own, labelled with their versions, which
  nodes build to hand their jobs; it declares the hosts its build reaches, cache.nixos.org and
  GitHub's. It has no gitlab-runner, which jobs the hub places never run: a fleet whose nodes run their
  own gitlab-runner adds a definition that includes it. ./update-ci-tools.sh re-pins it.
• Fleet nodes give their jobs the CI tools the hub installed. A node whose config names no
  tools directory hands its jobs the tools it last built for the hub, once it has built any.
• vk-hub keeps its nodes on the CI tools it names. The new tools setting, fleet-wide
  or per node, names the tools nodes should have: the hub's own by default, a definition it
  holds, or none. The hub has every node that lacks them build them, so a new node's jobs get
  git with no operator step. A failed build is shown on the node and tried again after 5
  minutes, then 30 minutes, 2 hours and 6 hours; after that it waits for vk-hub nodes tools,
  which now asks for the setting's tools by default.
• vk build stages can declare the network their RUN steps need. A # vk: allow-name=… allow-ip=… line above a stage's FROM holds its steps to those hosts, on top of
  whatever the build allows; --build-net declared gives a stage that declares nothing no
  network at all. CI job builds honor it too. A build that was refused any host ends by
  listing them.
• Fleet nodes build CI tools with only the network their definition declares. Each stage
  reaches the hosts its Dockerfile declares, within the node's [egress.build], and nothing if
  it declares none. The node keeps what the build contacted and was refused, and a failed build
  names the hosts it was refused.
• Operators download and add CI tools from vk-hub's web UI. The Updates page offers each
  tools definition the hub holds, the built-in one included, as a download, and takes a new
  one as an uploaded tar or gzipped tar of its build context, checked and labelled as
  vk-hub tools add does.
• Fleet nodes never give jobs CI tools that changed after they were installed. A node
  records a checksum of every file of the tools it installs and checks them before each job
  that would get them; a job whose tools changed fails before it starts. The node reports the
  checksums, whether its tools are still intact, and what their build contacted and was
  refused. vk-hub nodes and the audit log flag a node whose tools changed, which the hub has
  build them anew, and one whose tools hold other files than other nodes built from the same
  definition. Tools installed by an earlier vk are rebuilt to record their checksums.
• vk-hub's web UI shows CI tools in full. Each tools definition has a page: the settings
  and nodes that use it, the network each stage of its build declares, each node's build and
  whether its tools are intact, what the build reached and was refused, and the files nodes
  installed, with those that differ between nodes flagged. A node's page shows its tools, their
  check and its automatic build with its next attempt; operators build a failed build's tools
  again from there, and remove a definition no setting names, after confirming.

Full Changelog: v0.107.1...v0.108.0

v0.107.1

Fixed

• A VM on a tap can no longer pass checksum or segmentation offload requests to the host
  kernel. A guest could otherwise reach host-kernel bugs in that path, such as the IPv4
  fragmentation flaw fixed in Linux 7.3 and backported to 7.2 and the longterm kernels; patching
  the host kernel is still recommended.

Full Changelog: v0.107.0...v0.107.1

v0.107.0

Added

• vk-hub manages node settings from the CLI and web UI. Use vk-hub settings set and
  unset, or the web UI's Settings page, to set capacity, VM sizing and a hub-managed registry
  fleet-wide or per node, with per-node settings
  winning over the fleet's; a value of the wrong type or out of bounds is refused at once.
• vk-hub shows what each node's settings come to. A node's page lists each setting's value
  and whether it comes from the node's config file, the hub, the host's size or a default, the
  hub's settings it refused, the release keys it trusts (flagged when it has keys but does not
  require signed updates), its automatic update and its vCPUs in use; vk-hub nodes shows the vCPUs
  too.
• A registry's CA reaches the nodes it is named to. vk-hub registries set-url --ca
  takes the CA a registry's certificate chains to, and nodes the hub names the registry to
  trust it with no config of their own.
• vk-hub keeps its nodes on a target release. vk-hub release target sets the vk the
  fleet should run: setting it rolls it out to the nodes running an older version, and from then
  on a node that falls behind, such as one that just joined, is updated on its own once it is
  ready. Nodes on a newer version, with their own gitlab-runner, or whose keys would refuse the
  release are left alone, a node running a pre-release or development build is shown as not
  updated automatically, and a node whose update failed waits for an operator's --retry. The
  target must be signed by one of [nodes] release_keys when the hub sets any, and cannot be
  deleted while it is the target. The web UI's Updates page shows the target and each node's
  status, and lets operators set or clear the target and resume its updates.
• A fleet node's jobs have a network with no config of its own. Unless its config file
  sets [net] mode, a node enrolled with a hub gives its jobs the per-job switch, held to its
  own [egress] and the hub's egress rules, so they can reach the registry proxy and run
  services. The hub cannot change this; a node that wants its jobs offline sets mode = "none".
• A fleet node checks update signatures with no config of its own. The release keys in the
  hub's [nodes] release_keys are pinned on each node when it joins, and from then on it
  requires updates signed by one of them, as it would with the keys in its own config, which
  still win; [node] require_signed = false in its config makes a signature optional, as it
  does for those keys. Nothing the hub says later changes them. The node reports which keys it
  trusts, where they come from and whether it requires a signature.
• A fleet node can use the registry its hub names. With no [registry] in its config, the
  node takes the hub's, CA included, and holds no registry credential of its own: its jobs'
  guests reach the registry with the token the hub gives each job, and so do the node's own
  pulls of the job's images.
• CI jobs can wait for vCPUs as they wait for memory. With [executor.schedule] cpu_overcommit set, a job waits while the vCPUs of the jobs' guests and their services would
  pass that many per CPU of the host, and vk tune asks the runner for no more jobs than that
  leaves room for, though always at least one. A fleet node does this at 2 unless its config or
  hub says otherwise, and vk-hub stops placing jobs on it past that.
• A fleet node takes settings from its hub. Capacity and VM sizing settings and the
  registry may come from the hub; anything the node's own config file sets still wins, and
  every vk on the node uses the result. The node refuses anything else the hub sends and
  reports each setting's value and where it comes from. vk node run restarts itself in
  place when they change.
• vk-hub namespaces can keep chosen branches' and tags' caches apart. A namespace path may
  hold the new {scope}, which is all for most jobs, so the namespace stays per project
  and protection; vk-hub namespaces add … --isolate <glob> gives the refs it matches a scope
  each, which no other ref's job reaches. The path's leading fixed part is reserved: no other
  namespace may reach it, so grants cannot add up across scopes.
• vk-hub adds a ci-build namespace for build caches, one per project, protection and
  isolated ref, kept 14 days, to every registry taking tokens, existing ones included unless a
  namespace already uses that name or root.
• [jobs] require_job_build_cache in hub.toml places jobs only on nodes that cache their
  builds by project, protection and scope; vk-hub nodes says why another takes none.

Changed

• pipeline/build/ images are pulled with the job's token, as they are published, even
  when the node has its own registry credential. That credential needs no read access to
  the pipeline namespace; pulls use it when the job's token expires.
• Nodes that lend guests the job's registry token and have no registry credential
  now use that token to pull placed jobs' images, replacing anonymous pulls; once the token
  has expired, they pull anonymously again.
• vk-hub places jobs only where a node's memory is free now as well as within its budget,
  keeping 15 % of the node's RAM spare, as vk tune does.
• A fleet node needs no memory budget or vCPU ceiling in its config. Unless its config or
  hub sets them, its jobs' memory is budgeted at the host's RAM less the smaller of 4 GiB and
  10 %, and a job may ask for up to all of the host's CPUs; the default job VM keeps 4 vCPUs
  and 4 GiB, less on a smaller host. vk tune works on such a node with no config.
• A node keeps each placed job's builds to its project, protection and scope. Built
  images, layers, build locks and Windows builds are cached under the job's ci-build prefix
  with its token, or not at all, no longer in [build] cache_registry, and what the node
  built for one scope is not found by another. A node whose guests' registry proxy lends its
  own credential (proxy_guests without guest_credential = "job") keeps the old behaviour.
  A placed job's Windows build takes install media from outside its build context only when
  [executor] windows_media lists them. Placed jobs build with the cache off, and say why,
  when the hub is older (upgrade it first), mints no tokens for the node's [registry], or
  has no ci-build namespace there because another namespace held that name or root, or
  when the node's [registry] is missing or local.
• vk-hub validates namespace paths for every job. Paths that could exceed the registry's
  limits with a job's identifiers are rejected when stored, instead of leaving that job
  without access.
• vk-hub rejects unusable registry paths. Namespace paths and operator token grants
  are rejected when stored or minted if they exceed 16 components or contain tags,
  manifests or blobs as a component. GitLab instances cannot use those names either.
  These paths were already refused by the registry.
• While a fleet is upgraded, hosts of this release and earlier ones may each build the same
  cached layer once instead of waiting for each other's build.

Fixed

• GitLab jobs without MICROVM_CPUS or MICROVM_MEM respect
  [executor.vm] max_cpus and max_mem. Previously, they used cpus and mem even
  when those exceeded the ceilings.
• vk-hub counts accepted jobs until a later heartbeat reports them. Previously, they
  stopped counting against memory and vCPUs before the node reported them, allowing offers
  the node would refuse.
• A vk-hub rollout no longer sends an older release to a node updated by hand meanwhile.
  A node that has reached the release's version or a newer one since the rollout started is
  skipped when its wave comes, instead of refusing the update and pausing or aborting the
  rollout, or being downgraded.
• A vk-gitlab relay whose session vk-hub ends is told why. A relay superseded by a newer
  session, removed or refused could previously see its connection fail instead of reading the
  hub's reason.

Full Changelog: v0.106.1...v0.107.0

v0.106.1

Fixed

• Security: vk-registry refuses a pushed manifest that names anything but a lowercase sha256
  digest. A manifest or index whose config, layers, child manifests or subject name another
  digest, sha512: included, is refused with DIGEST_INVALID instead of being stored.
  Manifests the pull-through relay fetches from an upstream are kept as they come.
• Security: vk checks every digest a bundle pulled from a registry names before using it.
  A MICROVM_IMAGE, build-cache entry or Windows layer whose manifest names anything but a
  lowercase sha256: digest now fails to restore, with an error naming it, instead of being
  assembled from whatever the node's local cache holds under that name.
• Security: a blob stored by vk-registry always matches its digest. A chunk sent to a blob
  upload while the upload was being completed could change the stored blob after its digest
  was checked. Such a chunk is now refused, and the upload completes with the bytes it
  verified.

Full Changelog: v0.106.0...v0.106.1

v0.106.0

Added

• A GitLab relay holds one HTTP/2 connection to vk-hub. The relay's session and its requests
  share it, and the session authenticates them. The hub tells the relay when its reservations
  and jobs change and the relay fetches them, so no request is held open waiting. API-key
  clients, nodes, registries and relays of earlier releases stay on HTTP/1.1, as before.

Changed

• vk-hub's GitLab server pages show each relay with its runners. Runner tables fit the
  page, with scheduling details folded under each runner. A runner without a lane says its
  jobs are refused. Operators add and remove runners there; the Overview's relays link there.
• vk-gitlab needs a hub of this release: upgrade the hub first. Against an older hub it
  logs that the hub must be upgraded and takes no jobs from GitLab until it is.

Fixed

• Faster small reads from vk-registry. Clients reading many small objects over one
  connection, such as a build cache fetching entries, no longer wait about 40 ms extra
  per read.
• Faster small reads through a node's registry proxy. A job reading many small objects
  from the registry, such as a build cache, no longer waits about 40 ms extra per read.

Full Changelog: v0.105.1...v0.106.0

v0.105.1

Fixed

• No reservation refused at relay start. A vk-gitlab runner asked the hub for capacity
  before its inventory told the hub GitLab had verified it, so the hub refused it once with
  "not verified with GitLab" and the runner showed unhealthy for a check interval. A runner
  now waits until the hub has its verified inventory, which goes out without coalescing.
• A runner GitLab did not answer at start is verified again. A verify that got no answer
  was retried only on the runner's next job, which the hub refused it, and otherwise a day
  later. It is now retried, backing off up to five minutes.

Full Changelog: v0.105.0...v0.105.1

v0.105.0

Added

• Boot a pipeline's own images. On a fleet, a job or service image
  pipeline/build/<name>[:job-<id>][@sha256:…] boots the image published under that name in
  the job's pipeline: job-<id>'s, or the latest. A missing image fails naming the producer
  job to rerun. Naming the pipeline namespace's images directly is refused for every job;
  [registry] pipeline_root (default ci-pipeline) says where that namespace lives.
• Build a pipeline image from a job. On a fleet, vk-agent build [-f FILE] [--target STAGE] [--build-arg NAME=VALUE]… --as NAME CONTEXT builds an image on the host
  from a context produced by the job and publishes it in the job's pipeline, printing the
  pipeline/build/NAME:job-<id>@sha256:… reference a later job boots. [executor] guest_build turns it off; it is off by default on a node whose registry proxy lends guests
  its own credential. guest_build_max_context and guest_build_max_entries cap a context.

Full Changelog: v0.104.0...v0.105.0