v0.13.0

What's Changed

There are two big new features in this release, both of which are not yet enabled in the default configuration. They goal is to promote both of them as a sensible default once they have been validated at a larger scale. If you do test them and have any issues or other feedback, please do not hesitate to open an issue.

1. Capacity Eviction: prevents node out-of-memory scenarios by ensuring the node has enough resources to restore a container before allowing it to happen. If the node does not have enough capacity, it will evict the requesting pod and forward requests to the new one as soon as it's available. This feature can be toggled on using a kustomize component.

   # config/production/kustomization.yaml
   components:
   [...]
   - ../capacity-request

2. Reuseport Activator (Experimental, requires Linux 6.6 or later): This enables a completely new activator which handles traffic to the container purely in the kernel and requires no userspace proxying. See the architecture document for more details on how this works. This feature can be toggled on using a kustomize component.

   # config/production/kustomization.yaml
   components:
   [...]
   - ../reuseport-activator

Apart from that this new release contains several bug fixes and dependency upgrades like CRIU v4.2.1 and Go 1.26.

• feat: improve kubelet address detection for ignoring probes by @ctrox in #189
• feat: auto-migrate on low capacity by @ctrox in #184
• chore: upgrade go and dependencies by @ctrox in #190
• feat: reuseport activator by @ctrox in #196
• fix: read probe request and set deadline by @ctrox in #199
• feat(capacity): ignore container with empty requests by @ctrox in #198
• ci: cleanup old dev images by @ctrox in #200
• chore: build criu v4.2.1 by @ctrox in #202
• build: add criu version default by @ctrox in #203
• chore: bump criu to v4.2.1 by @ctrox in #204
• fix: use tc instead of cgroup_skb by @ctrox in #205
• feat: improve capacity eviction by @ctrox in #201
• refactor: rename eviction-timeout for consistency by @ctrox in #206
• docs: update/unify config by @ctrox in #207
• fix: go fix and linter fixes by @ctrox in #208
• docs: fix some links by @ctrox in #209
• fix: activator init backoff by @ctrox in #211
• chore: update image versions by @ctrox in #210

Full Changelog: v0.12.1...v0.13.0

v0.12.1

What's Changed

• fix: ignore "wait: no child processes" error when attaching by @ctrox in #187
• fix: do not clean pin path on activator stop by @ctrox in #188
• fix: cleanup on checkpoint process exit by @ctrox in #191
• refactor: remove err return from ScheduleScaleDown by @ctrox in #192
• chore: update image versions by @ctrox in #193

Full Changelog: v0.12.0...v0.12.1

v0.12.0

What's Changed

The main change for this release is that the eBPF attachment is now being done by the shim again instead of the node daemonset. This makes the shim less dependent on the node component. It was initially delegated to the node component since loading eBPF programs caused significant memory allocations in the shim but with this new implementation this was addressed by doing the attachment in a separate short-lived shim process. For upgrading to this release, the order does not matter since the node component can still take over the attachment if older shim versions are detected that don't support it.

• chore: update grpc by @ctrox in #181
• feat: attach eBPF in shim by @ctrox in #180
• feat: skip shim managed attachments in manager by @ctrox in #183
• fix: reduce logging noise on start by @ctrox in #186
• chore: update image versions by @ctrox in #185

Full Changelog: v0.11.3...v0.12.0

v0.11.3

What's Changed

• ci: pin action versions by @ctrox in #176
• fix: use improved ps addr patch by @ctrox in #172
• fix: dir/file permissions on extract by @ctrox in #177
• fix: validate snapshot data before migration restore by @ctrox in #178
• chore: update image versions by @ctrox in #179

Full Changelog: v0.11.2...v0.11.3

v0.11.2

What's Changed

• fix: metrics socket connection leak by @ctrox in #167
• fix: add new annotations to containerd config by @ctrox in #168
• fix: lastActivity when container has empty ports by @ctrox in #169
• fix: do not scale down when execs are running by @ctrox in #170
• chore: update image versions by @ctrox in #171

Full Changelog: v0.11.1...v0.11.2

v0.11.1

What's Changed

• fix: close bpf maps/netns on stop by @ctrox in #164
• fix: ignore closed err by @ctrox in #165
• chore: update image versions by @ctrox in #166

Full Changelog: v0.11.0...v0.11.1

v0.11.0

What's Changed

• added annotations for configuring proxy timeouts

  zeropod.ctrox.dev/connect-timeout: "5s"
  zeropod.ctrox.dev/proxy-timeout: "5s"

• much improved checkpoint/restore failure handling

  • the last 20 lines of the criu log are now visible in the zeropod-node log
  • falls back to disabling checkpoint after failure
  • events are created on checkpoint/restore error
  • metrics have been added to record the error counts

• local checkpoint images are now deleted on restore, not just on pod deletion

Changelog

• feat: update resource requests earlier on migration by @ctrox in #146
• fix: allow resizing cpu/mem only by @ctrox in #147
• feat: cleanup image on restore by @ctrox in #148
• feat: fail migration of running container early by @ctrox in #149
• fix: unlock CR during evac by @ctrox in #150
• feat: improve checkpoint/restore fail handling by @ctrox in #151
• feat: add checkpoint/restore error metrics by @ctrox in #152
• Add configurable proxy timeout via annotations by @PhoenixSolutionsGroup in #154
• fix: return early if already restored by @ctrox in #155
• fix: do not migrate when checkpointing is disabled by @ctrox in #156
• feat: move runtimeclass install to config by @ctrox in #157
• fix: version information in manager/installer by @ctrox in #158
• fix: ensure eventTime is never empty by @ctrox in #159
• feat: add loop detection to activator by @ctrox in #160
• docs: document timeout annotations by @ctrox in #161
• chore: update dependencies by @ctrox in #162
• chore: update image versions by @ctrox in #163

New Contributors

• @PhoenixSolutionsGroup made their first contribution in #154

Full Changelog: v0.10.0...v0.11.0

v0.10.0

What's Changed

• cleanup of old migration resources: migration resources are now owned by the source/destination pod, resulting in automatic cleanup by the API Server on deletion of a pod. If this is undesired, for example while debugging, the cleanup can be disabled by setting the flag -auto-gc-migrations=false on the manager.
• cleanup of old checkpoint images: checkpoint images were not being cleaned up in all cases, especially when migration was disabled.
• various fixes and improvements for migration scalability.

Full Changelog

• fix: use net.JoinHostPort for ipv6 compatibility by @ctrox in #132
• fix: skip unix sockets as they can't be archived by @ctrox in #133
• chore: run modernize by @ctrox in #134
• feat: make migration timeouts and reconcile concurrency configurable by @ctrox in #136
• fix: remove image on deletion by @ctrox in #138
• fix: cleanup image of finished migrations by @ctrox in #135
• feat: garbage collect migrations by default by @ctrox in #139
• test: disable pinning by @ctrox in #140
• feat: remove global checkpoint lock by @ctrox in #141
• fix: do not claim previously unclaimed migrations by @ctrox in #142
• feat: adjust checkpoint/restore duration buckets by @ctrox in #144
• chore: update image versions by @ctrox in #143

Full Changelog: v0.9.2...v0.10.0

v0.9.2

What's Changed

• feat: add flag to ignore localhost in socket tracker by @ctrox in #129
• feat: pin TCX links by @ctrox in #130
• chore: update image versions by @ctrox in #131

Full Changelog: v0.9.1...v0.9.2

v0.9.1

What's Changed

While validating the latest release on different systems, an issue was identified where the socket_tracker map would be emptied by the kernel sporadically. This includes a hotfix for this issue by changing the BPF map type.

Upgrade notes

If you are coming from release v0.8.0 or earlier and make use of scale to zero, ensure to re-schedule all pods using zeropod after updating the zeropod-node DaemonSet. This is due to an architectural change in the socket tracker. Pods running pre-v0.9.0 zeropod shims might not be able to detect the last TCP connection and will simply scale down after the configured duration is up.

Changelog

• fix: use non-LRU map type for socket tracker by @ctrox in #119
• chore: update image versions by @ctrox in #120

Full Changelog: v0.9.0...v0.9.1