Repository navigation
A patch release of 0.7.0. The default engine is kividb 1.2.1. Upgrade
now if you run engine 1.2.1, or will: since 1.2.1 the engine refuses a
write pause it cannot take at once, and operator 0.7.0 then switched over
without the pause, so a write the old master acknowledged could be lost
(#81). 0.7.1 also fences a force-deleted master's last writes before a
failover (#53), keeps a restore from mixing the snapshot with the earlier
cluster's append-only file (#58), fixes the images' CVEs (#76), adds an
OLM bundle for OperatorHub.io, and fixes three install paths.
Read before upgrading: every cluster restarts, and a master with no
replica restarts in place. The default agent image moves to 0.7.1, and
the default engine to 1.2.1, so every KividbCluster that leaves
spec.agentImage or spec.image unset gets a rolling restart when the
operator is upgraded. A master with replicas hands its role to an in-sync
replica first, as in every rollout. A master with none is deleted and
restarted on its volume, with no handover:
replicas: 0(one pod): the cluster serves nothing until the pod is
back.- A sharded cluster with
replicas: 0: each shard's master restarts
in turn. Its slots are unavailable while it does, and if it is down for
longer than the node timeout (5 s by default) withrequireFullCoverage
on (the default), the whole cluster refuses commands until it is back. - Data: with the append-only file on (the default since 0.7.0) the
master reloads everything it acknowledged. Withappendonly: "no"it
comes back with its shutdown snapshot only, and with nothing saved since
its last snapshot if that save does not finish within
terminationGracePeriodSeconds.
Before upgrading, give such clusters or shards at least one replica and
let it sync, or accept the restart: with the append-only file on, or with
the shutdown snapshot only. (Turning the append-only file on is a config
change, so it restarts such a master in place too.) To put the restart off, pin
both spec.image: docker.io/kividb/kividb:v1.2.0 and
spec.agentImage: docker.io/kividb/kividb-operator-agent:0.7.0 first:
nothing else in the pod template changes in 0.7.1. The 0.7.0 agent works
with the 0.7.1 operator, without the agent's fixes: it cuts every pause at
5 s, which aborts a switchover that needs longer (#81), and its restore
lacks the #58 fix. Unpin once the cluster has a replica or its
append-only file.
Engine 1.2.1 has upgrade notes of its own. The rollout restarts
replicas before their master, as 1.2.1 requires (a 1.2.0 replica cannot
full-sync from a 1.2.1 master that holds a key 1.2.0 cannot read). Read
the engine's "Before you upgrade"
first: among others, ACL categories are Redis's from the moment a pod
runs 1.2.1, GEORADIUS is refused on replicas, and there is no going
back to 1.2.0 once 1.2.1 has saved such a key. Do not change
spec.shards during the rollout: a slot move from a 1.2.0 node loses
the moved time series' compaction rules.
The CRD change is additive, and upgrading from 0.7.0 is safe. The
kividbclusters status gains pods[].podIP and fencedMaster (name,
podIP, runID), and new conditions (Switchover); nothing is removed
or renamed, and the other CRDs do not change. Apply the new
kividbclusters CRD before helm upgrade: Helm does not upgrade it.
Until it is applied, the API server drops the new fields and failovers do
not hold the old master's writes (FailoverUnfenced Warning). The CRDs
are attached to the GitHub Release. Coming from 0.6.x, read 0.7.0's
"Read before upgrading" first.
Security
- The images had golang.org/x/net v0.60.0 (CVE-2026-78661, high;
CVE-2026-97045, medium) and an out-of-date Debian 12 base (#76).
golang.org/x/net is now v0.62.0, the other golang.org/x modules are
current, the binaries are built with Go 1.26.11, and the images run on
gcr.io/distroless/static-debian13:nonroot(still no shell, still UID
65532). CI and the release workflow scan every image with Trivy and fail
on a fixable HIGH or CRITICAL.
Changed
- Default engine
docker.io/kividb/kividb:v1.2.1(it wasv1.2.0):
in the controller, the CRD, the chart, the GUI, the samples and the docs.
See "Read before upgrading" above. - Default agent image
docker.io/kividb/kividb-operator-agent:0.7.1.
A cluster that leavesspec.agentImageunset rolls to it in the same
rollout. The switchover and restore fixes below need it. - The e2e suite tests kividb 1.2.1 by default.
Added
- An OLM bundle for OperatorHub.io.
make bundle VERSION=… IMG=…builds
the bundle fromconfig/manifests(operator-sdk v1.36.0, kustomize v5.4.3),
andmake scorecardruns the scorecard tests. The bundle validates for
Kubernetes 1.27, 1.32 and 1.33 and installs through OLM 0.28.0 in
AllNamespaces mode.
Fixed
-
The plain-YAML install and
kubectl apply -k config/defaultmade the
manager Deployment with different selectors (#86). Applying one over the
other failed withspec.selector: ... field is immutable.
config/manager/manager.yamlnow selects onapp.kubernetes.io/managed-by
too, asconfig/defaultdoes; whatconfig/defaultand the OLM bundle
render is unchanged. A Deployment made before this fromconfig/manager
alone, by the plain-YAML install orkubectl apply -k config/manager, has
the old selector: delete it once (kubectl delete deploy -n kividb-operator-system kividb-operator-manager) before applying the new
manifests.
(#88, by SaiPisey2) -
docs/INSTALL.md's install without kustomize exited 1 (#67).
kubectl apply -f config/rbac/applied the RBAC and then failed on
config/rbac/kustomization.yaml, and on a new cluster the namespace did
not exist. The steps now create the namespace and name the RBAC files.
(#85, by SaiPisey2) -
A switchover no longer hands over without holding writes when the pause
could not take hold (#81). Since kividb 1.2.1 (kividbio/kividb#629),
CLIENT PAUSE ... WRITEwaits up to 5 s for the writes already in progress
and refuses, pausing nothing, if one has not finished (a bigFLUSHALL, a
DELof a huge key, a long script). The switchover took that refusal, and
a pause that got no answer within the agent's 5 s, for an engine before
kividb 1.1: it promoted the target without waiting for it to catch up,
and raisedSwitchoverWithoutPausetelling the user to upgrade an engine
that was already new. A write the old master acknowledged then could be
lost. Now such a switchover is called off (SwitchoverAborted) and tried
again; a pause that got no answer is lifted, in case it landed. Only an
engine that answers that it does not know the command is switched over
without the pause: every kividb has it, so any other refusal (LOADING,
BUSY) aborts too. The pause call is given 7 s, past the engine's wait
(or--agent-timeout, if longer), and the agent is asked to wait 6 s:
its exec request takes atimeoutMs(capped at 15 s), which the
operator sends for the pause only. An agent before this release ignores
it and cuts the pause at 5 s, which now aborts the switchover rather than
dropping the pause. The engine counts the pause from the request, so it
now lasts the call, the catch-up and a margin (17 s), and the old master
is a replica before it runs out. Aborted switchovers back off: 30 s,
doubling with each abort in a row up to 10 min, and 2 s between the tries
for a master pod being deleted; aSwitchovercondition (False,
Aborted) says why and when the next try is, until one goes through.
The failover fence (#53) waits for a pause the same way once the old
master has answered, and tries again after a refusal: every try is given
the whole call, and new ones start for 3 s after the first, so the fence
takes at most two pause calls and that window against a live master that
cannot pause (17 s with the default--agent-timeout, more with a longer
one). A pause that got no answer may have landed, so that
master is recorded instatus.fencedMasterand demoted after the
promotion like a fenced one. TheSwitchovercondition and the count of
aborts go away once no switchover is wanted any more, or the master is
another pod.
(#82, by lofoneh) -
kubectl apply -k config/defaultinstalled noKividbDbOpsCRD.
config/crd/kustomization.yamldid not list it, so the documented install
could not runKividbDbOpsoperations. After applying it again, restart
the manager (kubectl rollout restart): it looks for the CRD once, at
start-up. -
The full
KividbConfigsample was rejected by its own CRD. It set
tls: {enabled: false}, and the CRD requirescertSecretRefwhenevertls
is present. The sample now leavestlsout. -
A restore over a kept volume could mix the snapshot with the earlier
cluster's append-only file (#58). The restore Job removes the earlier
cluster'sdump.kdb.gen, and without it engine 1.2.0 and 1.2.1 both
replayappendonly.aofoverdump.kdb. The Job removed the earlier
cluster'sappendonly.aofonly after it had renamed the restored
dump.kdbin, so a Job stopped between the two (killed, evicted, or a
removal that failed) left the restored dump with the earlier cluster's
log. A cluster started on that volume, oncespec.bootstrapFromSnapshot
was removed after a failed bootstrap, came up with the writes of both.
Every file of the earlier cluster's engine now goes before a restored
file takes its name:appendonly.aofand everyappendonly.aof.*,
everydump.kdb.*(its state and temp files),dump.rdb, and the
superseded.*logs that engine 1.2.1 keeps aside. The engine never
removes those: they held the volume's space, and every start named
them with how to apply their writes. A Job stopped part-way now leaves
what is left of the earlier cluster's files, or part of the snapshot,
never both, and its retry ends with the snapshot's files only. The
manual restore indocs/BACKUP_RESTORE.mdremoves the same files. -
A force-deleted master's last writes were lost (#53). Its pod object
is gone at once and the operator fails over within the second, but its
container runs on until its replicas have left it, acknowledging the
writes that come in on connections clients already had to it: 176 lost in
the AKS test on engine 1.2.1, 689 to 954 per force-delete on kind. Before
promoting a replica, the operator now holds the old master's writes
(CLIENT PAUSE WRITEthrough its agent, for the pod's termination grace
period -- 10 s at least -- and 5 s more) at the address and engine run
status.pods[]last recorded for it (podIP, new, andrunID), waits
for the replica to catch up with it (5 s at most, less once the old
master stops answering), records it instatus.fencedMaster(new), and
once the replica is promoted makes the old master its replica and
disconnects its clients, as a switchover does. Should that fail, or the
operator restart, every pass holds its writes again and retries, every
2 s, until it is done or the engine there is gone (another engine
answers, or none does twice in a row); the first failure raises a
FencedMasterNotDemotedWarning. An address that does not answer is
asked again for about 1 s (one that refuses the connection, once); a
pause that goes unanswered is sent again for up to 3 s. A failover that
goes on without the fence, or before the replica caught up, raises a
FailoverUnfencedWarning. A failed promotion no longer lifts the old
master's pause: it may have gone through. The agent's preStop hook looks at a stopping master every 10 ms
instead of 250 ms.
hack/e2e/10-force-delete-fence.shforce-deletes the master under load
and counts the writes lost (skipped in CI:SKIP_FENCE=1).
(#77, by lofoneh)