feat(api): add spec.policy.ec for cold-data erasure coding - #19
Draft
tdakkota wants to merge 2 commits into
Draft
Conversation
oteldb models storage.policy.ec ({data, parity, after}) but the operator had no way to
reach it. Erasure coding only applies to a shared-nothing cluster — cluster mode plus a
private per-node backend — which is exactly and only what an OtelDBCluster deploys, so
this is the one topology where the policy is reachable, and it was not configurable.
The preconditions all fail silently upstream, so the operator validates them: a shared
backend makes the policy inert, a scheme wider than spec.replicas gets clamped by the
ring, and data+parity replaces the tenant's replication factor outright — for the
unflushed head too. That last one is refused rather than documented: setting it next to
spec.cluster.replicationFactor would leave the CR stating a factor the cluster ignores.
This was referenced Aug 18, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
spec.policy.ec— the cold-data erasure-coding tier oteldb merged inoteldb#1279 — mirroring
storagebackend.ECConfig'sshape (
{data, parity, after}) ontotenant.ECScheme.Erasure coding stores
(data+parity)/dataof the logical bytes and survivesparitynode losses:{4,2}is 1.5x for two tolerated losses, against 3x for RF=3.Why this belongs in the operator specifically
The engine gates EC on a shared-nothing cluster — cluster mode and
private_backend. That isprecisely and only what an
OtelDBClusterdeploys:spec.etcdis required, and the defaultfilebackend is one PVC per pod. A single-node oteldb cannot reach this policy, and neither can a cluster
on a shared bucket. So this is the one deployment shape where erasure coding is usable at all — and
until now it was not configurable here.
Validation, and the RF interaction
Every precondition fails silently upstream, so the operator (which owns the topology) turns each
into an
InvalidSpeccondition:spec.storage.backend: s3withoutspec.cluster.privateBackend: true.oteldb only logs a warning and carries on with full-copy parts.
data+parity > spec.replicas. The ring clamps the owner set to themembers it has instead of failing.
afterpastretention.maxAge— merge work whose output is dropped unread, the same rule theother cold tiers already get.
data+parity > 256, negativeafter— the boundsec.Scheme.Validateenforces.The interesting one: under EC the owner count is
data+parityand the tenant's replication factoris ignored —
Storage.rfForreturnsScheme.Shards()before it ever looks at RF, so this coversthe unflushed head as well as the converted parts. I chose to validate rather than document:
spec.policy.ectogether withspec.cluster.replicationFactoris rejected outright. Honouring oneand dropping the other would leave the CR stating a replication factor the cluster does not use, and
it is the same class of explicit-but-ignored setting the repo already rejects (
maxSeriesSoftwithout
maxSeries, dead merge tiers). DroppingreplicationFactorwhen adopting EC is a one-linespec edit. It is documented as well, on
ECSpec.Parityand in the README.Rack safety (
ceil((data+parity)/parity)distinct zones) is documented, not validated — theoperator cannot see the cluster's zone topology from the CR, and the engine converts and warns
rather than failing.
Shape
storage.policy.ecjoins the reservedextraConfigpaths, so the block stays modelled in full.spec.policy.ecrenders byte-identically to today: noeckey, and nopolicyblock atall if nothing else is set.
afteromits the key, which oteldb reads as zero — erasure-code every part.Stack
Based on #17 (
spec.cluster.privateBackend), which is based on #15 (spec.query), which is based on#14 (
spec.ingest). TheprivateBackenddependency is load-bearing, not just ordering: the ECvalidation reads
privateBackendOf(cr).Merge #14 → #15 → #17 → this.