Skip to content

Bootstrap registration: HA client, planner, controller wiring#22

Merged
as51340 merged 2 commits into
01-repo-reset-scaffoldfrom
03-bootstrap-registration
Jul 23, 2026
Merged

Bootstrap registration: HA client, planner, controller wiring#22
as51340 merged 2 commits into
01-repo-reset-scaffoldfrom
03-bootstrap-registration

Conversation

@as51340

@as51340 as51340 commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Implements specs/operator-mvp/issues/03-bootstrap-registration.md: a freshly provisioned MemgraphCluster becomes an actual HA cluster with no human action — all coordinators added, all data instances registered, one MAIN promoted.

What's in the slice

  • Memgraph HA client (internal/memgraph) — a narrow interface (ShowInstances, AddCoordinator, RegisterInstance, SetInstanceToMain) plus a Connector seam, implemented over the neo4j Bolt driver. The driver is referenced nowhere else; queries mirror the HA chart's registration grammar, run as autocommit (coordinator queries cannot run in transactions), and SHOW INSTANCES rows are parsed by column name so added/reordered columns don't break parsing.
  • Registration planner (internal/planner) — pure diff: declared topology + observed SHOW INSTANCES in, ordered commands out (coordinators, then data instances, then the initial MAIN promotion; empty when converged). MAIN is planned only when no MAIN exists — an existing MAIN, on any instance, is never overridden. Undeclared instances are left untouched (no unregistration in v1).
  • Topology derivation (internal/resources.DeclaredTopology) — per-pod registration identity from ordinals: coordinator ordinal N is Raft coordinator N+1, data ordinal N registers as instance_{N+1} (matching the chart's naming), addresses are the pods' stable headless-Service FQDNs. The FQDN derivation is shared with the coordinator start script so advertised and registered addresses cannot drift.
  • Controller wiring — after server-side-applying the workloads, reconcile gates on both StatefulSets reporting every pod ready, then finds the coordinator leader (a coordinator reporting itself leader is used directly, a follower redirects to the leader it reports, and a fresh leaderless cluster bootstraps against the first reachable coordinator, mirroring the chart), plans, and executes. Unreachable coordinators requeue on a delay rather than erroring — Bolt lagging pod readiness is a normal startup phase. After issuing commands the reconcile requeues to verify convergence rather than assuming success.

Everything is read-before-write and idempotent, so killing the operator mid-bootstrap just resumes with only the missing registrations.

Acceptance criteria

  • A fresh MemgraphCluster converges to fully registered: all coordinators added, all data instances registered, one MAIN promoted
  • MAIN promotion is issued only when no MAIN exists; an existing MAIN is never overridden
  • Killing and restarting the operator mid-bootstrap still converges with no duplicate or failed registrations
  • Planner covered by pure unit tests: fresh-cluster bootstrap, partially registered cluster, fully converged no-op, MAIN already present
  • Controller registration flow covered by envtest with the HA client mocked behind its interface
  • The HA client interface is the only place the Bolt driver is referenced

Testing

  • Planner: pure table-driven diff cases (fresh bootstrap, partial, converged no-op, MAIN present, leaderless-but-registered, undeclared instances ignored).
  • Topology: golden tests for names/IDs/FQDNs, plus a guard that the declared coordinator addresses match what the start script advertises.
  • Controller: envtest with an in-memory fake cluster behind the Connector seam. The fake rejects duplicate registrations and second MAIN promotions, so any non-read-before-write behavior fails loudly. Covers: no Bolt contact before pods are ready, full fresh bootstrap with a convergence-confirming follow-up, resuming a partial bootstrap, and executing against the leader a follower reports.
  • make lint, make lint-config, make test-unit, make test all green.

Live-cluster verification lands with the KinD e2e harness (issue 04).

Make a freshly provisioned cluster become an actual HA cluster without
human action (specs/operator-mvp/issues/03-bootstrap-registration.md):

- internal/memgraph: narrow HA client interface (show instances, add
  coordinator, register instance, set main) over the Bolt driver; the
  only place the driver is referenced, and the mock seam for tests.
- internal/planner: pure diff of declared topology against observed
  SHOW INSTANCES output into ordered registration commands, empty when
  converged. SET INSTANCE TO MAIN is planned only when no MAIN exists.
- internal/resources: DeclaredTopology derives per-pod registration
  identity (coordinator IDs, instance names, advertised FQDNs) from pod
  ordinals, sharing the FQDN derivation with the coordinator start
  script so advertised and registered addresses cannot drift.
- internal/controller: once both role StatefulSets report all pods
  ready, find the coordinator leader (follower views redirect to the
  leader they report; a fresh cluster bootstraps against the first
  reachable coordinator), plan, and execute. All interaction is
  read-before-write, so operator restarts mid-bootstrap are harmless.

Planner and topology derivation are covered by pure unit tests; the
controller registration flow by envtest with an in-memory fake cluster
that rejects duplicate registrations and second MAIN promotions.
@as51340 as51340 self-assigned this Jul 23, 2026
@as51340
as51340 merged commit a2eeb2f into 01-repo-reset-scaffold Jul 23, 2026
4 checks passed
as51340 added a commit that referenced this pull request Jul 24, 2026
* Bootstrap registration: HA client, planner, controller wiring

Make a freshly provisioned cluster become an actual HA cluster without
human action (specs/operator-mvp/issues/03-bootstrap-registration.md):

- internal/memgraph: narrow HA client interface (show instances, add
  coordinator, register instance, set main) over the Bolt driver; the
  only place the driver is referenced, and the mock seam for tests.
- internal/planner: pure diff of declared topology against observed
  SHOW INSTANCES output into ordered registration commands, empty when
  converged. SET INSTANCE TO MAIN is planned only when no MAIN exists.
- internal/resources: DeclaredTopology derives per-pod registration
  identity (coordinator IDs, instance names, advertised FQDNs) from pod
  ordinals, sharing the FQDN derivation with the coordinator start
  script so advertised and registered addresses cannot drift.
- internal/controller: once both role StatefulSets report all pods
  ready, find the coordinator leader (follower views redirect to the
  leader they report; a fresh cluster bootstraps against the first
  reachable coordinator), plan, and execute. All interaction is
  read-before-write, so operator restarts mid-bootstrap are harmless.

Planner and topology derivation are covered by pure unit tests; the
controller registration flow by envtest with an in-memory fake cluster
that rejects duplicate registrations and second MAIN promotions.

* feat: 0-based indexing for data instances
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant