RFC 0029: native Azure Blob storage

[roy2392]

Summary

Proposes RFC 0029 for native Azure Blob Storage support through canonical
az://<container>/<prefix> roots, the existing Lance + Apache Arrow
object_store seams, and managed identity.

The RFC also defines a reference Azure Container Apps deployment. Importantly,
it does not treat minReplicas = maxReplicas = 1 as a correctness lock:
Azure may temporarily pre-warm extra replicas. Instead, every supplied server
and bootstrap writer must pass through one fail-closed PID-1 admission path:
Tini forwards signals and subreaps descendants while its supervised wrapper
owns an infinite per-cluster Blob lease before an OmniGraph child can start.
This is explicitly scoped as a cooperative process-admission mutex, not
engine-level data fencing or multi-writer support.

Architecture

The image was produced and visually reviewed with the
msftse/azure-architecture-proposal diagram workflow. Its exact
reproducible prompt
is committed beside the PNG. The workflow repository is not linked because it
is not publicly reachable to an unauthenticated upstream reviewer.

Why this PR is RFC-only

This follows OmniGraph's public-contribution sequence:

1. originating Ideas Discussion #439;
2. this public RFC-only PR;
3. implementation in a separate PR only after the RFC is accepted.

The implementation prototype, deployment package, CI job, docs, and validation
record are intentionally excluded from this branch.

Prototype evidence and evidence boundary

A prototype based on upstream
c62829acc6f5414356304d1c240d95fad38b8984 has already passed:

• Azure URI/storage, engine, cluster, and server tests against Azurite;
• a raw Blob-layout assertion covering the cluster ledger, graph manifest, and
  Lance data below the same cluster root;
• a disposable public-Azure run with a user-assigned managed identity,
  anonymous Blob access disabled, container-scoped Blob RBAC, immutable ACR
  image, authenticated mutation/query, restart persistence, physical Blob
  inspection, retained-log credential scanning, and confirmed resource-group
  cleanup.

That first live run proves the native storage and identity path. It does not
prove the newly specified lease-admitted mutable topology. The RFC therefore
requires a second adversarial live-Azure gate before an implementation PR may
claim support: competing app/job writers, unique lease IDs, lost responses,
graceful long-write drain, unexpected exit, hard-kill stuck-lease behavior,
one-child interval evidence, and explicit break-glass recovery.

Checks for this RFC branch

• bash scripts/check-agents-md.sh — 74 links / 69 docs OK
• git diff --check — clean
• architecture PNG visually reviewed; reproducible prompt included
• no implementation code, credentials, subscription/tenant IDs, live Azure
  resource names or endpoint URLs, tokens, or proof-row values included

Requested feedback

1. Does the canonical az://<container>/<prefix> model fit OmniGraph's storage
   and URI contracts?
2. Is the fail-closed infinite Blob lease wrapper an acceptable external
   enforcement of the existing one-live-writer-process support boundary for the
   first Container Apps reference?
3. Which additional acceptance gates or Azure environments should be in scope
   before the separate implementation PR?

Greptile Summary

The PR proposes native Azure Blob Storage through canonical az:// roots and a managed-identity-based Azure Container Apps deployment.

• Defines Azure URI, storage, conditional-write, authentication, and compatibility contracts.
• Specifies a fail-closed Blob-lease admission topology for server and bootstrap writers.
• Establishes Azurite, build, infrastructure, and adversarial live-Azure acceptance gates.

Confidence Score: 4/5

The RFC should not merge until it identifies an implementable managed-identity-capable client for the mandatory Blob lease protocol or revises the no-Azure-SDK constraint.

The current proposal still requires lease acquire, ownership verification, release, inspection, and break operations, but the named Lance, object_store, and StorageAdapter surfaces do not provide that protocol and the RFC excludes introducing another Azure SDK.

Files Needing Attention: docs/rfcs/0029-azure-blob-storage.md

Important Files Changed

Filename	Overview
docs/rfcs/0029-azure-blob-storage.md	Defines the Azure storage and deployment proposal; the previously reported lease-client specification gap remains unresolved.
docs/rfcs/assets/0029-azure-reference-architecture.prompt.md	Provides the reproducible prompt and constraints for the RFC architecture diagram.
docs/rfcs/assets/0029-azure-reference-architecture.png	Adds the rendered architecture illustration referenced by the RFC.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
    Clients[Clients] -->|HTTPS| Server[OmniGraph server]
    Job[Bootstrap job] --> Gate[Writer admission wrapper]
    Server --> Gate
    Gate -->|Exclusive lease| Lease[(Admission lease Blob)]
    Gate -->|Query / mutate / import| Root[(Azure Blob cluster root)]
    Identity[User-assigned managed identity] -.-> Lease
    Identity -.-> Root
    Registry[Azure Container Registry] --> Server
    Registry --> Job
    Server --> Logs[Log Analytics]
    Job --> Logs

Loading

_{Reviews (4): Last reviewed commit: "Merge branch 'main' into codex/rfc-0029-..." | Re-trigger Greptile}

[devin-ai-integration]

Devin Review found 1 potential issue.

View 2 additional findings in Devin Review.