Skip to content

feat(cli): aisix export — emit resources.yaml from a running etcd store - #761

Merged
moonming merged 1 commit into
mainfrom
feat/aisix-export
Jul 16, 2026
Merged

feat(cli): aisix export — emit resources.yaml from a running etcd store#761
moonming merged 1 commit into
mainfrom
feat/aisix-export

Conversation

@moonming

@moonming moonming commented Jul 14, 2026

Copy link
Copy Markdown
Collaborator

What

Adds an aisix export subcommand — the inverse of the file source (resources_file). It reads the resources currently stored in etcd and emits a resources.yaml that the file loader can load back, so a standalone deployment can move from Admin-API-plus-etcd to the declarative file (or take a portable, declarative backup of its live config).

aisix export --etcd <endpoints> [--prefix /aisix] [--reveal-secrets] [-o resources.yaml]

Default output is stdout; the secret-placeholder list and any warnings go to stderr.

How it stays faithful to the loader

Export decodes every kind through the same typed models the gateway loads (the shared snapshot builder), so the exported set is exactly what the running gateway would serve. It then rewrites each entry into idiomatic file form — the exact sugar the file source desugars on the way in:

  • Resugar references (canonical → file):
    • model.provider_key_idprovider_key: <that provider key's display_name>
    • rate_limit_policy.scope_ref for api_key / model scopes → the referenced entry's name; team / member / team_member pass through verbatim.
    • A reference that resolves to nothing in the export set is kept as a raw id and warned — a dangling reference is a real data issue to surface, not hide.
  • Strip ids — the file derives ids from names (UUIDv5), so no id is emitted. Canonical api-key documents carry no name, so a deterministic apikey-<hash…> display_name is synthesized from the entry's already-hashed key_hash (emitted verbatim — it round-trips exactly and is not plaintext).
  • Duplicate identities within a kind (possible in raw etcd, impossible in the file) are surfaced as warnings.
  • Entries the loader would reject during decode, and guardrail_attachments (which the file format has no collection for), are reported on stderr rather than dropped silently.

Secret policy

No live credential is written in cleartext by default. Each secret-bearing field is replaced with a derived, stable, greppable ${VAR} placeholder, and a companion "set these before loading" list is printed to stderr — never into the file:

  • provider_key.api_key, mcp_server.secret, a2a_agent.secret
  • guardrail moderation credentials (api_key, access_key_secret, and the nested Bedrock secret_access_key)
  • OTLP exporter auth headers

api_keys emit key_hash directly (already a hash — round-trips exactly; no placeholder). --reveal-secrets opts into emitting the real stored values inline; it is documented as unsafe and intended only for air-gapped, same-host migration.

The placeholder names use the AISIXSECRET_… prefix rather than AISIX_…: the gateway's config loader claims the AISIX_ prefix and rejects unknown keys, so an AISIX_… secret variable set in the data plane's environment would be misread as a bad config override and fail boot (the same reason the existing SLS_CRED_… / OBJSTORE_CRED_… conventions keep secret variables off that prefix).

Emission reuses the file source's own YAML library, so the output is guaranteed to re-parse (including the mandatory quoted _format_version: "1").

Tests

  • Rust unit (20): per-kind resugar (provider-key ref by name, scope_ref resolution, dangling-ref warn), synthetic api-key name + key_hash passthrough, duplicate-name warn, id-stripping, secret placeholder derivation + no-cleartext-by-default + --reveal-secrets, recursive/nested guardrail + header redaction, and YAML round-trip (format-version stays a string, placeholders survive).
  • e2e round-trip (tests/e2e/src/cases/export-roundtrip-e2e.test.ts): seed etcd → run aisix export → load the exported file into a file-mode gateway (supplying the real secret through the emitted placeholder variable) → assert identical observable behavior (/v1/models, a chat completion, and an allowed_models 403) against the etcd-mode gateway, and assert the exported file contains no live credential at default settings. This closes the loop: etcd → export → file → same behavior.

Full workspace suite green (2299 passed) run twice; cargo fmt --check and cargo clippy --workspace --all-targets -D warnings clean; resource schemas untouched.

Summary by CodeRabbit

  • New Features
    • Added an aisix export command to export gateway resources from etcd into a YAML “resources” document.
    • Supports custom endpoints, key prefix selection, optional output file, and an option to reveal secrets.
    • Automatically redacts secrets into deterministic ${VAR} placeholders, with operator-facing warning/blocking reporting.
    • Preserves stable ordering and reference integrity, including dollar-escaping for literal ${...} and improved handling of missing, dangling, duplicate, and omitted items.
  • Tests
    • Added end-to-end coverage verifying export/import round trips and equivalent gateway behavior.
    • Added expanded unit tests for export document generation, YAML emission, and redaction/reveal behavior.

@coderabbitai

coderabbitai Bot commented Jul 14, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@moonming, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 45 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 98014b2a-e9d5-4acb-b98a-aa17252adb45

📥 Commits

Reviewing files that changed from the base of the PR and between e6c9cef and bc9fd43.

⛔ Files ignored due to path filters (1)
  • Cargo.lock is excluded by !**/*.lock
📒 Files selected for processing (8)
  • crates/aisix-server/Cargo.toml
  • crates/aisix-server/src/export/document.rs
  • crates/aisix-server/src/export/document_tests.rs
  • crates/aisix-server/src/export/mod.rs
  • crates/aisix-server/src/export/secrets.rs
  • crates/aisix-server/src/export/yaml_emit.rs
  • crates/aisix-server/src/main.rs
  • tests/e2e/src/cases/export-roundtrip-e2e.test.ts
📝 Walkthrough

Walkthrough

Adds aisix export, which loads gateway resources from etcd, rewrites them into resources YAML, redacts secrets with environment placeholders, reports diagnostics, and validates etcd-to-file round-trip behavior.

Changes

Resource export workflow

Layer / File(s) Summary
Secret placeholder handling
crates/aisix-server/src/export/secrets.rs
Adds deterministic environment-variable naming and redaction for top-level fields, nested secret keys, headers, and string maps, with unit coverage.
Canonical snapshot transformation
crates/aisix-server/src/export/document.rs, crates/aisix-server/src/export/document_tests.rs
Converts snapshots into ordered resource collections, rewrites identifiers and references, escapes dollar values, filters guardrails, and records duplicate, dangling, collision, and unrepresentable-entry diagnostics.
Resources YAML emission
crates/aisix-server/src/export/yaml_emit.rs, crates/aisix-server/Cargo.toml
Emits ordered YAML using yaml-rust2, preserving scalar types, quoted format metadata, and placeholders.
Export CLI orchestration
crates/aisix-server/src/main.rs, crates/aisix-server/src/export/mod.rs
Adds CLI arguments, etcd loading, document construction, output writing, permission handling, and stderr reporting for rejections, warnings, and secrets.
Etcd-to-file round-trip validation
tests/e2e/src/cases/export-roundtrip-e2e.test.ts
Validates redaction, placeholder injection, exported diagnostics, and equivalent gateway behavior after loading the exported YAML.

Estimated code review effort: 4 (Complex) | ~60 minutes

Sequence Diagram(s)

sequenceDiagram
  participant CLI as aisix export
  participant Etcd as etcd
  participant Loader as Snapshot loader
  participant Builder as build_export_document
  participant Emitter as emit_yaml
  participant FileGateway as file-mode gateway
  CLI->>Etcd: load resources under prefix
  Etcd-->>Loader: return stored documents
  Loader-->>Builder: provide AisixSnapshot
  Builder-->>Emitter: provide redacted ExportDocument
  Emitter-->>CLI: return resources YAML
  CLI->>FileGateway: start with YAML and environment values
  FileGateway-->>CLI: serve equivalent gateway behavior
Loading

Possibly related issues

  • api7/AISIX-Cloud#1008: Covers resources YAML export and round-trip behavior, including format versioning and interpolation.

Possibly related PRs

  • api7/aisix#759: Introduces file-source parsing and interpolation semantics required to reload exported YAML.
  • api7/aisix#280: Introduces the rate-limit policy scope and reference shape that export resugaring handles.

Important

Pre-merge checks failed

Please resolve all errors before merging. Addressing warnings is optional.

❌ Failed checks (1 error)

Check name Status Explanation Resolution
Security Check ❌ Error Nested values in provider_key default_body_fields aren't recursively redacted, so secrets can leak; same-identity placeholder collisions can also cross-wire credentials. Recurse through default_body_fields objects/arrays when redacting, add nested-value tests, and warn on any duplicate env_var (including same identity) by tracking the raw field path.
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly matches the main change: a new CLI export command that emits a resources.yaml from etcd.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
E2e Test Quality Review ✅ Passed The new E2E cases exercise the real CLI→etcd→export→file-mode gateway flow plus redaction, with clear setup/cleanup, isolated prefixes, and readable assertions.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/aisix-export

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🧹 Nitpick comments (3)
crates/aisix-server/src/export/secrets.rs (1)

76-183: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Consider a shared redaction-context struct instead of repeating the 5-arg tail across redact_top_level, redact_by_key, and redact_headers.

All three functions carry the same (kind_token, kind, identity, reveal, out) tail. Bundling them into a small struct RedactionCtx<'a> { kind_token: &'a str, kind: &'static str, identity: &'a str, reveal: bool, out: &'a mut Vec<SecretPlaceholder> } would cut duplication and remove the risk of transposing two same-typed positional &str args (kind_token vs kind) at a call site.

♻️ Sketch
pub struct RedactionCtx<'a> {
    pub kind_token: &'a str,
    pub kind: &'static str,
    pub identity: &'a str,
    pub reveal: bool,
    pub out: &'a mut Vec<SecretPlaceholder>,
}

pub fn redact_top_level(doc: &mut Value, field: &str, ctx: &mut RedactionCtx) { ... }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/aisix-server/src/export/secrets.rs` around lines 76 - 183, Introduce a
shared RedactionCtx<'a> containing kind_token, kind, identity, reveal, and out,
then update redact_top_level, redact_by_key, and redact_headers to accept the
context instead of their repeated five-argument tail. Adjust recursive
redact_by_key calls and all callers to pass the context while preserving
existing redaction behavior and placeholder output.
tests/e2e/src/cases/export-roundtrip-e2e.test.ts (1)

110-114: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick win

afterAll cleanup isn't isolated — an early failure skips the remaining teardown.

If fileApp?.exit() rejects, etcdApp?.exit() and upstream?.close() never run, potentially leaking the spawned processes for subsequent test files.

♻️ Proposed fix
 afterAll(async () => {
-  await fileApp?.exit();
-  await etcdApp?.exit();
-  await upstream?.close();
+  await Promise.allSettled([fileApp?.exit(), etcdApp?.exit(), upstream?.close()]);
 });
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/e2e/src/cases/export-roundtrip-e2e.test.ts` around lines 110 - 114,
Update the afterAll teardown to isolate cleanup failures so a rejection from
fileApp?.exit() does not prevent etcdApp?.exit() or upstream?.close() from
running. Ensure each resource cleanup is attempted independently while
preserving the existing optional-resource behavior.
crates/aisix-server/src/export/yaml_emit.rs (1)

20-29: 🗄️ Data Integrity & Integration | 🔵 Trivial | ⚡ Quick win

Centralize the resources-file format version yaml_emit.rs and filesource/mod.rs both pin "1" separately. Keep a single shared constant so a future version bump updates the emitter and loader together.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/aisix-server/src/export/yaml_emit.rs` around lines 20 - 29, Centralize
the resources-file format version currently duplicated in yaml_emit.rs and
filesource/mod.rs into one shared public constant. Update emit_yaml and the
loader’s version validation to reference that constant, removing both local "1"
definitions so future version changes require a single update.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@crates/aisix-server/src/export/mod.rs`:
- Around line 84-96: Update the output-file handling in the export command
around args.output and Export::reveal_secrets so files written with
--reveal-secrets are created with owner-only permissions (0600) before any
secret-containing data is written. Preserve the existing write error reporting
and normal output behavior for exports without revealed secrets.

In `@crates/aisix-server/src/export/secrets.rs`:
- Around line 46-66: Add a post-pass in document.rs::build_export_document over
ExportDocument::secret_placeholders to detect repeated env_var values. Emit a
warning, matching the existing duplicate file-identity check, whenever two
distinct placeholders resolve to the same environment variable, including enough
identity context to identify both secrets. Preserve the existing placeholder
generation and other validation behavior.

In `@crates/aisix-server/src/main.rs`:
- Around line 87-89: Update the prefix argument definition in the main CLI
configuration to reuse the canonical default from aisix_core::EtcdConfig instead
of hardcoding "/aisix". Preserve the existing long option and String type while
ensuring aisix export stays aligned with future EtcdConfig prefix changes.

In `@tests/e2e/src/cases/export-roundtrip-e2e.test.ts`:
- Around line 56-66: Update the beforeAll setup around EtcdClient.ping so
unavailable etcd always follows the existing skip flow, including when
process.env.CI is set. Remove the hard Error path and ensure the test’s
established ctx.skip() handling remains responsible for skipping without
failing.

---

Nitpick comments:
In `@crates/aisix-server/src/export/secrets.rs`:
- Around line 76-183: Introduce a shared RedactionCtx<'a> containing kind_token,
kind, identity, reveal, and out, then update redact_top_level, redact_by_key,
and redact_headers to accept the context instead of their repeated five-argument
tail. Adjust recursive redact_by_key calls and all callers to pass the context
while preserving existing redaction behavior and placeholder output.

In `@crates/aisix-server/src/export/yaml_emit.rs`:
- Around line 20-29: Centralize the resources-file format version currently
duplicated in yaml_emit.rs and filesource/mod.rs into one shared public
constant. Update emit_yaml and the loader’s version validation to reference that
constant, removing both local "1" definitions so future version changes require
a single update.

In `@tests/e2e/src/cases/export-roundtrip-e2e.test.ts`:
- Around line 110-114: Update the afterAll teardown to isolate cleanup failures
so a rejection from fileApp?.exit() does not prevent etcdApp?.exit() or
upstream?.close() from running. Ensure each resource cleanup is attempted
independently while preserving the existing optional-resource behavior.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: d4ad96d1-be9d-4d5c-9125-90254ff5a78a

📥 Commits

Reviewing files that changed from the base of the PR and between 5357034 and 83acbca.

⛔ Files ignored due to path filters (1)
  • Cargo.lock is excluded by !**/*.lock
📒 Files selected for processing (7)
  • crates/aisix-server/Cargo.toml
  • crates/aisix-server/src/export/document.rs
  • crates/aisix-server/src/export/mod.rs
  • crates/aisix-server/src/export/secrets.rs
  • crates/aisix-server/src/export/yaml_emit.rs
  • crates/aisix-server/src/main.rs
  • tests/e2e/src/cases/export-roundtrip-e2e.test.ts

Comment thread crates/aisix-server/src/export/mod.rs
Comment thread crates/aisix-server/src/export/secrets.rs
Comment thread crates/aisix-server/src/main.rs Outdated
Comment thread tests/e2e/src/cases/export-roundtrip-e2e.test.ts
@moonming
moonming force-pushed the feat/aisix-export branch from 83acbca to bf7bf94 Compare July 16, 2026 02:20

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@crates/aisix-server/src/export/document.rs`:
- Around line 362-371: Make duplicate identity detection in the export flow
fatal instead of continuing to append the colliding document: when
seen.insert(id.clone()) fails, return an error or otherwise abort ordinary
export before resugar, redact, and out.push. If the exporter already exposes an
explicit lossy-mode option, only allow continuation when that mode is enabled;
otherwise preserve the existing warning context and add coverage proving
ordinary export cannot succeed with duplicate identities.
- Around line 126-148: Update guardrail export handling in the guardrails
emit_entries flow and its related import/serialization paths so attachment scope
is preserved; do not export guardrails when attachments cannot be represented
equivalently, unless the existing explicit lossy-export opt-in is enabled.
Ensure scoped guardrails cannot become gateway-wide policies after import.

In `@tests/e2e/src/cases/export-roundtrip-e2e.test.ts`:
- Around line 131-141: Extend the no-live-credential test around placeholderVars
to assert that stderr does not contain PROVIDER_SECRET, while preserving the
existing checks for the placeholder variable diagnostics and redacted YAML
output.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 9b0816a8-57fa-4ba7-91cc-b76f7e266626

📥 Commits

Reviewing files that changed from the base of the PR and between 83acbca and bf7bf94.

⛔ Files ignored due to path filters (1)
  • Cargo.lock is excluded by !**/*.lock
📒 Files selected for processing (7)
  • crates/aisix-server/Cargo.toml
  • crates/aisix-server/src/export/document.rs
  • crates/aisix-server/src/export/mod.rs
  • crates/aisix-server/src/export/secrets.rs
  • crates/aisix-server/src/export/yaml_emit.rs
  • crates/aisix-server/src/main.rs
  • tests/e2e/src/cases/export-roundtrip-e2e.test.ts
🚧 Files skipped from review as they are similar to previous changes (4)
  • crates/aisix-server/src/main.rs
  • crates/aisix-server/src/export/yaml_emit.rs
  • crates/aisix-server/src/export/mod.rs
  • crates/aisix-server/src/export/secrets.rs

Comment thread crates/aisix-server/src/export/document.rs
Comment thread crates/aisix-server/src/export/document.rs
Comment thread tests/e2e/src/cases/export-roundtrip-e2e.test.ts
@moonming
moonming force-pushed the feat/aisix-export branch from bf7bf94 to 7a24b99 Compare July 16, 2026 02:56
@moonming

Copy link
Copy Markdown
Collaborator Author

Review triage (pushed in 7a24b99)

CodeRabbit findings — all 4 addressed:

  • 🟠 Output file permissions → fixed (-o written owner-only 0600 on unix, --reveal-secrets in mind).
  • 🟠 Placeholder-name collision → fixed (post-pass warns when two identities derive the same ${VAR}).
  • 🟡 Canonical etcd-prefix default → fixed (--prefix defaults via EtcdConfig::default().prefix).
  • 🔵 RedactionCtx refactor → adopted — the redaction helpers now take a RedactionCtx { kind_token, kind, identity, reveal, out } and are 2–4 args; the same-typed kind_token/kind transposition risk is gone. (clippy's too_many_arguments independently forced this once redact_string_map landed.)

Independent audit (fresh cold-context agent, per our merge-gate) also surfaced and I fixed:

  • HIGH — provider_key request.default_headers / default_body_fields were emitted in cleartext (structurally the same outbound-auth map as the OTLP headers that were already redacted). Now redacted via redact_string_map (string values only, so a safe_prompt: true flag survives). Closes the "no live credential by default" gap on the most sensitive kind.
  • MEDIUM — cache_policy applies_to: "api_key:<id>" silently went inert on reload (file-mode api_key ids are name-derived). Now resugared to the id the loader will derive; a dangling id is kept raw + warned. model:/all pass through.
  • MEDIUM — the e2e no-leak assertion covered only one field on one kind. Added a per-kind breadth check (provider api_key + default_headers, mcp secret, guardrail api_key, OTLP headers), each asserted present-by-name and secret-absent so a dropped seed can't pass vacuously.
  • LOW — clearer duplicate-identity warning (says the file will fail to load) and a wider (16-hex) synthesized api-key name.

Gates: cargo fmt --check + cargo clippy --workspace --all-targets -D warnings clean; full workspace suite green twice (2310 tests); the export round-trip + no-leak e2e green against a live etcd; resource schemas untouched.

@moonming
moonming force-pushed the feat/aisix-export branch 2 times, most recently from c3d18b0 to 66a40ee Compare July 16, 2026 03:20
@moonming

Copy link
Copy Markdown
Collaborator Author

Second review round triaged (head 66a40ee)

CodeRabbit's re-review of the fix commit surfaced 4 more — all addressed:

  • 🟠 Scoped guardrails widening on import → the exporter is now attachment-aware. Only guardrails already gateway-wide in etcd (an env-scoped guardrail_attachment) are emitted; an attachment-scoped or unattached guardrail (which would widen to all traffic, or activate a never-attached rule) is omitted with a warning.
  • 🟠 Emitting a non-loadable file as success → diagnostics split into warnings (file still loads) vs blocking (won't load). Identity collisions and dangling model/rate-limit refs are blocking: the file is still written for inspection, but the command exits non-zero so a scripted migration can't mistake a broken export for a finished one.
  • 🟠 e2e hard-fail on etcd-unavailable under CI → removed; defers to ctx.skip() per the harness convention.
  • 🟡 stderr secret-leak assertion → both e2e tests now assert stderr carries no secret alongside the file checks.

Gates re-run on the final code: cargo fmt --check + clippy --workspace --all-targets -D warnings clean; full workspace suite green twice (2311 tests); export round-trip + per-kind no-leak e2e green against live etcd; branch rebased on current main; schemas untouched.

@moonming
moonming force-pushed the feat/aisix-export branch from 66a40ee to e6c9cef Compare July 16, 2026 03:28
Add an `aisix export` subcommand that reads the resources currently stored
in etcd and emits a resources.yaml the file source (`resources_file`) can
load back — the inverse of the file loader and the migration/backup path
for a standalone deployment moving from Admin-API-plus-etcd to the
declarative file.

Export decodes each kind through the same typed models the gateway loads
(via the shared snapshot builder), then rewrites the set into idiomatic
file form:
- resugar references: model.provider_key_id -> `provider_key: <name>`;
  rate_limit_policy.scope_ref (api_key/model) -> the referenced entry's
  name; cache_policy.applies_to `api_key:<id>` -> the id the file loader
  will derive for that key. A dangling model/rate-limit reference (or an
  identity collision) leaves the file non-loadable: it is surfaced and
  the command exits non-zero (the file is still written for inspection)
  so a scripted migration can't mistake a broken export for a finished one.
- ids are dropped (the file derives them from names); api_keys get a
  deterministic `apikey-<hash…>` display_name synthesized from their
  already-hashed key_hash, which is emitted verbatim.
- guardrails: the file has no attachment collection, so a file guardrail
  applies gateway-wide. Only guardrails already gateway-wide in etcd (an
  env-scoped attachment) are exported; an attachment-scoped or unattached
  guardrail would silently widen to all traffic on import, so it is
  omitted with a warning.

Secrets are never emitted in cleartext by default: provider_key api_key
and its request.default_headers / default_body_fields, mcp/a2a secret,
guardrail moderation credentials, and OTLP exporter auth headers are
replaced with derived `${VAR}` placeholders, and a companion "set these
before loading" list is printed to stderr (never into the file).
`--reveal-secrets` opts into inline values for air-gapped same-host
migration; the `-o` file is written owner-only (0600 on unix).

Emission reuses the file source's own YAML library so the output is
guaranteed to re-parse, and every literal string value is `$`-escaped so
a stored value containing `$` or a literal `${...}` (e.g. a guardrail
blocklist rule for `${jndi:...}`) round-trips through the loader's
interpolation instead of being re-interpreted.

Round-trip proven end to end: a unit test builds -> emits -> re-loads
through the real file source, and e2e seeds etcd -> `aisix export` ->
loads the file into a file-mode gateway -> asserts identical /v1/models,
chat completion, and allowed_models 403 against the etcd-mode gateway,
plus a per-kind no-live-credential assertion (file and stderr) at
default settings.
@moonming
moonming force-pushed the feat/aisix-export branch from e6c9cef to bc9fd43 Compare July 16, 2026 03:32

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (1)
tests/e2e/src/cases/export-roundtrip-e2e.test.ts (1)

203-273: 🔒 Security & Privacy | 🔵 Trivial | ⚡ Quick win

Cover a2a_agents.secret in the no-leak E2E test.

The test claims every secret-bearing kind but only seeds provider keys, MCP, guardrails, and OTLP. Seed a valid A2A agent with a distinct marker, assert it is exported, and check both YAML and stderr for leakage.

As per coding guidelines, prioritize source-blind E2E coverage and never serialize authentication secrets without redaction.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/e2e/src/cases/export-roundtrip-e2e.test.ts` around lines 203 - 273,
Extend the test default export redaction case around the existing marker and
seed setup to include a valid A2A agent with a distinct a2a_agents.secret
marker. Include the agent name in the exported-entry presence assertions, and
add its marker to the YAML and stderr leak checks so the test verifies redaction
while preserving the existing placeholder assertion.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@crates/aisix-server/src/export/document.rs`:
- Around line 314-327: Update the placeholder collision tracking in the loop
using var_owner so ownership includes both the placeholder identity and its raw
field path, rather than identity alone. Compare the full (identity, field) owner
when detecting collisions, while preserving the existing warning for distinct
owners that derive the same environment variable.

In `@crates/aisix-server/src/export/secrets.rs`:
- Around line 175-187: Update redact_string_map to recursively traverse nested
objects and arrays under the selected field, redacting every non-empty string
while preserving non-string scalars. Extend field-key generation and placeholder
labels to identify nested values consistently, and add coverage for nested
objects and array elements such as authentication tokens.

---

Nitpick comments:
In `@tests/e2e/src/cases/export-roundtrip-e2e.test.ts`:
- Around line 203-273: Extend the test default export redaction case around the
existing marker and seed setup to include a valid A2A agent with a distinct
a2a_agents.secret marker. Include the agent name in the exported-entry presence
assertions, and add its marker to the YAML and stderr leak checks so the test
verifies redaction while preserving the existing placeholder assertion.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 1d3cfe75-3232-4cbb-91dd-f3fb37a97926

📥 Commits

Reviewing files that changed from the base of the PR and between bf7bf94 and e6c9cef.

⛔ Files ignored due to path filters (1)
  • Cargo.lock is excluded by !**/*.lock
📒 Files selected for processing (8)
  • crates/aisix-server/Cargo.toml
  • crates/aisix-server/src/export/document.rs
  • crates/aisix-server/src/export/document_tests.rs
  • crates/aisix-server/src/export/mod.rs
  • crates/aisix-server/src/export/secrets.rs
  • crates/aisix-server/src/export/yaml_emit.rs
  • crates/aisix-server/src/main.rs
  • tests/e2e/src/cases/export-roundtrip-e2e.test.ts
🚧 Files skipped from review as they are similar to previous changes (2)
  • crates/aisix-server/src/main.rs
  • crates/aisix-server/src/export/yaml_emit.rs

Comment on lines +314 to +327
let mut var_owner: BTreeMap<&str, &str> = BTreeMap::new();
for placeholder in &placeholders {
match var_owner.get(placeholder.env_var.as_str()) {
Some(&owner) if owner != placeholder.identity => diag.warnings.push(format!(
"secret placeholder ${{{}}} is derived for both {:?} and {:?}; their names \
collapse to the same environment variable, so one real credential cannot be \
supplied to each — rename one entry to disambiguate",
placeholder.env_var, owner, placeholder.identity
)),
Some(_) => {}
None => {
var_owner.insert(&placeholder.env_var, &placeholder.identity);
}
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟠 Major | ⚡ Quick win

Include the field when detecting placeholder collisions.

var_owner treats every placeholder from the same identity as equivalent. For example, headers x-auth-token and x.auth.token on one provider key derive the same variable but receive no warning, causing one credential to feed both fields. Track (identity, field)—preferably the raw field path—as the owner.

This is the same collision class as the earlier cross-identity finding, but the same-identity variant remains unresolved.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/aisix-server/src/export/document.rs` around lines 314 - 327, Update
the placeholder collision tracking in the loop using var_owner so ownership
includes both the placeholder identity and its raw field path, rather than
identity alone. Compare the full (identity, field) owner when detecting
collisions, while preserving the existing warning for distinct owners that
derive the same environment variable.

Comment on lines +175 to +187
pub fn redact_string_map(doc: &mut Value, field: &str, label: &str, ctx: &mut RedactionCtx<'_>) {
if ctx.reveal {
return;
}
let Some(Value::Object(map)) = doc.get_mut(field) else {
return;
};
for (name, value) in map.iter_mut() {
if is_nonempty_str(value) {
let field_key = format!("{}_{}", sanitize(field), sanitize(name));
*value = ctx.placeholder(&field_key, format!("{label} {name}"));
}
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟠 Major | ⚡ Quick win

Recursively redact nested default_body_fields strings.

This loop only redacts direct string children. Values such as {"auth":{"token":"live-secret"}} or arrays retain their credentials verbatim. Traverse nested objects and arrays, preserving non-string scalars, and add nested-value coverage.

As per coding guidelines, “Do not log, serialize, or return API keys, tokens, OAuth client secrets, or authentication headers without redaction.”

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/aisix-server/src/export/secrets.rs` around lines 175 - 187, Update
redact_string_map to recursively traverse nested objects and arrays under the
selected field, redacting every non-empty string while preserving non-string
scalars. Extend field-key generation and placeholder labels to identify nested
values consistently, and add coverage for nested objects and array elements such
as authentication tokens.

Source: Coding guidelines

@moonming
moonming merged commit 1f28acb into main Jul 16, 2026
12 checks passed
@moonming
moonming deleted the feat/aisix-export branch July 16, 2026 03:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant