feat(config): load identity blob from GCP Secret Manager#72
Merged
keanji-x merged 3 commits intoaptos-nodefrom Apr 24, 2026
Merged
feat(config): load identity blob from GCP Secret Manager#72keanji-x merged 3 commits intoaptos-nodefrom
keanji-x merged 3 commits intoaptos-nodefrom
Conversation
Adds a parallel GCP-backed variant to both `Identity::FromFile` (network
identity) and `InitialSafetyRulesConfig::FromFile` (consensus identity).
Secret payload is the same YAML bytes as the on-disk IdentityBlob, so
the only operational change is uploading the existing identity file to
Secret Manager instead of mounting it on the VM.
Config YAML:
identity:
type: from_gcp_secret
resource: projects/<P>/secrets/<S>/versions/latest
initial_safety_rules_config:
from_gcp_secret:
identity_blob_secret: projects/<P>/secrets/<S>/versions/latest
waypoint: ...
Auth: GCE metadata server (zero-config on GCE VMs with a bound SA that
has roles/secretmanager.secretAccessor and --scopes=cloud-platform), or
GCP_ACCESS_TOKEN env var for non-GCE dev/CI. Intentionally avoids
pulling the google-cloud-auth async stack into aptos-config.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… errors
Three small hardening improvements on top of the initial GCP Secret Manager
identity loader, all backwards-compatible with existing file-based configs.
1. Cargo feature `gcp-secret-manager` (off by default)
`reqwest` and `base64` are now `optional = true` in `aptos-config`.
Binaries that only load identity from disk no longer transitively link
the HTTP/TLS stack through `aptos-config`.
`gcp_secret::fetch_secret` still exists with the feature off; it returns
an error pointing the operator at the missing feature flag. Enum variants
(`Identity::FromGcpSecret`, `InitialSafetyRulesConfig::FromGcpSecret`) are
intentionally kept unconditional so downstream `match` sites compile the
same with or without the feature.
A forwarding feature is exposed on the top-level `gaptos` crate:
`gaptos/gcp-secret-manager` → `aptos-config/gcp-secret-manager`.
2. Per-process cache for fetched payloads
`NetworkConfig::identity_key` and `NetworkConfig::peer_id` each call
`IdentityBlob::from_gcp_secret` independently, and a validator runs both
a validator_network and one-or-more full_node_networks. Without the
cache, startup would issue N × (metadata_token + secretmanager.access)
HTTP calls for the same payload.
The cache lives in `gcp_secret::CACHE` (`OnceLock<Mutex<HashMap<_,_>>>`)
and is keyed by normalized resource so `projects/p/secrets/s` and
`projects/p/secrets/s/versions/latest` collapse to one entry.
3. Resource name included in load-failure messages
`network_config.rs` changes the bare `.unwrap()` to `unwrap_or_else`
with a panic message that includes the GCP resource. `safety_rules_config.rs`
adds `.with_context(|| format!("... {resource}"))` on the two
`IdentityBlob::from_gcp_secret` call sites. Failures now identify which
secret path is broken instead of surfacing a bare anyhow Error.
Verified:
- \`cargo check -p aptos-config\` (no feature) → 0 new warnings
- \`cargo check -p aptos-config --features gcp-secret-manager\` → 0 new warnings
- \`cargo check -p aptos-safety-rules\` → clean
- \`cargo test -p aptos-config --lib gcp_secret\` → 3/3 pass
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`InitialSafetyRulesConfig::overriding_identity_blob_paths_mut` is a smoke-test-only helper that returns `&mut Vec<PathBuf>`. The initial commit added the `FromGcpSecret` variant but missed this match site (it's behind `#[cfg(feature = "smoke-test")]` so `cargo check` without the feature does not surface it). With smoke tests only ever exercising the file-based identity path, the new arm is an `unreachable!()` with an explanatory message. Backwards-compatible: the method signature is unchanged; existing smoke-test callers continue to use the `FromFile` arm. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
ByteYue
approved these changes
Apr 24, 2026
3 tasks
keanji-x
added a commit
to Galxe/gravity-reth
that referenced
this pull request
Apr 24, 2026
Follows Galxe/gravity-aptos#72 which introduced Identity::FromGcpSecret and InitialSafetyRulesConfig::FromGcpSecret on the aptos-node branch. The gravity-sdk consumers import ConfigStorage from gaptos (which will bump to this same aptos-node tip in the companion sdk PR). Without this bump, greth ends up with api-types @ 1d1153b6fd while sdk's gaptos would be at the post-merge aptos-node tip — cargo sees two distinct api-types packages keyed by source rev, and the impl of `ConfigStorage for PipeExecLayerApi` here no longer resolves against sdk's imported ConfigStorage trait (fails with E0599 'fetch_config_bytes not found'). api-types contents are unchanged between 1d1153b6fd and e9544c8cb3 — the three intermediate commits only touch aptos-config/config/ (adds the GCP Secret Manager identity variants behind an off-by-default cargo feature). This bump is a pure version alignment, no behavior change in greth. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
keanji-x
added a commit
to Galxe/gravity-reth
that referenced
this pull request
Apr 25, 2026
) Follows Galxe/gravity-aptos#72 which introduced Identity::FromGcpSecret and InitialSafetyRulesConfig::FromGcpSecret on the aptos-node branch. The gravity-sdk consumers import ConfigStorage from gaptos (which will bump to this same aptos-node tip in the companion sdk PR). Without this bump, greth ends up with api-types @ 1d1153b6fd while sdk's gaptos would be at the post-merge aptos-node tip — cargo sees two distinct api-types packages keyed by source rev, and the impl of `ConfigStorage for PipeExecLayerApi` here no longer resolves against sdk's imported ConfigStorage trait (fails with E0599 'fetch_config_bytes not found'). api-types contents are unchanged between 1d1153b6fd and e9544c8cb3 — the three intermediate commits only touch aptos-config/config/ (adds the GCP Secret Manager identity variants behind an off-by-default cargo feature). This bump is a pure version alignment, no behavior change in greth. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Lchangliang
pushed a commit
that referenced
this pull request
Apr 25, 2026
* feat(config): load identity blob from GCP Secret Manager
Adds a parallel GCP-backed variant to both `Identity::FromFile` (network
identity) and `InitialSafetyRulesConfig::FromFile` (consensus identity).
Secret payload is the same YAML bytes as the on-disk IdentityBlob, so
the only operational change is uploading the existing identity file to
Secret Manager instead of mounting it on the VM.
Config YAML:
identity:
type: from_gcp_secret
resource: projects/<P>/secrets/<S>/versions/latest
initial_safety_rules_config:
from_gcp_secret:
identity_blob_secret: projects/<P>/secrets/<S>/versions/latest
waypoint: ...
Auth: GCE metadata server (zero-config on GCE VMs with a bound SA that
has roles/secretmanager.secretAccessor and --scopes=cloud-platform), or
GCP_ACCESS_TOKEN env var for non-GCE dev/CI. Intentionally avoids
pulling the google-cloud-auth async stack into aptos-config.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(config): gate gcp-secret-manager behind feature + cache + richer errors
Three small hardening improvements on top of the initial GCP Secret Manager
identity loader, all backwards-compatible with existing file-based configs.
1. Cargo feature `gcp-secret-manager` (off by default)
`reqwest` and `base64` are now `optional = true` in `aptos-config`.
Binaries that only load identity from disk no longer transitively link
the HTTP/TLS stack through `aptos-config`.
`gcp_secret::fetch_secret` still exists with the feature off; it returns
an error pointing the operator at the missing feature flag. Enum variants
(`Identity::FromGcpSecret`, `InitialSafetyRulesConfig::FromGcpSecret`) are
intentionally kept unconditional so downstream `match` sites compile the
same with or without the feature.
A forwarding feature is exposed on the top-level `gaptos` crate:
`gaptos/gcp-secret-manager` → `aptos-config/gcp-secret-manager`.
2. Per-process cache for fetched payloads
`NetworkConfig::identity_key` and `NetworkConfig::peer_id` each call
`IdentityBlob::from_gcp_secret` independently, and a validator runs both
a validator_network and one-or-more full_node_networks. Without the
cache, startup would issue N × (metadata_token + secretmanager.access)
HTTP calls for the same payload.
The cache lives in `gcp_secret::CACHE` (`OnceLock<Mutex<HashMap<_,_>>>`)
and is keyed by normalized resource so `projects/p/secrets/s` and
`projects/p/secrets/s/versions/latest` collapse to one entry.
3. Resource name included in load-failure messages
`network_config.rs` changes the bare `.unwrap()` to `unwrap_or_else`
with a panic message that includes the GCP resource. `safety_rules_config.rs`
adds `.with_context(|| format!("... {resource}"))` on the two
`IdentityBlob::from_gcp_secret` call sites. Failures now identify which
secret path is broken instead of surfacing a bare anyhow Error.
Verified:
- \`cargo check -p aptos-config\` (no feature) → 0 new warnings
- \`cargo check -p aptos-config --features gcp-secret-manager\` → 0 new warnings
- \`cargo check -p aptos-safety-rules\` → clean
- \`cargo test -p aptos-config --lib gcp_secret\` → 3/3 pass
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(config): cover FromGcpSecret in overriding_identity_blob_paths_mut
`InitialSafetyRulesConfig::overriding_identity_blob_paths_mut` is a
smoke-test-only helper that returns `&mut Vec<PathBuf>`. The initial
commit added the `FromGcpSecret` variant but missed this match site
(it's behind `#[cfg(feature = "smoke-test")]` so `cargo check` without
the feature does not surface it).
With smoke tests only ever exercising the file-based identity path, the
new arm is an `unreachable!()` with an explanatory message.
Backwards-compatible: the method signature is unchanged; existing
smoke-test callers continue to use the `FromFile` arm.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a GCP Secret Manager source for the validator identity blob alongside the existing on-disk file source. Secret payload is the same YAML bytes as the on-disk
IdentityBlob, so the ops change is just uploading the existing identity file to Secret Manager instead of mounting it on the VM.Two parallel enum variants:
Identity::FromGcpSecret(network / validator-network identity)InitialSafetyRulesConfig::FromGcpSecret(consensus identity incl. BLS key + overrides)Both carry a
projects/<P>/secrets/<S>/versions/<V>resource path (short form without/versions/<V>defaults tolatest). Fetch happens once at startup through the sameidentity_blob()/identity_key()/peer_id()call sites, so downstream consensus/network code is untouched.Pilot test (GCE VM, single-node cluster)
Verified end-to-end on
gravity-mainnet-node-la-0:--features gcp-secret-manager, deployed single-node cluster viacluster/scriptsgravity-sm-pilot-identity)from_gcp_secretin all three identity slots (safety_rules, validator_network, full_node_networks)GCP_ACCESS_TOKENfallback since VM has no bound SA)account_addressexactly matched the SM-uploaded identitymake start→ block production reached epoch 2 round ~2800 (19 blocks in 10s)Config
Auth
Picked in order:
GCP_ACCESS_TOKENenv var — escape hatch for non-GCE dev/CI (e.g.export GCP_ACCESS_TOKEN=$(gcloud auth print-access-token)).roles/secretmanager.secretAccessorand the VM is created with--scopes=cloud-platform.Deliberately does not pull
google-cloud-auth(async, drags tonic/tokio) intoaptos-config— the crate today has no async deps. Reusesreqwest+base64which are already workspace deps.Hardening commits
gcp-secret-manager(off by default).reqwestandbase64areoptional = trueinaptos-config. Binaries that only load identity from disk no longer transitively link the HTTP/TLS stack throughaptos-config. Forwarding featuregaptos/gcp-secret-manageris exposed for downstream crates.NetworkConfig::identity_key+NetworkConfig::peer_id+ safety_rules don't each issue their own metadata_token + secretmanager.access round-trip at startup..unwrap()becomesunwrap_or_elsewith a panic message that includes the GCP resource;safety_rules_config.rsadds.with_contexton the two call sites.Scope boundaries
aptos-genesis/builder.rsstill usesIdentity::from_file— genesis/keygen writes the YAML locally, then ops uploads it to Secret Manager. No reason to read from GCP during genesis.IdentityBlob::to_file(still used by keygen).safety_rulesOnDiskStorage (BLS consensus key still materializes tosecure_storage.jsonat first boot). That's a separate concern to address viaSecureBackend::Vaultor a newGcpSecretbackend — out of scope here.gravity_cli/signerGCP KMS signer — that's EVM secp256k1 signing, orthogonal to the identity blob (which carries x25519 / Ed25519 / BLS12-381).Test plan
cargo check -p aptos-config(no feature) — 0 new warningscargo check -p aptos-config --features gcp-secret-manager— 0 new warningscargo check -p aptos-safety-rules— clean (with and without smoke-test feature)cargo test -p aptos-config --lib gcp_secret— 3/3 pass (normalize_* unit tests)Companion PRs (ordered merge)
gravity-api-typespin to match this PR's merge SHA — prevents dual api-types in downstream graph (PipeExecLayerApiimpl ConfigStoragetrait mismatch). Will be opened after this merges.gaptos+grethrevs, adds cluster.tomlidentity = { source = "gcp_secret" }support and deploy.sh rendering.🤖 Generated with Claude Code