Key migration and management fixes

[bvscd]

No description provided.

[Copilot]

Pull request overview

This PR updates the secrets vault storage API to support key migration/copy flows (including preserving extractability), adds a dedicated CLI subcommand for copying from file-backed vaults to HashiCorp Vault, and adjusts several node RPC/serialization behaviors for improved diagnostics and toncenter-style response parity.

Changes:

• Secrets-vault: evolve Storage trait (store gains override_extractable, list_metadata simplified) and add test helpers (clear, is_empty), plus HashiCorp KV path/layout adjustments and optional mTLS HTTP client support.
• CLI: replace the generic copy command with copy-file-to-hashicorp and add a migration-oriented load helper for file JSON storage.
• Node: improve archive/log diagnostics, tweak sync bookkeeping, and adjust RPC JSON shaping/serializers + tests for updated response formats.

Reviewed changes

Copilot reviewed 31 out of 31 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
src/secrets-vault/src/vault.rs	Adapts vault wrapper to updated storage trait; exposes storage accessor and test-only clear.
src/secrets-vault/src/tests/vault/vault_trait.rs	Updates tests to new list_metadata() signature.
src/secrets-vault/src/tests/vault/vault_builder.rs	Switches tests to new vault.clear() helper.
src/secrets-vault/src/tests/storage/storage_trait.rs	Updates storage tests to new store(..., override_extractable) and list_metadata() APIs; adds lifecycle test.
src/secrets-vault/src/tests/storage/hashicorp_api.rs	Updates expected HashiCorp KV URL layouts in tests.
src/secrets-vault/src/tests/storage/file_storage_json.rs	Updates test calls to new store signature.
src/secrets-vault/src/tests/storage/file_json_migration.rs	Updates migration test calls to new store signature.
src/secrets-vault/src/tests/fixture.rs	Removes ad-hoc clear helpers in favor of trait-level test helpers.
src/secrets-vault/src/storage/storage_trait.rs	Removes ListMode, extends store, adds test-only helpers.
src/secrets-vault/src/storage/hashicorp.rs	Implements updated trait; changes listing strategy and adds test-only clear/is_empty.
src/secrets-vault/src/storage/hashicorp_token_provider.rs	Minor test import cleanup.
src/secrets-vault/src/storage/hashicorp_api.rs	Refactors KV path builders; removes unused listing; adds mTLS-capable HTTP client builder and test-only helpers.
src/secrets-vault/src/storage/file_json.rs	Implements updated store and adds migration-oriented load helper; adds test-only clear/is_empty.
src/secrets-vault/src/lib.rs	Minor formatting-only change.
src/secrets-vault/cli/README.md	Renames and documents the new copy-file-to-hashicorp command.
src/secrets-vault/cli/main.rs	Wires the new copy command into the CLI.
src/secrets-vault/cli/list.rs	Updates list to new list_metadata() signature.
src/secrets-vault/cli/copy_file_to_hashicorp.rs	New copy implementation from file vault to HashiCorp vault.
src/secrets-vault/cli/COPY_FILE_TO_HASHICORP.md	Updates runbook to use new command name.
src/secrets-vault/Cargo.toml	Switches reqwest to rustls with explicit feature set.
src/node/storage/src/archives/archive_slice.rs	Adds warning log on offsets DB inconsistency.
src/node/storage/src/archives/archive_manager.rs	Improves error context when archive file read fails.
src/node/src/sync.rs	Persists last applied MC block id when skipping already applied blocks.
src/node/src/rpc_server/tests/test_serializers.rs	Adds serializer regression tests for the new transaction/account field behavior.
src/node/src/rpc_server/tests/test_handlers.rs	Updates handler tests for new response shape / @extra placement and init shard formatting.
src/node/src/rpc_server/serializers.rs	Refactors transaction/message serialization to support multiple formats and toncenter-like output.
src/node/src/rpc_server/mod.rs	Moves @extra to REST/JSONRPC envelopes and changes JSONRPC response framing.
src/node/src/rpc_server/handlers.rs	Adjusts handlers for new serialization signature and envelope-provided @extra; adds toncenter parity tweaks.
src/node/src/network/liteserver.rs	Makes LT lookup behavior more resilient with warnings and clearer error.
src/node/src/config.rs	Updates vault listing calls to new list_metadata() signature.
src/node-control/commands/src/commands/nodectl/key_cmd.rs	Updates vault listing calls to new list_metadata() signature.

Comments suppressed due to low confidence (2)

src/secrets-vault/src/storage/hashicorp_api.rs:791

• kv_hard_delete has the same prefix/store ordering issue as list_kv_subdir (it uses .../metadata/<kv_prefix>/<rest>). When rest is like blobs/<name> or meta/<name>, the generated URL does not match the kv_meta_path(store, rest) convention and will fail to delete keys under a configured kv_prefix. Consider constructing the URL via kv_meta_path(store, Some(rest)) (or split rest into store + key) so deletes hit the correct KV v2 metadata endpoint.

    pub async fn kv_hard_delete(&self, rest: &str) -> anyhow::Result<()> {
        let url = match &self.kv_prefix {
            Some(p) => format!("{}/{}/metadata/{}/{}", self.addr, self.kv_mount, p, rest),
            None => format!("{}/{}/metadata/{}", self.addr, self.kv_mount, rest),
        };

src/secrets-vault/src/tests/storage/hashicorp_api.rs:63

• kv_user_meta_path() is used in production as kv_user_meta_path(Self::escape(secret_name)) (i.e., it expects just the secret name). These tests use values like "blobs" / "blobs/k", which implies an extra path prefix and can mask incorrect usage (e.g., generating .../data/meta/<prefix>/blobs/k). Consider updating the test inputs to plain key names to match call sites and the new data/meta/<prefix>/<key> layout.

#[test]
fn kv_meta_path_with_prefix() {
    let c = test_client("transit", "ton", Some("mainnet"));
    assert_eq!(
        c.kv_user_meta_path("blobs/k"),
        "http://vault:8200/v1/ton/data/meta/mainnet/blobs/k"
    );

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.