Skip to content

fix: redact credentials in client config log output - #833

Open
yuhi-sa wants to merge 1 commit into
zilliztech:mainfrom
yuhi-sa:fix/redact-credentials-in-logs
Open

fix: redact credentials in client config log output#833
yuhi-sa wants to merge 1 commit into
zilliztech:mainfrom
yuhi-sa:fix/redact-credentials-in-logs

Conversation

@yuhi-sa

@yuhi-sa yuhi-sa commented Aug 3, 2026

Copy link
Copy Markdown

Problem

Many client constructors log the connection config verbatim, e.g.:

log.info(f"AWS_OpenSearch client config: {self.db_config}")

DBConfig.to_dict() has to hand the driver its plaintext credentials, so even though configs store passwords as pydantic SecretStr, the dict that gets logged contains them in plaintext:

AWS_OpenSearch client config: {'hosts': [{'host': '...', 'port': 443}], 'http_auth': ('admin', 'MyRealPassword!'), ...}

This goes to both the console and the rotating log file (RotatingFileHandler, 10MB x 5 backups), so real database passwords persist on disk and leak into anything that collects logs (CI output, shared benchmark reports, support bundles).

The same pattern exists in 16 client files (23 log sites): aws_opensearch, oss_opensearch, aliyun_opensearch, lindorm, hologres, memorydb, pgvector, pgvectorscale, pgvecto_rs, pgdiskann, alloydb, vectorchord, adbpg, cockroachdb, oceanbase, seekdb.

Fix

  • Add a redact_sensitive() helper in vectordb_bench/backend/utils.py that recursively masks credential values:
    • keys named password / api_key / token (the same keys the existing result-file redaction masks)
    • http_auth entries: (user, secret) pairs keep the user and mask the secret; opaque auth objects (e.g. AWS4Auth used by OpenSearch Serverless) are masked entirely
  • Apply it at all 23 log sites, e.g. {self.db_config} becomes {redact_sensitive(self.db_config)}
  • Reuse the helper for the existing result-file redaction in models.py (TestResult._redact_sensitive_fields was a duplicate of the same logic, introduced in feat: Add VectorDBBench Cloud Leaderboard benchmark cases and client support #775), removing the duplication

After the fix:

AWS_OpenSearch client config: {'hosts': [{'host': '...', 'port': 443}], 'http_auth': ('admin', '**********'), ...}

Notes

  • The only observable change is in log and error-message text; the config dict passed to the drivers is untouched.
  • Result files keep the same mask string and sensitive keys as before; the shared helper additionally understands http_auth, which never appears in result files.
  • These 23 sites are all the places in the repo that log a full config dict; the remaining clients only log non-sensitive sub-keys (e.g. self.db_config['database']) and are left as-is.
  • I kept the fix mechanical (wrap at each log site) rather than changing to_dict()'s contract or adding a global logging filter, to keep the diff easy to audit. Happy to centralize it (e.g. a redacted repr on DBConfig) as a follow-up if you'd prefer.

Tests

  • Added 6 test cases for redact_sensitive() in tests/test_utils.py (sensitive keys, http_auth pair / opaque object, nested structures).
  • make lint passes and all 27 tests in tests/test_utils.py pass locally.

Client constructors log db_config / connect_config verbatim, which
writes plaintext passwords (e.g. opensearch http_auth tuples) to the
console and the rotating log file. Mask them with a shared
redact_sensitive() helper, and reuse it for the existing result-file
redaction in models.py.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@sre-ci-robot

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: yuhi-sa
To complete the pull request process, please assign xuanyang-cn after the PR has been reviewed.
You can assign the PR to them by writing /assign @xuanyang-cn in a comment when ready.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@yuhi-sa

yuhi-sa commented Aug 3, 2026

Copy link
Copy Markdown
Author

/assign @XuanYang-cn

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants