Skip to content

[Enhancement] Introduce unified sensitive-field redaction for FE SQL logs #65875

Description

@wenzhenghu

Search before asking

  • I searched the existing open and closed issues and found no similar issue.

Description

FE diagnostic logs are produced by multiple SQL processing paths, and these paths do not currently share one consistent policy for handling sensitive property values.

Removing the complete SQL text from logs makes production troubleshooting significantly harder. The desired behavior is to preserve the SQL structure, identifiers, and other non-sensitive content while redacting only values associated with confirmed sensitive fields, such as passwords, access keys, secret keys, tokens, SASL credentials, and equivalent connector-specific properties.

This issue tracks a general FE logging improvement and hardening effort. It is not intended to describe or track a specific vulnerability.

Proposed solution

Introduce a shared SQL log redaction facility with the following properties:

  1. Preserve non-sensitive SQL content and statement structure so that logs remain useful for diagnosis.
  2. Replace only the values of known sensitive fields. Reuse a central sensitive-key registry where possible and support statement-specific keys for resources, stages, routine load, catalogs, repositories, and similar property-bearing statements.
  3. Provide consistent handling for:
    • successfully parsed statements;
    • parser-error paths;
    • SQL dialect conversion paths;
    • statements whose parser or plan object is unavailable.
  4. Use a fail-safe fallback: if the redactor cannot safely process a statement, do not emit the original SQL from that path. Log metadata such as query ID, statement type, length, user, dialect, and error type for that event instead.
  5. Migrate the main FE diagnostic logging entry points to the shared facility, including:
    • StmtExecutor and ConnectProcessor;
    • HTTP statement execution endpoints;
    • parser and dialect-conversion logging;
    • Load command logging;
    • CBO/rewrite logging;
    • query profile logging.
  6. Keep the regular diagnostic-log policy aligned with, but clearly separated from, audit-log encryption behavior.

Tests

Add log-capture tests that verify:

  • known sensitive values never appear in emitted logs;
  • SQL structure and non-sensitive content remain available;
  • statement-specific sensitive keys are handled;
  • parser failures and dialect-conversion failures use the safe fallback;
  • all migrated entry points apply the same policy.

Are you willing to submit PR?

  • Yes, I am willing to submit a PR.

Code of Conduct

  • I agree to follow this project's Code of Conduct.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions