introspect: -exclude config to skip transient tables/dictionaries

[orian]

Why

The committed clickhouse-schema/ dumps were carrying transient objects — ClickHouse atomic-replace temporaries (_tmp_replace_*), migration/DAG scratch tables (tmp_dag_*, tmp_person_0007, infi_clickhouse_orm_migrations_tmp), backups (*_backup, *_backup_test), staging (*_staging), backfills (*_temp_backfill). They pollute review and, because their DDL often can't be parsed, a single one aborts a node dump unless -allow-raw is set.

What

A tool-level exclude config: an HCL file of glob patterns; matching objects are skipped before their DDL is parsed.

exclude {
  patterns = ["_tmp_replace_*", "tmp_*", "*_backup", "*_backup_*", "*_staging", "*_backfill"]
}

hclexp introspect   -database posthog -exclude exclude.hcl -out posthog.hcl
hclexp dump-cluster -cluster ops -out-dir ./prod -exclude exclude.hcl

• internal/loader/hcl/exclude.go — ExcludeMatcher (globs via filepath.Match, matched against both the bare name and <db>.<name>, nil-safe) + LoadExcludeConfig decoding exclude { patterns = [...] }, validated at load.
• IntrospectWithExclude(...) wraps Introspect (which stays nil-exclude, back-compatible); processIntrospectRowsOpt skips matching rows before introspectOneObject, so an excluded object's unparseable DDL never aborts the dump and it isn't captured as raw either.
• CLI: introspect and dump-cluster gain -exclude <file>, threaded through introspectSchema/dumpNode.
• examples/exclude.hcl — a starter config covering the transient patterns observed in production dumps.

Testing

• exclude_test.go — matcher (bare + qualified globs, nil-safe); config load (parse / empty / invalid glob); and the committed examples/exclude.hcl matched against the real transient names from the dumps (_tmp_replace_5c4a29…, tmp_dag_team_*, person_distinct_id_backup, data_temp, …_temp_backfill, …_staging) while keeping genuine objects (events, sharded_query_log_archive, person).
• introspect_test.go — an excluded object with unparseable DDL is skipped in strict mode (no error, not captured as raw) — proves skip-before-parse.
• Full unit + cmd + snapshot suites, go vet, golangci-lint (0 issues), gofmt green. No live ClickHouse smoke (the row path is unit-covered with fake rows).

Notes

• Patterns are globs (not regex) — intuitive and the * covers hex-suffixed temp names. *_old/*_new are left commented in the example since some are intentionally kept (e.g. query_log_archive_old).

🤖 Generated with Claude Code