Skip to content

v3.1.0: Auto-discovery of sensitive data

Choose a tag to compare

@valehdba valehdba released this 08 Apr 20:31
· 116 commits to main since this release

pgclone v3.1.0 — Auto-Discovery of Sensitive Data

Automatically scan a source database schema for columns that look like sensitive data and get suggested masking strategies — ready to paste into your clone command.

New Function

SELECT pgclone_discover_sensitive(
    'host=prod dbname=myapp user=readonly',
    'public'
);

Returns JSON grouped by table:

{
  "employees": {"email": "email", "full_name": "name", "phone": "phone", "salary": "random_int", "ssn": "null"},
  "users": {"password": "hash", "api_key": "hash", "ip_address": "hash"}
}

Detection Patterns (~40 rules across 10 categories)

Category Detected Patterns
Email email, e_mail
Name first_name, last_name, full_name, firstname, lastname, *_name
Phone phone, mobile, telephone, fax
SSN / National ID ssn, social_security, national_id, tax_id
Financial salary, income, wage, compensation
Credentials password, passwd, secret, token, api_key, apikey
Address address, street, zip, zipcode, postal
Date of Birth birth, dob, date_of_birth
Credit Card card_number, credit_card, ccn, cvv
IP Address ip_address, ipaddress

Typical Workflow

-- 1. Discover what needs masking
SELECT pgclone_discover_sensitive(conn, 'public');

-- 2. Use the output directly in a masked clone
SELECT pgclone_table(conn, 'public', 'employees', true, 'employees_dev',
'{"mask": {"email": "email", "full_name": "name", "salary": "random_int", "ssn": "null"}}');

What's New

  • pgclone_discover_sensitive(conninfo, schema) SQL function
  • Case-insensitive pattern matching against column names
  • Scans regular tables and partitioned tables
  • 6 new pgTAP tests (59 total, 19 test groups)