Skip to content

v3.0.0: Data masking / anonymization engine

Choose a tag to compare

@valehdba valehdba released this 08 Apr 20:30
· 119 commits to main since this release

pgclone v3.0.0 — Data Masking / Anonymization Engine

Clone PostgreSQL tables with column-level data anonymization applied during transfer. Masking happens server-side as SQL expressions inside COPY (SELECT ...) TO STDOUT — zero row-by-row processing overhead.

8 Masking Strategies

Strategy Example Output NULL Handling
email a***@example.com Preserves NULL
name XXXX Preserves NULL
phone ***-4567 Preserves NULL
partial Jo***son (configurable prefix/suffix) Preserves NULL
hash 5d41402abc4b2a76b9719d911017c592 (MD5, deterministic) Preserves NULL
null NULL Always NULL
random_int Random in [min, max] Always produces value
constant Fixed replacement value Always produces value

Usage

-- Clone with email and name masking
SELECT pgclone_table(
    'host=prod dbname=myapp user=readonly',
    'public', 'users', true, 'users',
    '{"mask": {"email": "email", "full_name": "name", "ssn": "null"}}'
);

-- Parameterized masks
SELECT pgclone_table(conn, 'hr', 'employees', true, 'employees_dev',
'{"mask": {"salary": {"type": "random_int", "min": 40000, "max": 200000},
"last_name": {"type": "partial", "prefix": 1, "suffix": 2}}}'
);

-- Combine with WHERE filter
SELECT pgclone_table(conn, 'public', 'users', true, 'users_dev',
'{"mask": {"email": "email"}, "where": "status = ''active''"}'
);

What's New

  • MaskRule struct and pgclone_build_mask_expr() for extensible mask strategy architecture
  • Masking applied server-side in COPY (SELECT ...) TO STDOUT — no external dependencies
  • Queries pg_catalog.pg_attribute for column names when masks active without explicit column list
  • Fully composable with existing columns, where, indexes, constraints, triggers options
  • test_schema.employees test fixture with realistic sensitive data
  • 15 new pgTAP tests (53 total, 18 test groups)

Install

git clone https://github.com/valehdba/pgclone.git
cd pgclone && make && sudo make install
CREATE EXTENSION pgclone;
SELECT pgclone_version();  -- pgclone 3.0.0