Boilersuit ETL - Extract, Transform, Obfuscate and Load Test data
A few lines of code will copy table structures and data from various sources into
your target DB system. You can use any JDBC connection as source or target, or use
the built-in executeQueryDumpFile parameter, which will have Boilersuit ETL
write into a textfile instead of an actual running DB instance:
To obfuscate data, sanitizers can be defined. Any value inserted into target will be replaced by
values generated by the configured ValueSanitizer. They will respect types, e.g. String-like attributes
will turn into random hash codes, Number-like attributes will turn into numbers:
There are a range of ValueSanitizers available, some simple, some more sophisticated - check below.
Queries accross DBs are not per se possible, but since results obtained from DB are stored as Java objects, we can solve things in a very basic way as shown in the following sample cases.
Assume you want to load only T1 (in MSSQL) entries for names that are contained in T2 (Postgres):
final Result t1Entries = mssqlSource.query("SELECT * FROM T1"); // { "joe", 50000 }, { "bob", 60000 }, ...
final Set relevantName = t1Entries.getColumnValues("name"); // { "joe", "bob" }
final Result relevantT2Entries = pgSource.query("SELECT * FROM T2" + pgSource.wherePropertyIn("name", relevantNames));
target.insert(relevantT2Entries);To obfuscate data, a range of ValueSanitizers are available. The following picture gives an overview;
for detailed information, check Javadoc comments.
