Writing subset to SQL file instead of to destination DB

[DonDebonair]

Hi folks!

This looks like a wonderful tool! I'm currently using Jailer to create DB subsets. That works fine, but you have to create the configuration using a GUI and it's overall too overcomplicated for my needs. Condenser looks like it hits the sweet spot between utility and simplicity.

One thing that is missing for my particular use case: I want to extract the subset of the DB and in a separate step load the data into a new DB (multiple DBs actually, such as development, staging etc.) Condenser is built in such a way that it immediately ingests the subset into the target DB.

How hard would it be to have an option to create a SQL file as the destination?

If you take PRs and can point me to the right place in the code where changes would need to be made, I can maybe help out

[mwcaisse]

Hi Don!

Updating the code to output to a SQL file directly without an intermediate database might be challenging, as it inserts / copy rows using INSERT INTO table SELECT statements (after copying rows to a temp table of course) as seen here Plus the challenges of creating / outputting valid SQL.

If you can still have a destination database then you could always create a dump of it in SQL using either pg_dump or mysqldump and both could likely be configured to only dump data or the whole database depending on how you want to restore. Something like that could either be performed manually after, or as a config option and at the end of the subset.

We do accept PRs and if you want to take this on and submit a PR, we would be happy to review it and help you get it merged.

[DonDebonair]

Thanks for the reply @mwcaisse! I still feel it should be possible to output SQL, but maybe I'm underestimating the complexity or overestimating my skills, or both 😅 .
I'm gonna have a look in the next few weeks.