Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

handle directory-based input for parallel pgdump #20

Open
moradology opened this issue Feb 27, 2018 · 2 comments
Open

handle directory-based input for parallel pgdump #20

moradology opened this issue Feb 27, 2018 · 2 comments

Comments

@moradology
Copy link

The OSM database is pretty hefty which means that dumping in as parallel a fashion as possible is desirable. Doing this, however, results in a directory output (https://www.postgresql.org/docs/9.4/static/backup-dump.html) and it looks right now as though planet-dump-ng expects a single file based on the documentation in planet-dump-ng --help and a little bit of digging in the source.

How difficult would it be to deal with the multipart table images created in parallel dumps?

@moradology
Copy link
Author

Actually, it looks like pg_restore is being used behind the scenes here, which means that directories are potentially OK as inputs. Perhaps this just indicates that some documentation additions would be helpful. Happy to provide the PRs if there's a desire for that

@zerebubuth
Copy link
Owner

Yup! You're absolutely right, the code just forks off a separate copy of pg_restore to read each table. Since dump_file is just a string passed through from the arguments, it should work with either a file or directory dump, although I've only tested with files.

Please try it and let me know how it goes! PRs to improve the docs or code would be very welcome.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants