This repository has been archived by the owner on Mar 19, 2021. It is now read-only.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Related to #20 (comment), adds
VACUUM
commands for each table after data is added to them.Strictly speaking, in most cases we only need
SORT ONLY
rather thanFULL
. However the scripts doDELETE
rows ifforce_reload
is set, and the redshift docs state that the saving forSORT ONLY
isn't normally huge, so it seems prudent to opt forFULL
.I timed some
VACUUM
commands earlier and they took about five seconds to complete. I didn't check the stats beforehand though to see whether they had much to do, so that may not be a fair indication of how the import scripts will be affected in practice.You'll notice the weird extra
END;
that eachVACUUM
block starts with. This is the only way I found to work around the errorVACUUM cannot run inside a transaction block
. I also triedSET AUTOCOMMIT
, which didn't work.@rfk, r?