Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

Changing partition interval after creating the table #30

Closed
terrorobe opened this Issue Jul 2, 2014 · 2 comments

Comments

Projects
None yet
2 participants

I've got a few tables where the initial partition interval estimation was quite off, either creating partitions only a few hundred megabytes in size or on the other end of the spectrum, tens of gigabytes.

Is there a way to change the interval for future segments without either interfering with the partition creation/dropping mechanic or having to undo and redo all the partitions effectively copying all the data?

What I see as a problem is, that the retention/dropping logic expects partitions to have a specific format - losing that automation would be a compromise which would be ok (at least for me).

@keithf4 keithf4 added the question label Jul 2, 2014

Owner

keithf4 commented Jul 2, 2014

You could try updating the interval column in the part_config table to your new interval and then also change the last_partition column to be the name of the last partition that would've existed had you used the new interval. I would make sure run_maintenance() is not running until you've gotten everything changed. You'll have to be careful, though, since premake already precreated other partitions with the old interval. Changing the interval I think will cause it to precreate all the new ones.
I haven't tested this out, so not 100% it will work. But yes, if that works, it will completely break the retention system for that partition set.

Honestly, I think you'd be better off re-partitioning in the long run. You can use the python script to do it in batches with a configured wait time so it would have minimal IO impact. The only issue would be the period of time when the undo is close to finishing and the next period of time while the majority of data is in the parent. Queries run during that time would likely be a lot slower on large partition sets.

Owner

keithf4 commented Jul 2, 2014

Looking at this more you'll have to update the datetime_string in part_config as well.

@keithf4 keithf4 closed this Aug 12, 2014

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment