Permalink
Browse files

v1.0.0 New functions to undo partitioning. Optimization of data parti…

…tioning process. Bug fixes. See CHANGELOG for more details.
  • Loading branch information...
keithf4 committed Mar 3, 2013
1 parent b3d166d commit 78200bcd1b331df0d1d88250b13a30d3c8bc23e6
Showing with 7,232 additions and 262 deletions.
  1. +12 −0 CHANGELOG
  2. +3 −3 META.json
  3. +2 −2 README.md
  4. +57 −16 doc/pg_partman.md
  5. +1 −1 pg_partman.control
  6. +9 −11 sql/functions/create_id_function.sql
  7. +0 −68 sql/functions/create_prev_id_partition.sql
  8. +0 −96 sql/functions/create_prev_time_partition.sql
  9. +6 −6 sql/functions/create_time_function.sql
  10. +21 −10 sql/functions/create_time_partition.sql
  11. +77 −0 sql/functions/partition_data_id.sql
  12. +96 −0 sql/functions/partition_data_time.sql
  13. +13 −7 sql/functions/run_maintenance.sql
  14. +124 −0 sql/functions/undo_partition.sql
  15. +166 −0 sql/functions/undo_partition_id.sql
  16. +166 −0 sql/functions/undo_partition_time.sql
  17. +1 −0 sql/tables/tables.sql
  18. +33 −3 test/test-id-dynamic.sql
  19. +26 −2 test/test-id-static-start-100.sql
  20. +26 −3 test/test-id-static.sql
  21. +57 −14 test/{test-time-dynamic.sql → test-time-dynamic-daily.sql}
  22. +386 −0 test/test-time-dynamic-half-hour.sql
  23. +232 −0 test/test-time-dynamic-hourly.sql
  24. +232 −0 test/test-time-dynamic-monthly.sql
  25. +386 −0 test/test-time-dynamic-quarter-hour.sql
  26. +232 −0 test/test-time-dynamic-quarterly.sql
  27. +221 −0 test/test-time-dynamic-weekly.sql
  28. +231 −0 test/test-time-dynamic-yearly.sql
  29. +65 −20 test/{test-time-static.sql → test-time-static-daily.sql}
  30. +595 −0 test/test-time-static-half-hour.sql
  31. +358 −0 test/test-time-static-hourly.sql
  32. +358 −0 test/test-time-static-monthly.sql
  33. +596 −0 test/test-time-static-quarter-hour.sql
  34. +358 −0 test/test-time-static-quarterly.sql
  35. +344 −0 test/test-time-static-weekly.sql
  36. +357 −0 test/test-time-static-yearly.sql
  37. +1,385 −0 updates/pg_partman--0.4.2--1.0.0.sql
View
@@ -1,3 +1,15 @@
+1.0.0
+-- New functions to undo partitioning. These all either move or copy data from the child tables and put it into the parent. All have an option to allow you either uninherit the child tables (default) or drop them when all their data has been put into the parent.
+ -- undo_partition_time() & undo_partition_id are functions that move the data from the child partitions to the parent tables. Data is deleted from the child table and inserted to the parent. These functions allow smaller interval batches to be given as a parameter and are better able to handle larger partitioning sets.
+ -- undo_partition() can work on an any parent/child table set in PostgreSQL, not just partition sets created by pg_partman. Just pass it the name of the parent table. This method only copies the data out of the child tables instead of deleting it, allowing you to keep all the partitioned data if desired. Because of this it can only process an entire partition at a time and cannot handle batches smaller than the partition interval.
+-- Changed create_prev_id_partition() to partition_data_id() & create_prev_time_partition() to partition_data_time(). This clarifies what these actually do since they don't always create a partition nor is it always necessarily "previous" data.
+-- Changed how the above functions work to move data from parent into partitions. You can now feed them a smaller interval value for the rows that you'd like moved instead of it always moving exactly one entire partition of data. This allows smaller batch sizes when you've got a lot of data even in just one partition. That interval is now the second parameter. A third parameter can tell it how many of those interval batches you'd like to move in a single run of the function. Both of these parameters are optional. If not given, the interval defaults to the partition interval and the batch count is one (so it works exactly like it used to with no parameters but the parent table given).
+-- Partition premake system is now able to catch up if it falls behind for some reason. Also makes it so that if the premake value is increased, within the next few runs it will have that many partitions premade automatically.
+-- Bug fix: create_time_partition() & create_time_function() now handle the "timestamp with time zone" data type much better. Was getting some mismatches in the trigger rules and table constraints when timestamptz was in use on server not running in UTC/GMT time. Would cause constraint violations during data insert at certain time boundaries. If you ran into this issue, there are two ways to fix it: 1) Manually recreate the constraints for the most recent partitions and any future partitions already created. You may have to move some data around as well. 2) Use the new undo functions to move all the data back into the parent table and then repartition again using the partition_data_* functions. This will fix the issue for all partitions.
+-- Bug fix: Determining how many partitions to premake in run_maintenance() is now more accurate. Previous date math would occasionally premake 1 extra partition depending on the time differences. This can still occur with weekly partitioning due to differing month lengths (especially February) and daylight savings. Doesn't hurt anything and will self-correct.
+-- Much more complete pgTAP test suite.
+
+
0.4.2
-- The static partitioning trigger function can now handle partitions based on the configured premake value. For example, the default premake value is 4 so it can now handle data for the current partition, 4 previous partitions and 4 future partitions. Changing the premake value will cause the trigger function to be changed appropriately the next time a partition is automatically created. Except for initial setup, at no time does the automated partitioning system create old partitions (see the create_prev_* functions if you need to do this). If you change the premake value and there is no previous partition for it to put data in, it will go to the parent table.
-- create_parent() now accounts for the new static partitioning rules. For time-static, it will create the current partition as well as previous and future partitions equal to the configured premake number (default premake being 4, you will end up with 9 partitions). For id-static, it will only create previous partitions if the resulting rules handle id values greater than zero. So if you're starting from zero you will only have future partitions created, and no previous.
View
@@ -1,7 +1,7 @@
{
"name": "pg_partman",
"abstract": "Extension to manage partitioned tables by time or ID",
- "version": "0.4.2",
+ "version": "1.0.0",
"maintainer": [
"Keith Fiske <keith@omniti.com>"
],
@@ -20,9 +20,9 @@
},
"provides": {
"pg_partmaint": {
- "file": "sql/pg_partman--0.4.2.sql",
+ "file": "sql/pg_partman--1.0.0.sql",
"docfile": "doc/pg_partman.md",
- "version": "0.4.2",
+ "version": "1.0.0",
"abstract": "Extension to manage partitioned tables by time or ID"
}
},
View
@@ -23,7 +23,7 @@ Functions must either be run as a superuser or you can set the ownership of the
UPGRADE
-------
-Make sure all the upgrade scripts for the version you have installed up to the most recent version are in the $BASEDIR/share/extension folder.
+Make sure all the upgrade scripts for the version you have installed up to the most recent version are in the $/share/extension folder.
ALTER EXTENSION pg_partman UPDATE TO '<latest version>';
@@ -44,7 +44,7 @@ This will turn your table into a parent table and premake 4 future partitions an
This should be enough to get you started. Please see the pg_partman.md file in the doc folder for more information on the types of partitioning supported and what the parameters in the create_parent() function mean.
LICENSE AND COPYRIGHT
--------------------
+---------------------
PG Partition Manager (pg_partman) is released under the PostgreSQL License, a liberal Open Source license, similar to the BSD or MIT licenses.
Oops, something went wrong.

0 comments on commit 78200bc

Please sign in to comment.