Skip to content

Commit 867c9ad

Browse files
committed
Happy birthday!
0 parents  commit 867c9ad

File tree

12 files changed

+410
-0
lines changed

12 files changed

+410
-0
lines changed

.gitignore

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
/.bundle/
2+
/.yardoc
3+
/_yardoc/
4+
/coverage/
5+
/doc/
6+
/pkg/
7+
/spec/reports/
8+
/tmp/

.travis.yml

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
---
2+
sudo: false
3+
language: ruby
4+
cache: bundler
5+
rvm:
6+
- 2.6.5
7+
before_install: gem install bundler -v 2.0.2

Gemfile

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
source "https://rubygems.org"
2+
3+
# Specify your gem's dependencies in pg_partition_manager.gemspec
4+
gemspec

Gemfile.lock

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
PATH
2+
remote: .
3+
specs:
4+
pg_partition_manager (0.1.0)
5+
pg (~> 1.0)
6+
7+
GEM
8+
remote: https://rubygems.org/
9+
specs:
10+
ast (2.4.0)
11+
jaro_winkler (1.5.3)
12+
minitest (5.12.2)
13+
parallel (1.18.0)
14+
parser (2.6.5.0)
15+
ast (~> 2.4.0)
16+
pg (1.1.4)
17+
rainbow (3.0.0)
18+
rake (10.5.0)
19+
rubocop (0.72.0)
20+
jaro_winkler (~> 1.5.1)
21+
parallel (~> 1.10)
22+
parser (>= 2.6)
23+
rainbow (>= 2.2.2, < 4.0)
24+
ruby-progressbar (~> 1.7)
25+
unicode-display_width (>= 1.4.0, < 1.7)
26+
rubocop-performance (1.4.1)
27+
rubocop (>= 0.71.0)
28+
ruby-progressbar (1.10.1)
29+
standard (0.1.4)
30+
rubocop (~> 0.72.0)
31+
rubocop-performance (~> 1.4.0)
32+
unicode-display_width (1.6.0)
33+
34+
PLATFORMS
35+
ruby
36+
37+
DEPENDENCIES
38+
bundler (~> 2.0)
39+
minitest (~> 5.0)
40+
pg_partition_manager!
41+
rake (~> 10.0)
42+
standard
43+
44+
BUNDLED WITH
45+
2.0.2

LICENSE.txt

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
The MIT License (MIT)
2+
3+
Copyright (c) 2019 Honeybadger Industries, LLC
4+
5+
Permission is hereby granted, free of charge, to any person obtaining a copy
6+
of this software and associated documentation files (the "Software"), to deal
7+
in the Software without restriction, including without limitation the rights
8+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9+
copies of the Software, and to permit persons to whom the Software is
10+
furnished to do so, subject to the following conditions:
11+
12+
The above copyright notice and this permission notice shall be included in
13+
all copies or substantial portions of the Software.
14+
15+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
21+
THE SOFTWARE.

README.md

Lines changed: 83 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,83 @@
1+
# PgPartitionManager
2+
3+
Inspired by [pg_partman][1], this gem helps you manage [partitioned tables][2] in PostgreSQL >= 10.
4+
5+
## Installation
6+
7+
Add this line to your application's Gemfile:
8+
9+
```ruby
10+
gem 'pg_partition_manager'
11+
```
12+
13+
And then execute:
14+
15+
```shell
16+
bundle
17+
```
18+
19+
Or install it yourself as:
20+
21+
```shell
22+
gem install pg_partition_manager
23+
```
24+
25+
## Usage
26+
27+
This is meant to be used via a daily cron job, to ensure that new tables are created before they are needed, and old tables are dropped when they aren't needed anymore.
28+
29+
Imagine a cron job like this:
30+
31+
```shell
32+
@daily cd /app && ./bin/bundle exec ./script/make_partitions.rb
33+
```
34+
35+
And a Ruby script like this:
36+
37+
```ruby
38+
#!/usr/bin/env/ruby
39+
40+
require "pg_partition_manager"
41+
42+
PartitionManager::Time.process([
43+
{parent_table: "public.events", period: "month", premake: 1, retain: 3},
44+
{parent_table: "public.stats", period: "week", premake: 4, retain: 4},
45+
{parent_table: "public.observations", period: "day", retain: 7},
46+
]
47+
```
48+
49+
If the cron job runs on Monday, September 30th, 2019, and you had created the `events`, `stats`, and `observations` tables in your public schema, the following tables would be created:
50+
* `public.events_p2019_09_01`
51+
* `public.events_p2019_10_01`
52+
* `public.stats_p2019_09_30`
53+
* `public.stats_p2019_10_07`
54+
* `public.stats_p2019_10_14`
55+
* `public.stats_p2019_10_21`
56+
* `public.observations_p2019_09_30`
57+
* `public.observations_p2019_10_01`
58+
* `public.observations_p2019_10_02`
59+
* `public.observations_p2019_10_03`
60+
* `public.observations_p2019_10_04`
61+
62+
The `premake` option specifies how many tables to create for dates after the current period, and the `retain` option specifies how many tables to keep for dates before the current period.
63+
64+
This gem uses the [pg gem][3] to connect to your database, and it assumes the DATABASE\_URL environment variable is populated with connection info. If this environment variable isn't defined, a connection to the server running on localhost will be attempted.
65+
66+
67+
## Development
68+
69+
After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake test` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
70+
71+
To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
72+
73+
## Contributing
74+
75+
Bug reports and pull requests are welcome on GitHub at https://github.com/honeybadger-io/pg_partition_manager.
76+
77+
## License
78+
79+
The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
80+
81+
[1]: https://github.com/pgpartman/pg_partman
82+
[2]: https://www.postgresql.org/docs/current/ddl-partitioning.html
83+
[3]: https://rubygems.org/gems/pg

Rakefile

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
require "bundler/gem_tasks"
2+
require "rake/testtask"
3+
require "standard/rake"
4+
5+
Rake::TestTask.new(:test) do |t|
6+
t.libs << "test"
7+
t.libs << "lib"
8+
t.test_files = FileList["test/**/*_test.rb"]
9+
end
10+
11+
task default: [:standard, :test]

lib/pg_partition_manager.rb

Lines changed: 90 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,90 @@
1+
require "pg_partition_manager/version"
2+
require "date"
3+
require "pg"
4+
5+
module PgPartitionManager
6+
class Error < StandardError; end
7+
8+
class Time
9+
def initialize(partition, start: Date.today, db: nil)
10+
raise ArgumentError, "Period must be 'month', 'week', or 'day'" unless ["month", "week", "day"].include?(partition[:period])
11+
12+
@partition = partition
13+
@start =
14+
case partition[:period]
15+
when "month"
16+
start - start.day + 1 # First day of the current month
17+
when "week"
18+
start - (start.cwday - 1) # First calendar day of the current week
19+
when "day"
20+
start
21+
end
22+
@db = db || PG.connect(ENV["DATABASE_URL"])
23+
end
24+
25+
# Drop the tables that contain data that should be expired based on the
26+
# retention period
27+
def drop_tables
28+
schema, table = @partition[:parent_table].split(".")
29+
table_suffix = retention.to_s.tr("-", "_")
30+
31+
result = @db.exec("select nspname, relname from pg_class c inner join pg_namespace n on n.oid = c.relnamespace where nspname = '#{schema}' and relname like '#{table}_p%' and relkind = 'r' and relname < '#{table}_p#{table_suffix}' order by 1, 2")
32+
result.map do |row|
33+
child_table = "#{row["nspname"]}.#{row["relname"]}"
34+
@db.exec("drop table if exists #{child_table}")
35+
child_table
36+
end
37+
end
38+
39+
# Create tables to hold future data
40+
def create_tables
41+
schema, table = @partition[:parent_table].split(".")
42+
start = @start
43+
stop = period_end(start)
44+
45+
# Note that this starts in the *current* period, so we start at 0 rather
46+
# than 1 for the range, to be sure the current period gets a table *and*
47+
# we make the number of desired future tables
48+
(0..(@partition[:premake] || 4)).map do |month|
49+
child_table = "#{schema}.#{table}_p#{start.to_s.tr("-", "_")}"
50+
@db.exec("create table if not exists #{child_table} partition of #{schema}.#{table} for values from ('#{start}') to ('#{stop}')")
51+
start = stop
52+
stop = period_end(start)
53+
child_table
54+
end
55+
end
56+
57+
# Return the date for the oldest table to keep, based on the retention setting
58+
def retention
59+
case @partition[:period]
60+
when "month"
61+
@start << @partition[:retain] || 6 # Default to 6 months
62+
when "week"
63+
@start - ((@partition[:retain] || 4) * 7) # Default to 4 weeks
64+
when "day"
65+
@start - (@partition[:retain] || 7) # Default to 7 days
66+
end
67+
end
68+
69+
# Return the begin and end dates for the next partition range
70+
def period_end(start)
71+
case @partition[:period]
72+
when "month"
73+
start >> 1
74+
when "week"
75+
start + 7
76+
when "day"
77+
start + 1
78+
end
79+
end
80+
81+
# A convenience method for doing all the maintenance for a list of partitions
82+
def self.process(partitions)
83+
partitions.each do |part|
84+
pm = new(part)
85+
pm.drop_tables
86+
pm.create_tables
87+
end
88+
end
89+
end
90+
end

lib/pg_partition_manager/version.rb

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
module PgPartitionManager
2+
VERSION = "0.1.0"
3+
end

pg_partition_manager.gemspec

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
lib = File.expand_path("lib", __dir__)
2+
$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
3+
require "pg_partition_manager/version"
4+
5+
Gem::Specification.new do |spec|
6+
spec.name = "pg_partition_manager"
7+
spec.version = PgPartitionManager::VERSION
8+
spec.authors = ["Benjamin Curtis"]
9+
spec.email = ["ben@honeybadger.io"]
10+
11+
spec.summary = "Manage PostgreSQL table partitions"
12+
# spec.description = %q{TODO: Write a longer description or delete this line.}
13+
spec.homepage = "https://github.com/honeybadger-io/pg_partition_manager"
14+
spec.license = "MIT"
15+
16+
spec.metadata["homepage_uri"] = spec.homepage
17+
spec.metadata["source_code_uri"] = "https://github.com/honeybadger-io/pg_partition_manager"
18+
# spec.metadata["changelog_uri"] = "TODO: Put your gem's CHANGELOG.md URL here."
19+
20+
# Specify which files should be added to the gem when it is released.
21+
# The `git ls-files -z` loads the files in the RubyGem that have been added into git.
22+
spec.files = Dir.chdir(File.expand_path("..", __FILE__)) do
23+
`git ls-files -z`.split("\x0").reject { |f| f.match(%r{^(test|spec|features)/}) }
24+
end
25+
spec.bindir = "exe"
26+
spec.executables = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
27+
spec.require_paths = ["lib"]
28+
29+
spec.add_development_dependency "bundler", "~> 2.0"
30+
spec.add_development_dependency "rake", "~> 10.0"
31+
spec.add_development_dependency "minitest", "~> 5.0"
32+
spec.add_development_dependency "standard"
33+
34+
spec.add_dependency "pg", "~> 1.0"
35+
end

0 commit comments

Comments
 (0)