Skip to content

Commit 2f1c0f1

Browse files
committed
Merge branch 'rel_future_beta' into rel_future_update_node
2 parents 7293034 + 6c45e16 commit 2f1c0f1

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

69 files changed

+8342
-2418
lines changed

.travis.yml

+13-5
Original file line numberDiff line numberDiff line change
@@ -11,13 +11,21 @@ compiler:
1111
- gcc
1212

1313
before_install:
14-
- sudo sh ./travis/apt.postgresql.org.sh
14+
- if [[ "$TRAVIS_OS_NAME" == "linux" ]]; then sudo apt-get -y install -qq wget ca-certificates; fi
15+
- if [[ "$TRAVIS_OS_NAME" == "linux" ]]; then source ./travis/dep-ubuntu-postgres.sh; fi
16+
- if [[ "$TRAVIS_OS_NAME" == "linux" ]]; then source ./travis/dep-ubuntu-llvm.sh; fi
17+
- if [[ "$TRAVIS_OS_NAME" == "linux" ]]; then sudo apt-get update -qq; fi
1518

1619
env:
17-
- PGVERSION=9.6 CHECK_CODE=true
18-
- PGVERSION=9.6 CHECK_CODE=false
19-
- PGVERSION=9.5 CHECK_CODE=true
20-
- PGVERSION=9.5 CHECK_CODE=false
20+
global:
21+
- LLVM_VER=4.0
22+
matrix:
23+
- PG_VER=10 CHECK_CODE=true
24+
- PG_VER=10 CHECK_CODE=false
25+
- PG_VER=9.6 CHECK_CODE=true
26+
- PG_VER=9.6 CHECK_CODE=false
27+
- PG_VER=9.5 CHECK_CODE=true
28+
- PG_VER=9.5 CHECK_CODE=false
2129

2230
script: bash ./travis/pg-travis-test.sh
2331

META.json

+3-3
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
"name": "pg_pathman",
33
"abstract": "Partitioning tool",
44
"description": "The `pg_pathman` module provides optimized partitioning mechanism and functions to manage partitions.",
5-
"version": "1.3.1",
5+
"version": "1.4.1",
66
"maintainer": [
77
"Ildar Musin <i.musin@postgrespro.ru>",
88
"Dmitry Ivanov <d.ivanov@postgrespro.ru>",
@@ -22,9 +22,9 @@
2222
"generated_by": "Ildar Musin",
2323
"provides": {
2424
"pg_pathman": {
25-
"file": "pg_pathman--1.3.sql",
25+
"file": "pg_pathman--1.4.sql",
2626
"docfile": "README.md",
27-
"version": "1.3.1",
27+
"version": "1.4.1",
2828
"abstract": "Partitioning tool"
2929
}
3030
},

Makefile

+10-7
Original file line numberDiff line numberDiff line change
@@ -10,21 +10,23 @@ OBJS = src/init.o src/relation_info.o src/utils.o src/partition_filter.o \
1010
src/compat/pg_compat.o src/compat/relation_tags.o src/compat/expand_rte_hook.o \
1111
src/compat/rowmarks_fix.o src/partition_update.o $(WIN32RES)
1212

13-
PG_CPPFLAGS = -I$(CURDIR)/src/include
13+
override PG_CPPFLAGS += -I$(CURDIR)/src/include
1414

1515
EXTENSION = pg_pathman
1616

17-
EXTVERSION = 1.4
17+
EXTVERSION = 1.5
1818

1919
DATA_built = pg_pathman--$(EXTVERSION).sql
2020

2121
DATA = pg_pathman--1.0--1.1.sql \
2222
pg_pathman--1.1--1.2.sql \
23-
pg_pathman--1.2--1.3.sql
23+
pg_pathman--1.2--1.3.sql \
24+
pg_pathman--1.3--1.4.sql
2425

2526
PGFILEDESC = "pg_pathman - partitioning tool for PostgreSQL"
2627

27-
REGRESS = pathman_basic \
28+
REGRESS = pathman_array_qual \
29+
pathman_basic \
2830
pathman_bgw \
2931
pathman_calamity \
3032
pathman_callbacks \
@@ -37,16 +39,17 @@ REGRESS = pathman_basic \
3739
pathman_interval \
3840
pathman_join_clause \
3941
pathman_lateral \
42+
pathman_mergejoin \
4043
pathman_only \
44+
pathman_param_upd_del \
4145
pathman_permissions \
46+
pathman_rebuild_updates \
4247
pathman_rowmarks \
4348
pathman_runtime_nodes \
44-
pathman_update_trigger \
45-
pathman_updates \
4649
pathman_update_node \
50+
pathman_update_trigger \
4751
pathman_utility_stmt
4852

49-
5053
EXTRA_REGRESS_OPTS=--temp-config=$(top_srcdir)/$(subdir)/conf.add
5154

5255
EXTRA_CLEAN = pg_pathman--$(EXTVERSION).sql ./isolation_output

README.md

+63-35
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,12 @@
77

88
The `pg_pathman` module provides optimized partitioning mechanism and functions to manage partitions.
99

10-
The extension is compatible with PostgreSQL 9.5+.
10+
The extension is compatible with:
11+
* PostgreSQL 9.5, 9.6, 10;
12+
* Postgres Pro Standard 9.5, 9.6;
13+
* Postgres Pro Enterprise;
14+
15+
By the way, we have a growing Wiki [out there](https://github.com/postgrespro/pg_pathman/wiki).
1116

1217
## Overview
1318
**Partitioning** means splitting one large table into smaller pieces. Each row in such table is moved to a single partition according to the partitioning key. PostgreSQL supports partitioning via table inheritance: each partition must be created as a child table with CHECK CONSTRAINT. For example:
@@ -41,6 +46,7 @@ More interesting features are yet to come. Stay tuned!
4146
## Feature highlights
4247

4348
* HASH and RANGE partitioning schemes;
49+
* Partitioning by expression and composite key;
4450
* Both automatic and manual partition management;
4551
* Support for integer, floating point, date and other types, including domains;
4652
* Effective query planning for partitioned tables (JOINs, subselects etc);
@@ -55,9 +61,11 @@ More interesting features are yet to come. Stay tuned!
5561
* Various GUC toggles and configurable settings.
5662

5763
## Roadmap
64+
65+
* Multi-level partitioning (ver 1.5);
66+
* Improved referential integrity + foreign keys on partitioned tables (ver 1.5);
5867

59-
* Implement LIST partitioning scheme;
60-
* Optimize hash join (both tables are partitioned by join key).
68+
Take a look at [this page](https://github.com/postgrespro/pg_pathman/wiki/Roadmap);
6169

6270
## Installation guide
6371
To install `pg_pathman`, execute this in the module's directory:
@@ -97,47 +105,49 @@ SET pg_pathman.enable = t;
97105
### Partition creation
98106
```plpgsql
99107
create_hash_partitions(relation REGCLASS,
100-
attribute TEXT,
108+
expr TEXT,
101109
partitions_count INTEGER,
102110
partition_data BOOLEAN DEFAULT TRUE,
103111
partition_names TEXT[] DEFAULT NULL,
104112
tablespaces TEXT[] DEFAULT NULL)
105113
```
106-
Performs HASH partitioning for `relation` by integer key `attribute`. The `partitions_count` parameter specifies the number of partitions to create; it cannot be changed afterwards. If `partition_data` is `true` then all the data will be automatically copied from the parent table to partitions. Note that data migration may took a while to finish and the table will be locked until transaction commits. See `partition_table_concurrently()` for a lock-free way to migrate data. Partition creation callback is invoked for each partition if set beforehand (see `set_init_callback()`).
114+
Performs HASH partitioning for `relation` by partitioning expression `expr`. The `partitions_count` parameter specifies the number of partitions to create; it cannot be changed afterwards. If `partition_data` is `true` then all the data will be automatically copied from the parent table to partitions. Note that data migration may took a while to finish and the table will be locked until transaction commits. See `partition_table_concurrently()` for a lock-free way to migrate data. Partition creation callback is invoked for each partition if set beforehand (see `set_init_callback()`).
107115

108116
```plpgsql
109-
create_range_partitions(relation REGCLASS,
110-
attribute TEXT,
111-
start_value ANYELEMENT,
112-
p_interval ANYELEMENT,
113-
p_count INTEGER DEFAULT NULL
114-
partition_data BOOLEAN DEFAULT TRUE)
117+
create_range_partitions(relation REGCLASS,
118+
expression TEXT,
119+
start_value ANYELEMENT,
120+
p_interval ANYELEMENT,
121+
p_count INTEGER DEFAULT NULL
122+
partition_data BOOLEAN DEFAULT TRUE)
123+
124+
create_range_partitions(relation REGCLASS,
125+
expression TEXT,
126+
start_value ANYELEMENT,
127+
p_interval INTERVAL,
128+
p_count INTEGER DEFAULT NULL,
129+
partition_data BOOLEAN DEFAULT TRUE)
115130

116-
create_range_partitions(relation REGCLASS,
117-
attribute TEXT,
118-
start_value ANYELEMENT,
119-
p_interval INTERVAL,
120-
p_count INTEGER DEFAULT NULL,
121-
partition_data BOOLEAN DEFAULT TRUE)
131+
create_range_partitions(relation REGCLASS,
132+
expression TEXT,
133+
bounds ANYARRAY,
134+
partition_names TEXT[] DEFAULT NULL,
135+
tablespaces TEXT[] DEFAULT NULL,
136+
partition_data BOOLEAN DEFAULT TRUE)
122137
```
123-
Performs RANGE partitioning for `relation` by partitioning key `attribute`, `start_value` argument specifies initial value, `p_interval` sets the default range for auto created partitions or partitions created with `append_range_partition()` or `prepend_range_partition()` (if `NULL` then auto partition creation feature won't work), `p_count` is the number of premade partitions (if not set then `pg_pathman` tries to determine it based on attribute values). Partition creation callback is invoked for each partition if set beforehand.
138+
Performs RANGE partitioning for `relation` by partitioning expression `expr`, `start_value` argument specifies initial value, `p_interval` sets the default range for auto created partitions or partitions created with `append_range_partition()` or `prepend_range_partition()` (if `NULL` then auto partition creation feature won't work), `p_count` is the number of premade partitions (if not set then `pg_pathman` tries to determine it based on expression's values). The `bounds` array can be built using `generate_range_bounds()`. Partition creation callback is invoked for each partition if set beforehand.
124139

125140
```plpgsql
126-
create_partitions_from_range(relation REGCLASS,
127-
attribute TEXT,
128-
start_value ANYELEMENT,
129-
end_value ANYELEMENT,
130-
p_interval ANYELEMENT,
131-
partition_data BOOLEAN DEFAULT TRUE)
141+
generate_range_bounds(p_start ANYELEMENT,
142+
p_interval INTERVAL,
143+
p_count INTEGER)
132144

133-
create_partitions_from_range(relation REGCLASS,
134-
attribute TEXT,
135-
start_value ANYELEMENT,
136-
end_value ANYELEMENT,
137-
p_interval INTERVAL,
138-
partition_data BOOLEAN DEFAULT TRUE)
145+
generate_range_bounds(p_start ANYELEMENT,
146+
p_interval ANYELEMENT,
147+
p_count INTEGER)
139148
```
140-
Performs RANGE-partitioning from specified range for `relation` by partitioning key `attribute`. Partition creation callback is invoked for each partition if set beforehand.
149+
Builds `bounds` array for `create_range_partitions()`.
150+
141151

142152
### Data migration
143153

@@ -157,7 +167,7 @@ Stops a background worker performing a concurrent partitioning task. Note: worke
157167
```plpgsql
158168
create_hash_update_trigger(parent REGCLASS)
159169
```
160-
Creates the trigger on UPDATE for HASH partitions. The UPDATE trigger isn't created by default because of the overhead. It's useful in cases when the key attribute might change.
170+
Creates the trigger on UPDATE for HASH partitions. The UPDATE trigger isn't created by default because of the overhead. It's useful in cases when the partitioning expression's value might change.
161171
```plpgsql
162172
create_range_update_trigger(parent REGCLASS)
163173
```
@@ -297,9 +307,10 @@ When INSERTing new data beyond the partitioning range, use SpawnPartitionsWorker
297307
```plpgsql
298308
CREATE TABLE IF NOT EXISTS pathman_config (
299309
partrel REGCLASS NOT NULL PRIMARY KEY,
300-
attname TEXT NOT NULL,
310+
expr TEXT NOT NULL,
301311
parttype INTEGER NOT NULL,
302-
range_interval TEXT);
312+
range_interval TEXT,
313+
cooked_expr TEXT);
303314
```
304315
This table stores a list of partitioned tables.
305316

@@ -341,7 +352,7 @@ RETURNS TABLE (
341352
parent REGCLASS,
342353
partition REGCLASS,
343354
parttype INT4,
344-
partattr TEXT,
355+
expr TEXT,
345356
range_min TEXT,
346357
range_max TEXT)
347358
AS 'pg_pathman', 'show_partition_list_internal'
@@ -352,6 +363,23 @@ AS SELECT * FROM show_partition_list();
352363
```
353364
This view lists all existing partitions, as well as their parents and range boundaries (NULL for HASH partitions).
354365

366+
#### `pathman_cache_stats` --- per-backend memory consumption
367+
```plpgsql
368+
-- helper SRF function
369+
CREATE OR REPLACE FUNCTION @extschema@.show_cache_stats()
370+
RETURNS TABLE (
371+
context TEXT,
372+
size INT8,
373+
used INT8,
374+
entries INT8)
375+
AS 'pg_pathman', 'show_cache_stats_internal'
376+
LANGUAGE C STRICT;
377+
378+
CREATE OR REPLACE VIEW @extschema@.pathman_cache_stats
379+
AS SELECT * FROM @extschema@.show_cache_stats();
380+
```
381+
Shows memory consumption of various caches.
382+
355383

356384
## Custom plan nodes
357385
`pg_pathman` provides a couple of [custom plan nodes](https://wiki.postgresql.org/wiki/CustomScanAPI) which aim to reduce execution time, namely:

0 commit comments

Comments
 (0)