Allow alter column type on distributed hypertable #3592

pmwkaa · 2021-09-17T09:27:29Z

Unlock and support ALTER TABLE ALTER TYPE on distributed
hypertable.

codecov · 2021-09-17T09:41:42Z

Codecov Report

Merging #3592 (8f36ddf) into master (98f4cd6) will decrease coverage by 0.00%.
The diff coverage is 97.22%.

@@            Coverage Diff             @@
##           master    #3592      +/-   ##
==========================================
- Coverage   90.80%   90.79%   -0.01%     
==========================================
  Files         212      212              
  Lines       36528    36531       +3     
==========================================
+ Hits        33168    33169       +1     
- Misses       3360     3362       +2

Impacted Files	Coverage Δ
src/guc.c	`100.00% <ø> (ø)`
src/import/allpaths.c	`80.99% <ø> (ø)`
src/nodes/chunk_dispatch_plan.c	`100.00% <ø> (ø)`
src/nodes/chunk_insert_state.c	`97.76% <ø> (ø)`
tsl/src/debug.c	`40.21% <ø> (ø)`
tsl/src/remote/dist_ddl.c	`97.02% <ø> (-0.75%)`	⬇️
src/nodes/hypertable_insert.c	`96.25% <93.75%> (-0.14%)`	⬇️
src/copy.c	`90.40% <100.00%> (ø)`
src/hypertable.c	`88.02% <100.00%> (ø)`
src/nodes/chunk_dispatch.c	`98.21% <100.00%> (ø)`
... and 7 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 55840eb...8f36ddf. Read the comment docs.

svenklemm

I think this needs a couple more tests. Any ALTER TYPE on partitioning columns should be blocked. Also we should have an example with a conversion function with USING, e.g. alter table x alter COLUMN float_col type int USING float_col::int;

gayyappan

We need a test with distributed hypertables that have compressed chunks: ALTER type after chunks are compressed

erimatnor · 2021-09-22T09:24:08Z

tsl/test/expected/dist_ddl.out

+ERROR:  cannot change the type of a hash-partitioned column
+\set ON_ERROR_STOP 1
+-- Should be able to change if not hash partitioned though
+ALTER TABLE disttable ALTER COLUMN time TYPE timestamp;


Are we sure it is safe to do this? For instance, changing the time type from bigint to int might change the min/max allowed value and expand or shrink the check constraints on chunks.

I think it might be safest to block changes in types of all partitioning columns. Perhaps going from int to bigint is safe, but we'd have to think through the cases.

I am actually surprised this isn't already blocked in TimescaleDB. Doesn't seem to be a multi-node specific issue.

I was actually following the same test used in test/partition.sql for this https://github.com/timescale/timescaledb/blob/master/test/sql/partition.sql#L54

Sure, but I am still not sure this is safe to do. I don't remember the reasoning behind allowing changing the type of non-hash-partitioned columns. We should think about blocking it in another PR.

It is not safe to do in general, it will only work for a limited subset of alter table alter type

gayyappan

Changes look good to me.

erimatnor · 2021-10-01T12:02:28Z

tsl/test/expected/dist_ddl.out

+ERROR:  cannot change the type of a hash-partitioned column
+\set ON_ERROR_STOP 1
+-- Should be able to change if not hash partitioned though
+ALTER TABLE disttable ALTER COLUMN time TYPE timestamp;


Sure, but I am still not sure this is safe to do. I don't remember the reasoning behind allowing changing the type of non-hash-partitioned columns. We should think about blocking it in another PR.

fabriziomello

LGTM

Unlock and support ALTER TABLE ALTER TYPE on distributed hypertable. Fix timescale#3582

@binakot

This release adds major new features since the 2.4.2 release. We deem it moderate priority for upgrading. This release includes these noteworthy features: * Continuous Aggregates for Distributed Hypertables * Support for PostgreSQL 14 * Experimental: Support for timezones in `time_bucket_ng()`, including the `origin` argument This release also includes several bug fixes. **Features** * timescale#3034 Add support for PostgreSQL 14 * timescale#3435 Add continuous aggregates for distributed hypertables * timescale#3505 Add support for timezones in `time_bucket_ng()` * timescale#3598 Improve evaluation of stable functions such as now() on access node * timescale#3717 Support transparent decompression on individual chunks **Bugfixes** * timescale#3580 Fix memory context bug executing TRUNCATE * timescale#3592 Allow alter column type on distributed hypertable * timescale#3618 Fix execution of refresh_caggs from user actions * timescale#3625 Add shared dependencies when creating chunk * timescale#3626 Fix memory context bug executing TRUNCATE * timescale#3627 Schema qualify UDTs in multi-node * timescale#3638 Allow owner change of a data node * timescale#3654 Fix index attnum mapping in reorder_chunk * timescale#3661 Fix SkipScan path generation with constant DISTINCT column * timescale#3667 Fix compress_policy for multi txn handling * timescale#3673 Fix distributed hypertable DROP within a procedure * timescale#3701 Allow anyone to use size utilities on distributed hypertables * timescale#3708 Fix crash in get_aggsplit * timescale#3709 Fix ordered append pathkey check * timescale#3712 Fix GRANT/REVOKE ALL IN SCHEMA handling * timescale#3724 Fix inserts into compressed chunks on hypertables with caggs * timescale#3727 Fix DirectFunctionCall crash in distributed_exec * timescale#3728 Fix SkipScan with varchar column * timescale#3733 Fix ANALYZE crash with custom statistics for custom types * timescale#3747 Always reset expr context in DecompressChunk **Thanks** * @binakot and @sebvett for reporting an issue with DISTINCT queries * @hardikm10, @DavidPavlicek and @pafiti for reporting bugs on TRUNCATE * @mjf for reporting an issue with ordered append and JOINs * @phemmer for reporting the issues on multinode with aggregate queries and evaluation of now() * @abolognino for reporting an issue with INSERTs into compressed hypertables that have cagg * @tanglebones for reporting the ANALYZE crash with custom types on multinode

@binakot

This release adds major new features since the 2.4.2 release. We deem it moderate priority for upgrading. This release includes these noteworthy features: * Continuous Aggregates for Distributed Hypertables * Support for PostgreSQL 14 * Experimental: Support for timezones in `time_bucket_ng()`, including the `origin` argument This release also includes several bug fixes. **Features** * timescale#3034 Add support for PostgreSQL 14 * timescale#3435 Add continuous aggregates for distributed hypertables * timescale#3505 Add support for timezones in `time_bucket_ng()` **Bugfixes** * timescale#3580 Fix memory context bug executing TRUNCATE * timescale#3592 Allow alter column type on distributed hypertable * timescale#3598 Improve evaluation of stable functions such as now() on access node * timescale#3618 Fix execution of refresh_caggs from user actions * timescale#3625 Add shared dependencies when creating chunk * timescale#3626 Fix memory context bug executing TRUNCATE * timescale#3627 Schema qualify UDTs in multi-node * timescale#3638 Allow owner change of a data node * timescale#3654 Fix index attnum mapping in reorder_chunk * timescale#3661 Fix SkipScan path generation with constant DISTINCT column * timescale#3667 Fix compress_policy for multi txn handling * timescale#3673 Fix distributed hypertable DROP within a procedure * timescale#3701 Allow anyone to use size utilities on distributed hypertables * timescale#3708 Fix crash in get_aggsplit * timescale#3709 Fix ordered append pathkey check * timescale#3712 Fix GRANT/REVOKE ALL IN SCHEMA handling * timescale#3717 Support transparent decompression on individual chunks * timescale#3724 Fix inserts into compressed chunks on hypertables with caggs * timescale#3727 Fix DirectFunctionCall crash in distributed_exec * timescale#3728 Fix SkipScan with varchar column * timescale#3733 Fix ANALYZE crash with custom statistics for custom types * timescale#3747 Always reset expr context in DecompressChunk **Thanks** * @binakot and @sebvett for reporting an issue with DISTINCT queries * @hardikm10, @DavidPavlicek and @pafiti for reporting bugs on TRUNCATE * @mjf for reporting an issue with ordered append and JOINs * @phemmer for reporting the issues on multinode with aggregate queries and evaluation of now() * @abolognino for reporting an issue with INSERTs into compressed hypertables that have cagg * @tanglebones for reporting the ANALYZE crash with custom types on multinode

@binakot

This release adds major new features since the 2.4.2 release. We deem it moderate priority for upgrading. This release includes these noteworthy features: * Continuous Aggregates for Distributed Hypertables * Support for PostgreSQL 14 * Experimental: Support for timezones in `time_bucket_ng()`, including the `origin` argument This release also includes several bug fixes. **Features** * #3034 Add support for PostgreSQL 14 * #3435 Add continuous aggregates for distributed hypertables * #3505 Add support for timezones in `time_bucket_ng()` **Bugfixes** * #3580 Fix memory context bug executing TRUNCATE * #3592 Allow alter column type on distributed hypertable * #3598 Improve evaluation of stable functions such as now() on access node * #3618 Fix execution of refresh_caggs from user actions * #3625 Add shared dependencies when creating chunk * #3626 Fix memory context bug executing TRUNCATE * #3627 Schema qualify UDTs in multi-node * #3638 Allow owner change of a data node * #3654 Fix index attnum mapping in reorder_chunk * #3661 Fix SkipScan path generation with constant DISTINCT column * #3667 Fix compress_policy for multi txn handling * #3673 Fix distributed hypertable DROP within a procedure * #3701 Allow anyone to use size utilities on distributed hypertables * #3708 Fix crash in get_aggsplit * #3709 Fix ordered append pathkey check * #3712 Fix GRANT/REVOKE ALL IN SCHEMA handling * #3717 Support transparent decompression on individual chunks * #3724 Fix inserts into compressed chunks on hypertables with caggs * #3727 Fix DirectFunctionCall crash in distributed_exec * #3728 Fix SkipScan with varchar column * #3733 Fix ANALYZE crash with custom statistics for custom types * #3747 Always reset expr context in DecompressChunk **Thanks** * @binakot and @sebvett for reporting an issue with DISTINCT queries * @hardikm10, @DavidPavlicek and @pafiti for reporting bugs on TRUNCATE * @mjf for reporting an issue with ordered append and JOINs * @phemmer for reporting the issues on multinode with aggregate queries and evaluation of now() * @abolognino for reporting an issue with INSERTs into compressed hypertables that have cagg * @tanglebones for reporting the ANALYZE crash with custom types on multinode

@binakot

This release adds major new features since the 2.4.2 release. We deem it moderate priority for upgrading. This release includes these noteworthy features: * Continuous Aggregates for Distributed Hypertables * Support for PostgreSQL 14 * Experimental: Support for timezones in `time_bucket_ng()`, including the `origin` argument This release also includes several bug fixes. **Features** * #3034 Add support for PostgreSQL 14 * #3435 Add continuous aggregates for distributed hypertables * #3505 Add support for timezones in `time_bucket_ng()` **Bugfixes** * #3580 Fix memory context bug executing TRUNCATE * #3592 Allow alter column type on distributed hypertable * #3598 Improve evaluation of stable functions such as now() on access node * #3618 Fix execution of refresh_caggs from user actions * #3625 Add shared dependencies when creating chunk * #3626 Fix memory context bug executing TRUNCATE * #3627 Schema qualify UDTs in multi-node * #3638 Allow owner change of a data node * #3654 Fix index attnum mapping in reorder_chunk * #3661 Fix SkipScan path generation with constant DISTINCT column * #3667 Fix compress_policy for multi txn handling * #3673 Fix distributed hypertable DROP within a procedure * #3701 Allow anyone to use size utilities on distributed hypertables * #3708 Fix crash in get_aggsplit * #3709 Fix ordered append pathkey check * #3712 Fix GRANT/REVOKE ALL IN SCHEMA handling * #3717 Support transparent decompression on individual chunks * #3724 Fix inserts into compressed chunks on hypertables with caggs * #3727 Fix DirectFunctionCall crash in distributed_exec * #3728 Fix SkipScan with varchar column * #3733 Fix ANALYZE crash with custom statistics for custom types * #3747 Always reset expr context in DecompressChunk **Thanks** * @binakot and @sebvett for reporting an issue with DISTINCT queries * @hardikm10, @DavidPavlicek and @pafiti for reporting bugs on TRUNCATE * @mjf for reporting an issue with ordered append and JOINs * @phemmer for reporting the issues on multinode with aggregate queries and evaluation of now() * @abolognino for reporting an issue with INSERTs into compressed hypertables that have cagg * @tanglebones for reporting the ANALYZE crash with custom types on multinode

@binakot

This release adds major new features since the 2.4.2 release. We deem it moderate priority for upgrading. This release includes these noteworthy features: * Continuous Aggregates for Distributed Hypertables * Support for PostgreSQL 14 * Experimental: Support for timezones in `time_bucket_ng()`, including the `origin` argument This release also includes several bug fixes. **Features** * #3034 Add support for PostgreSQL 14 * #3435 Add continuous aggregates for distributed hypertables * #3505 Add support for timezones in `time_bucket_ng()` **Bugfixes** * #3580 Fix memory context bug executing TRUNCATE * #3592 Allow alter column type on distributed hypertable * #3598 Improve evaluation of stable functions such as now() on access node * #3618 Fix execution of refresh_caggs from user actions * #3625 Add shared dependencies when creating chunk * #3626 Fix memory context bug executing TRUNCATE * #3627 Schema qualify UDTs in multi-node * #3638 Allow owner change of a data node * #3654 Fix index attnum mapping in reorder_chunk * #3661 Fix SkipScan path generation with constant DISTINCT column * #3667 Fix compress_policy for multi txn handling * #3673 Fix distributed hypertable DROP within a procedure * #3701 Allow anyone to use size utilities on distributed hypertables * #3708 Fix crash in get_aggsplit * #3709 Fix ordered append pathkey check * #3712 Fix GRANT/REVOKE ALL IN SCHEMA handling * #3717 Support transparent decompression on individual chunks * #3724 Fix inserts into compressed chunks on hypertables with caggs * #3727 Fix DirectFunctionCall crash in distributed_exec * #3728 Fix SkipScan with varchar column * #3733 Fix ANALYZE crash with custom statistics for custom types * #3747 Always reset expr context in DecompressChunk **Thanks** * @binakot and @sebvett for reporting an issue with DISTINCT queries * @hardikm10, @DavidPavlicek and @pafiti for reporting bugs on TRUNCATE * @mjf for reporting an issue with ordered append and JOINs * @phemmer for reporting the issues on multinode with aggregate queries and evaluation of now() * @abolognino for reporting an issue with INSERTs into compressed hypertables that have cagg * @tanglebones for reporting the ANALYZE crash with custom types on multinode

@binakot

This release adds major new features since the 2.4.2 release. We deem it moderate priority for upgrading. This release includes these noteworthy features: * Continuous Aggregates for Distributed Hypertables * Support for PostgreSQL 14 * Experimental: Support for timezones in `time_bucket_ng()`, including the `origin` argument This release also includes several bug fixes. **Features** * #3034 Add support for PostgreSQL 14 * #3435 Add continuous aggregates for distributed hypertables * #3505 Add support for timezones in `time_bucket_ng()` **Bugfixes** * #3580 Fix memory context bug executing TRUNCATE * #3592 Allow alter column type on distributed hypertable * #3598 Improve evaluation of stable functions such as now() on access node * #3618 Fix execution of refresh_caggs from user actions * #3625 Add shared dependencies when creating chunk * #3626 Fix memory context bug executing TRUNCATE * #3627 Schema qualify UDTs in multi-node * #3638 Allow owner change of a data node * #3654 Fix index attnum mapping in reorder_chunk * #3661 Fix SkipScan path generation with constant DISTINCT column * #3667 Fix compress_policy for multi txn handling * #3673 Fix distributed hypertable DROP within a procedure * #3701 Allow anyone to use size utilities on distributed hypertables * #3708 Fix crash in get_aggsplit * #3709 Fix ordered append pathkey check * #3712 Fix GRANT/REVOKE ALL IN SCHEMA handling * #3717 Support transparent decompression on individual chunks * #3724 Fix inserts into compressed chunks on hypertables with caggs * #3727 Fix DirectFunctionCall crash in distributed_exec * #3728 Fix SkipScan with varchar column * #3733 Fix ANALYZE crash with custom statistics for custom types * #3747 Always reset expr context in DecompressChunk **Thanks** * @binakot and @sebvett for reporting an issue with DISTINCT queries * @hardikm10, @DavidPavlicek and @pafiti for reporting bugs on TRUNCATE * @mjf for reporting an issue with ordered append and JOINs * @phemmer for reporting the issues on multinode with aggregate queries and evaluation of now() * @abolognino for reporting an issue with INSERTs into compressed hypertables that have cagg * @tanglebones for reporting the ANALYZE crash with custom types on multinode

pmwkaa requested a review from a team as a code owner September 17, 2021 09:27

pmwkaa requested review from berkley, fabriziomello and duncan-tsdb and removed request for a team September 17, 2021 09:27

svenklemm reviewed Sep 17, 2021

View reviewed changes

gayyappan reviewed Sep 17, 2021

View reviewed changes

pmwkaa force-pushed the dist_alter_type branch from 5d4f244 to 12dd156 Compare September 20, 2021 12:37

pmwkaa requested review from svenklemm and gayyappan September 20, 2021 12:38

erimatnor reviewed Sep 22, 2021

View reviewed changes

NunoFilipeSantos assigned pmwkaa Sep 23, 2021

gayyappan approved these changes Sep 24, 2021

View reviewed changes

erimatnor approved these changes Oct 1, 2021

View reviewed changes

fabriziomello approved these changes Oct 1, 2021

View reviewed changes

pmwkaa force-pushed the dist_alter_type branch 2 times, most recently from 21720f8 to 9f234bb Compare October 4, 2021 13:00

Allow alter column type on distributed hypertable

8f36ddf

Unlock and support ALTER TABLE ALTER TYPE on distributed hypertable. Fix timescale#3582

pmwkaa force-pushed the dist_alter_type branch from 9f234bb to 8f36ddf Compare October 4, 2021 14:18

pmwkaa merged commit a60b2e5 into timescale:master Oct 4, 2021

fabriziomello mentioned this pull request Oct 27, 2021

Release 2.5.0 #3744

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Allow alter column type on distributed hypertable #3592

Allow alter column type on distributed hypertable #3592

pmwkaa commented Sep 17, 2021

codecov bot commented Sep 17, 2021 •

edited

Loading

svenklemm left a comment

gayyappan left a comment •

edited

Loading

erimatnor Sep 22, 2021

pmwkaa Sep 23, 2021 •

edited

Loading

erimatnor Oct 1, 2021

svenklemm Oct 2, 2021

gayyappan left a comment

erimatnor Oct 1, 2021

fabriziomello left a comment

Allow alter column type on distributed hypertable #3592

Allow alter column type on distributed hypertable #3592

Conversation

pmwkaa commented Sep 17, 2021

codecov bot commented Sep 17, 2021 • edited Loading

Codecov Report

svenklemm left a comment

Choose a reason for hiding this comment

gayyappan left a comment • edited Loading

Choose a reason for hiding this comment

erimatnor Sep 22, 2021

Choose a reason for hiding this comment

pmwkaa Sep 23, 2021 • edited Loading

Choose a reason for hiding this comment

erimatnor Oct 1, 2021

Choose a reason for hiding this comment

svenklemm Oct 2, 2021

Choose a reason for hiding this comment

gayyappan left a comment

Choose a reason for hiding this comment

erimatnor Oct 1, 2021

Choose a reason for hiding this comment

fabriziomello left a comment

Choose a reason for hiding this comment

codecov bot commented Sep 17, 2021 •

edited

Loading

gayyappan left a comment •

edited

Loading

pmwkaa Sep 23, 2021 •

edited

Loading