Fix tablespace for compressed hypertable and corresponding toast #5525

konskov · 2023-04-05T07:03:18Z

If a hypertable uses a non default tablespace, the compressed hypertable
and its corresponding toast table and index is still created in the
default tablespace.
This PR fixes this unexpected behavior and creates the compressed
hypertable and its toast table and index in the same tablespace as
the hypertable.

Fixes #5520

codecov · 2023-04-24T07:29:11Z

Codecov Report

Merging #5525 (5cdf106) into main (df32ad4) will decrease coverage by 0.26%.
The diff coverage is 100.00%.

❗ Current head 5cdf106 differs from pull request most recent head de44cc1. Consider uploading reports for the commit de44cc1 to get more accurate results

@@            Coverage Diff             @@
##             main    #5525      +/-   ##
==========================================
- Coverage   91.13%   90.88%   -0.26%     
==========================================
  Files         230      229       -1     
  Lines       47872    54061    +6189     
==========================================
+ Hits        43627    49131    +5504     
- Misses       4245     4930     +685

Impacted Files	Coverage Δ
tsl/src/compression/create.c	`96.32% <100.00%> (-0.49%)`	⬇️

... and 204 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

github-actions · 2023-04-24T07:36:11Z

@gayyappan, @nikkhils: please review this pull request.

Powered by pull-review

svenklemm · 2023-04-24T11:06:26Z

tsl/test/expected/compression_ddl.out

@@ -644,8 +652,9 @@ AS sub;
                1
 (1 row)

+-- SELECT pg_sleep(200000);


why do we need the sleep here?

we don't 🤦‍♀️ removed.

https://github.com/kgyrtkirk/pg-dev-tools/tree/main/psql-debug

kgyrtkirk · 2023-04-26T11:07:32Z

tsl/src/compression/create.c

+	create->tablespacename =
+		(!OidIsValid(tablespace_oid)) ? NULL : get_tablespace_name(tablespace_oid);


I don't think we need Elvis here - it must be set correctly

In my opinion, it would be good to add an assert in case of OidIsValid(tablespace_oid) and get_tablespace_name returns NULL.

I have added the assertion

tsl/src/compression/create.c

tsl/test/sql/compression_ddl.sql

If a hypertable uses a non default tablespace, the compressed hypertable and its corresponding toast table and index is still created in the default tablespace. This PR fixes this unexpected behavior and creates the compressed hypertable and its toast table and index in the same tablespace as the hypertable. Fixes timescale#5520

timescale-automation · 2023-05-02T12:30:47Z

Automated backport to 2.10.x not done: cherry-pick failed.

Git status

HEAD detached at origin/2.10.x
You are currently cherry-picking commit 6e65172c.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	modified:   CHANGELOG.md
	modified:   tsl/src/compression/create.c
	modified:   tsl/test/sql/compression_ddl.sql

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   tsl/test/expected/compression_ddl.out

Job log

@kovetskiy

This release includes these noteworthy features: * compressed hypertable enhancements: * UPDATE/DELETE support * ON CONFLICT DO UPDATE * Join support for hierarchical Continougs Aggregates * performance improvements **Features** * timescale#5212 Allow pushdown of reference table joins * timescale#5221 Improve Realtime Continuous Aggregate performance * timescale#5252 Improve unique constraint support on compressed hypertables * timescale#5339 Support UPDATE/DELETE on compressed hypertables * timescale#5344 Enable JOINS for Hierarchical Continuous Aggregates * timescale#5361 Add parallel support for partialize_agg() * timescale#5417 Refactor and optimize distributed COPY * timescale#5454 Add support for ON CONFLICT DO UPDATE for compressed hypertables * timescale#5547 Skip Ordered Append when only 1 child node is present * timescale#5510 Propagate vacuum/analyze to compressed chunks * timescale#5584 Reduce decompression during constraint checking * timescale#5530 Optimize compressed chunk resorting **Bugfixes** * timescale#5396 Fix SEGMENTBY columns predicates to be pushed down * timescale#5427 Handle user-defined FDW options properly * timescale#5442 Decompression may have lost DEFAULT values * timescale#5459 Fix issue creating dimensional constraints * timescale#5570 Improve interpolate error message on datatype mismatch * timescale#5573 Fix unique constraint on compressed tables * timescale#5615 Add permission checks to run_job() * timescale#5614 Enable run_job() for telemetry job * timescale#5578 Fix on-insert decompression after schema changes * timescale#5613 Quote username identifier appropriately * timescale#5525 Fix tablespace for compressed hypertable and corresponding toast * timescale#5642 Fix ALTER TABLE SET with normal tables * timescale#5666 Reduce memory usage for distributed analyze * timescale#5668 Fix subtransaction resource owner **Thanks** * @kovetskiy and @DZDomi for reporting peformance regression in Realtime Continuous Aggregates * @ollz272 for reporting an issue with interpolate error messages

@kovetskiy

This release contains new features and bug fixes since the 2.10.3 release. We deem it moderate priority for upgrading. This release includes these noteworthy features: * Support for DML operations on compressed chunks: * UPDATE/DELETE support * Support for unique constraints on compressed chunks * Support for `ON CONFLICT DO UPDATE` * Support for `ON CONFLICT DO NOTHING` * Join support for hierarchical Continuous Aggregates **Features** * timescale#5212 Allow pushdown of reference table joins * timescale#5221 Improve Realtime Continuous Aggregate performance * timescale#5252 Improve unique constraint support on compressed hypertables * timescale#5339 Support UPDATE/DELETE on compressed hypertables * timescale#5344 Enable JOINS for Hierarchical Continuous Aggregates * timescale#5361 Add parallel support for partialize_agg() * timescale#5417 Refactor and optimize distributed COPY * timescale#5454 Add support for ON CONFLICT DO UPDATE for compressed hypertables * timescale#5547 Skip Ordered Append when only 1 child node is present * timescale#5510 Propagate vacuum/analyze to compressed chunks * timescale#5584 Reduce decompression during constraint checking * timescale#5530 Optimize compressed chunk resorting * timescale#5639 Support sending telemetry event reports **Bugfixes** * timescale#5396 Fix SEGMENTBY columns predicates to be pushed down * timescale#5427 Handle user-defined FDW options properly * timescale#5442 Decompression may have lost DEFAULT values * timescale#5459 Fix issue creating dimensional constraints * timescale#5570 Improve interpolate error message on datatype mismatch * timescale#5573 Fix unique constraint on compressed tables * timescale#5615 Add permission checks to run_job() * timescale#5614 Enable run_job() for telemetry job * timescale#5578 Fix on-insert decompression after schema changes * timescale#5613 Quote username identifier appropriately * timescale#5525 Fix tablespace for compressed hypertable and corresponding toast * timescale#5642 Fix ALTER TABLE SET with normal tables * timescale#5666 Reduce memory usage for distributed analyze * timescale#5668 Fix subtransaction resource owner **Thanks** * @kovetskiy and @DZDomi for reporting peformance regression in Realtime Continuous Aggregates * @ollz272 for reporting an issue with interpolate error messages

@kovetskiy

This release contains new features and bug fixes since the 2.10.3 release. We deem it moderate priority for upgrading. This release includes these noteworthy features: * Support for DML operations on compressed chunks: * UPDATE/DELETE support * Support for unique constraints on compressed chunks * Support for `ON CONFLICT DO UPDATE` * Support for `ON CONFLICT DO NOTHING` * Join support for hierarchical Continuous Aggregates **Features** * timescale#5212 Allow pushdown of reference table joins * timescale#5221 Improve Realtime Continuous Aggregate performance * timescale#5252 Improve unique constraint support on compressed hypertables * timescale#5339 Support UPDATE/DELETE on compressed hypertables * timescale#5344 Enable JOINS for Hierarchical Continuous Aggregates * timescale#5361 Add parallel support for partialize_agg() * timescale#5417 Refactor and optimize distributed COPY * timescale#5454 Add support for ON CONFLICT DO UPDATE for compressed hypertables * timescale#5547 Skip Ordered Append when only 1 child node is present * timescale#5510 Propagate vacuum/analyze to compressed chunks * timescale#5584 Reduce decompression during constraint checking * timescale#5530 Optimize compressed chunk resorting * timescale#5639 Support sending telemetry event reports **Bugfixes** * timescale#5396 Fix SEGMENTBY columns predicates to be pushed down * timescale#5427 Handle user-defined FDW options properly * timescale#5442 Decompression may have lost DEFAULT values * timescale#5459 Fix issue creating dimensional constraints * timescale#5570 Improve interpolate error message on datatype mismatch * timescale#5573 Fix unique constraint on compressed tables * timescale#5615 Add permission checks to run_job() * timescale#5614 Enable run_job() for telemetry job * timescale#5578 Fix on-insert decompression after schema changes * timescale#5613 Quote username identifier appropriately * timescale#5525 Fix tablespace for compressed hypertable and corresponding toast * timescale#5642 Fix ALTER TABLE SET with normal tables * timescale#5666 Reduce memory usage for distributed analyze * timescale#5668 Fix subtransaction resource owner **Thanks** * @kovetskiy and @DZDomi for reporting peformance regression in Realtime Continuous Aggregates * @ollz272 for reporting an issue with interpolate error messages

@kovetskiy

This release contains new features and bug fixes since the 2.10.3 release. We deem it moderate priority for upgrading. This release includes these noteworthy features: * Support for DML operations on compressed chunks: * UPDATE/DELETE support * Support for unique constraints on compressed chunks * Support for `ON CONFLICT DO UPDATE` * Support for `ON CONFLICT DO NOTHING` * Join support for hierarchical Continuous Aggregates **Features** * #5212 Allow pushdown of reference table joins * #5221 Improve Realtime Continuous Aggregate performance * #5252 Improve unique constraint support on compressed hypertables * #5339 Support UPDATE/DELETE on compressed hypertables * #5344 Enable JOINS for Hierarchical Continuous Aggregates * #5361 Add parallel support for partialize_agg() * #5417 Refactor and optimize distributed COPY * #5454 Add support for ON CONFLICT DO UPDATE for compressed hypertables * #5547 Skip Ordered Append when only 1 child node is present * #5510 Propagate vacuum/analyze to compressed chunks * #5584 Reduce decompression during constraint checking * #5530 Optimize compressed chunk resorting * #5639 Support sending telemetry event reports **Bugfixes** * #5396 Fix SEGMENTBY columns predicates to be pushed down * #5427 Handle user-defined FDW options properly * #5442 Decompression may have lost DEFAULT values * #5459 Fix issue creating dimensional constraints * #5570 Improve interpolate error message on datatype mismatch * #5573 Fix unique constraint on compressed tables * #5615 Add permission checks to run_job() * #5614 Enable run_job() for telemetry job * #5578 Fix on-insert decompression after schema changes * #5613 Quote username identifier appropriately * #5525 Fix tablespace for compressed hypertable and corresponding toast * #5642 Fix ALTER TABLE SET with normal tables * #5666 Reduce memory usage for distributed analyze * #5668 Fix subtransaction resource owner **Thanks** * @kovetskiy and @DZDomi for reporting peformance regression in Realtime Continuous Aggregates * @ollz272 for reporting an issue with interpolate error messages

@kovetskiy

This release contains new features and bug fixes since the 2.10.3 release. We deem it moderate priority for upgrading. This release includes these noteworthy features: * Support for DML operations on compressed chunks: * UPDATE/DELETE support * Support for unique constraints on compressed chunks * Support for `ON CONFLICT DO UPDATE` * Support for `ON CONFLICT DO NOTHING` * Join support for hierarchical Continuous Aggregates **Features** * timescale#5212 Allow pushdown of reference table joins * timescale#5221 Improve Realtime Continuous Aggregate performance * timescale#5252 Improve unique constraint support on compressed hypertables * timescale#5339 Support UPDATE/DELETE on compressed hypertables * timescale#5344 Enable JOINS for Hierarchical Continuous Aggregates * timescale#5361 Add parallel support for partialize_agg() * timescale#5417 Refactor and optimize distributed COPY * timescale#5454 Add support for ON CONFLICT DO UPDATE for compressed hypertables * timescale#5547 Skip Ordered Append when only 1 child node is present * timescale#5510 Propagate vacuum/analyze to compressed chunks * timescale#5584 Reduce decompression during constraint checking * timescale#5530 Optimize compressed chunk resorting * timescale#5639 Support sending telemetry event reports **Bugfixes** * timescale#5396 Fix SEGMENTBY columns predicates to be pushed down * timescale#5427 Handle user-defined FDW options properly * timescale#5442 Decompression may have lost DEFAULT values * timescale#5459 Fix issue creating dimensional constraints * timescale#5570 Improve interpolate error message on datatype mismatch * timescale#5573 Fix unique constraint on compressed tables * timescale#5615 Add permission checks to run_job() * timescale#5614 Enable run_job() for telemetry job * timescale#5578 Fix on-insert decompression after schema changes * timescale#5613 Quote username identifier appropriately * timescale#5525 Fix tablespace for compressed hypertable and corresponding toast * timescale#5642 Fix ALTER TABLE SET with normal tables * timescale#5666 Reduce memory usage for distributed analyze * timescale#5668 Fix subtransaction resource owner **Thanks** * @kovetskiy and @DZDomi for reporting peformance regression in Realtime Continuous Aggregates * @ollz272 for reporting an issue with interpolate error messages

@kovetskiy

This release contains new features and bug fixes since the 2.10.3 release. We deem it moderate priority for upgrading. This release includes these noteworthy features: * Support for DML operations on compressed chunks: * UPDATE/DELETE support * Support for unique constraints on compressed chunks * Support for `ON CONFLICT DO UPDATE` * Support for `ON CONFLICT DO NOTHING` * Join support for hierarchical Continuous Aggregates **Features** * timescale#5212 Allow pushdown of reference table joins * timescale#5221 Improve Realtime Continuous Aggregate performance * timescale#5252 Improve unique constraint support on compressed hypertables * timescale#5339 Support UPDATE/DELETE on compressed hypertables * timescale#5344 Enable JOINS for Hierarchical Continuous Aggregates * timescale#5361 Add parallel support for partialize_agg() * timescale#5417 Refactor and optimize distributed COPY * timescale#5454 Add support for ON CONFLICT DO UPDATE for compressed hypertables * timescale#5547 Skip Ordered Append when only 1 child node is present * timescale#5510 Propagate vacuum/analyze to compressed chunks * timescale#5584 Reduce decompression during constraint checking * timescale#5530 Optimize compressed chunk resorting * timescale#5639 Support sending telemetry event reports **Bugfixes** * timescale#5396 Fix SEGMENTBY columns predicates to be pushed down * timescale#5427 Handle user-defined FDW options properly * timescale#5442 Decompression may have lost DEFAULT values * timescale#5459 Fix issue creating dimensional constraints * timescale#5570 Improve interpolate error message on datatype mismatch * timescale#5573 Fix unique constraint on compressed tables * timescale#5615 Add permission checks to run_job() * timescale#5614 Enable run_job() for telemetry job * timescale#5578 Fix on-insert decompression after schema changes * timescale#5613 Quote username identifier appropriately * timescale#5525 Fix tablespace for compressed hypertable and corresponding toast * timescale#5642 Fix ALTER TABLE SET with normal tables * timescale#5666 Reduce memory usage for distributed analyze * timescale#5668 Fix subtransaction resource owner **Thanks** * @kovetskiy and @DZDomi for reporting peformance regression in Realtime Continuous Aggregates * @ollz272 for reporting an issue with interpolate error messages

github-actions bot assigned konskov Apr 5, 2023

konskov force-pushed the compression_tablespace_bugfix_5520 branch 2 times, most recently from 32a1d06 to bd09136 Compare April 5, 2023 12:21

konskov force-pushed the compression_tablespace_bugfix_5520 branch 2 times, most recently from f17376e to ddfbfef Compare April 24, 2023 07:14

konskov force-pushed the compression_tablespace_bugfix_5520 branch from ddfbfef to ae4b786 Compare April 24, 2023 07:34

konskov changed the title ~~Add tablespace arg when creating compression hypertable~~ Fix tablespace for compressed hypertable and corresponding toast Apr 24, 2023

konskov marked this pull request as ready for review April 24, 2023 07:35

github-actions bot requested review from gayyappan and nikkhils April 24, 2023 07:36

svenklemm reviewed Apr 24, 2023

View reviewed changes

konskov force-pushed the compression_tablespace_bugfix_5520 branch from ae4b786 to 8f68fca Compare April 24, 2023 11:47

konskov requested a review from svenklemm April 24, 2023 12:11

konskov force-pushed the compression_tablespace_bugfix_5520 branch from 8f68fca to f7d0ded Compare April 26, 2023 10:04

kgyrtkirk approved these changes Apr 26, 2023

View reviewed changes

jnidzwetzki reviewed Apr 26, 2023

View reviewed changes

tsl/test/sql/compression_ddl.sql Outdated Show resolved Hide resolved

konskov force-pushed the compression_tablespace_bugfix_5520 branch 3 times, most recently from 76aa0aa to 5cdf106 Compare May 2, 2023 07:27

konskov requested a review from jnidzwetzki May 2, 2023 07:45

svenklemm approved these changes May 2, 2023

View reviewed changes

konskov force-pushed the compression_tablespace_bugfix_5520 branch from 5cdf106 to de44cc1 Compare May 2, 2023 11:51

konskov enabled auto-merge (rebase) May 2, 2023 12:20

jnidzwetzki approved these changes May 2, 2023

View reviewed changes

konskov merged commit 6e65172 into timescale:main May 2, 2023
58 of 59 checks passed

timescale-automation added the auto-backport-not-done Automated backport of this PR has failed non-retriably (e.g. conflicts) label May 2, 2023

konskov deleted the compression_tablespace_bugfix_5520 branch May 2, 2023 16:10

kgyrtkirk mentioned this pull request May 17, 2023

Release 2.11.0 #5695

Merged

kgyrtkirk mentioned this pull request May 19, 2023

Release 2.11.0 - fix date #5702

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix tablespace for compressed hypertable and corresponding toast #5525

Fix tablespace for compressed hypertable and corresponding toast #5525

konskov commented Apr 5, 2023 •

edited

codecov bot commented Apr 24, 2023 •

edited

github-actions bot commented Apr 24, 2023

svenklemm Apr 24, 2023

konskov Apr 24, 2023

kgyrtkirk Apr 26, 2023

kgyrtkirk Apr 26, 2023

jnidzwetzki Apr 26, 2023

konskov May 2, 2023

timescale-automation commented May 2, 2023

		create->tablespacename =
		(!OidIsValid(tablespace_oid)) ? NULL : get_tablespace_name(tablespace_oid);

Fix tablespace for compressed hypertable and corresponding toast #5525

Fix tablespace for compressed hypertable and corresponding toast #5525

Conversation

konskov commented Apr 5, 2023 • edited

codecov bot commented Apr 24, 2023 • edited

Codecov Report

github-actions bot commented Apr 24, 2023

svenklemm Apr 24, 2023

Choose a reason for hiding this comment

konskov Apr 24, 2023

Choose a reason for hiding this comment

kgyrtkirk Apr 26, 2023

Choose a reason for hiding this comment

kgyrtkirk Apr 26, 2023

Choose a reason for hiding this comment

jnidzwetzki Apr 26, 2023

Choose a reason for hiding this comment

konskov May 2, 2023

Choose a reason for hiding this comment

timescale-automation commented May 2, 2023

Git status

konskov commented Apr 5, 2023 •

edited

codecov bot commented Apr 24, 2023 •

edited