New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
compression policies on distributed hypertables #3116
compression policies on distributed hypertables #3116
Conversation
0dbda27
to
7616285
Compare
@@ -371,6 +372,63 @@ policy_refresh_cagg_read_and_validate_config(Jsonb *config, PolicyContinuousAggD | |||
} | |||
} | |||
|
|||
/* | |||
* Invoke compress_chunk via fmgr so that the call can be deparsed and sent to |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there a way to do this by invoking the parser on a query/SQL statement instead of doing it manually? Might be easier to adapt in the future if we'd want to, e.g., invoke compress chunks on multiple chunks in a single statement. Something like:
SELECT compress_chunk(ch)
FROM show_chunks(hypertable) ch LIMIT 10;
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll do the research and get back to you.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A question here, how would this work with 2PC? Meaning, this query would generate 10 2PC transactions, correct? Would those transactions happen concurrently? Is the concurrency limited by max_prepared_transactions
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've temporarily added a policy_invoke_compress_chunks()
function in addition to the policy_invoke_compress_chunk()
function.
I replaced it at the access node level to see how it compares. If we are talking about doing that at the data node level then for sure it is too much work for this PR and would need to discuss it.
We usually do not refer to private issues from public issues. It might be better to create a public issue for this and refer to it. |
Codecov Report
@@ Coverage Diff @@
## master #3116 +/- ##
==========================================
+ Coverage 90.17% 90.31% +0.14%
==========================================
Files 215 215
Lines 35391 35390 -1
==========================================
+ Hits 31914 31964 +50
+ Misses 3477 3426 -51
Continue to review full report at Codecov.
|
0a9e0ab
to
af6d23d
Compare
b57a38a
to
68c27d0
Compare
406f199
to
7bea1cb
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
7bea1cb
to
b19f946
Compare
After discussing with @svenklemm I'm changing |
491e084
to
7e293e4
Compare
Add support for compression policies on Access Nodes. Extend the compress_chunk() function to maintain compression state per chunk on the Access Node.
7e293e4
to
32f04b0
Compare
This release adds major new features since the 2.2.1 release. We deem it moderate priority for upgrading. This release adds support for inserting data into compressed chunks and improves performance when inserting data into distributed hypertables. It also adds support for triggers and compression policies on distributed hypertables. The bug fixes in this release addresses issues related to the handling of privileges on compressed hypertables, locking, and triggers with transition tables. **Features** * timescale#3116 Add distributed hypertable compression policies * timescale#3162 Use COPY when executing distributed INSERTs * timescale#3199 Add GENERATED column support on distributed hypertables * timescale#3210 Add trigger support on distributed hypertables * timescale#3230 Support for inserts into compressed chunks **Bugfixes** * timescale#3209 Propagate grants to compressed hypertables * timescale#3229 Use correct lock mode when updating chunk * timescale#3241 Fix assertion failure in decompress_chunk_plan_create * timescale#3243 Fix assertion failure in decompress_chunk_plan_create * timescale#3250 Fix constraint triggers on hypertables * timescale#3251 Fix segmentation fault due to incorrect call to chunk_scan_internal * timescale#3252 Fix blocking triggers with transition tables **Thanks** * @yyjdelete for reporting a crash with decompress_chunk and identifying the bug in the code
This release adds major new features since the 2.2.1 release. We deem it moderate priority for upgrading. This release adds support for inserting data into compressed chunks and improves performance when inserting data into distributed hypertables. It also adds support for triggers and compression policies on distributed hypertables. The bug fixes in this release addresses issues related to the handling of privileges on compressed hypertables, locking, and triggers with transition tables. **Features** * timescale#3116 Add distributed hypertable compression policies * timescale#3162 Use COPY when executing distributed INSERTs * timescale#3199 Add GENERATED column support on distributed hypertables * timescale#3210 Add trigger support on distributed hypertables * timescale#3230 Support for inserts into compressed chunks **Bugfixes** * timescale#3209 Propagate grants to compressed hypertables * timescale#3229 Use correct lock mode when updating chunk * timescale#3241 Fix assertion failure in decompress_chunk_plan_create * timescale#3243 Fix assertion failure in decompress_chunk_plan_create * timescale#3250 Fix constraint triggers on hypertables * timescale#3251 Fix segmentation fault due to incorrect call to chunk_scan_internal * timescale#3252 Fix blocking triggers with transition tables **Thanks** * @yyjdelete for reporting a crash with decompress_chunk and identifying the bug in the code
This release adds major new features since the 2.2.1 release. We deem it moderate priority for upgrading. This release adds support for inserting data into compressed chunks and improves performance when inserting data into distributed hypertables. It also adds support for triggers and compression policies on distributed hypertables. The bug fixes in this release addresses issues related to the handling of privileges on compressed hypertables, locking, and triggers with transition tables. **Features** * timescale#3116 Add distributed hypertable compression policies * timescale#3162 Use COPY when executing distributed INSERTs * timescale#3199 Add GENERATED column support on distributed hypertables * timescale#3210 Add trigger support on distributed hypertables * timescale#3230 Support for inserts into compressed chunks **Bugfixes** * timescale#3209 Propagate grants to compressed hypertables * timescale#3229 Use correct lock mode when updating chunk * timescale#3241 Fix assertion failure in decompress_chunk_plan_create * timescale#3243 Fix assertion failure in decompress_chunk_plan_create * timescale#3250 Fix constraint triggers on hypertables * timescale#3251 Fix segmentation fault due to incorrect call to chunk_scan_internal * timescale#3252 Fix blocking triggers with transition tables **Thanks** * @yyjdelete for reporting a crash with decompress_chunk and identifying the bug in the code
This release adds major new features since the 2.2.1 release. We deem it moderate priority for upgrading. This release adds support for inserting data into compressed chunks and improves performance when inserting data into distributed hypertables. It also adds support for triggers and compression policies on distributed hypertables. The bug fixes in this release addresses issues related to the handling of privileges on compressed hypertables, locking, and triggers with transition tables. **Features** * timescale#3116 Add distributed hypertable compression policies * timescale#3162 Use COPY when executing distributed INSERTs * timescale#3199 Add GENERATED column support on distributed hypertables * timescale#3210 Add trigger support on distributed hypertables * timescale#3230 Support for inserts into compressed chunks **Bugfixes** * timescale#3209 Propagate grants to compressed hypertables * timescale#3229 Use correct lock mode when updating chunk * timescale#3241 Fix assertion failure in decompress_chunk_plan_create * timescale#3243 Fix assertion failure in decompress_chunk_plan_create * timescale#3250 Fix constraint triggers on hypertables * timescale#3251 Fix segmentation fault due to incorrect call to chunk_scan_internal * timescale#3252 Fix blocking triggers with transition tables **Thanks** * @yyjdelete for reporting a crash with decompress_chunk and identifying the bug in the code
This release adds major new features since the 2.2.1 release. We deem it moderate priority for upgrading. This release adds support for inserting data into compressed chunks and improves performance when inserting data into distributed hypertables. It also adds support for triggers and compression policies on distributed hypertables. The bug fixes in this release addresses issues related to the handling of privileges on compressed hypertables, locking, and triggers with transition tables. **Features** * timescale#3116 Add distributed hypertable compression policies * timescale#3162 Use COPY when executing distributed INSERTs * timescale#3199 Add GENERATED column support on distributed hypertables * timescale#3210 Add trigger support on distributed hypertables * timescale#3230 Support for inserts into compressed chunks **Bugfixes** * timescale#3213 Propagate grants to compressed hypertables * timescale#3229 Use correct lock mode when updating chunk * timescale#3243 Fix assertion failure in decompress_chunk_plan_create * timescale#3250 Fix constraint triggers on hypertables * timescale#3251 Fix segmentation fault due to incorrect call to chunk_scan_internal * timescale#3252 Fix blocking triggers with transition tables **Thanks** * @yyjdelete for reporting a crash with decompress_chunk and identifying the bug in the code
This release adds major new features since the 2.2.1 release. We deem it moderate priority for upgrading. This release adds support for inserting data into compressed chunks and improves performance when inserting data into distributed hypertables. It also adds support for triggers and compression policies on distributed hypertables. The bug fixes in this release addresses issues related to the handling of privileges on compressed hypertables, locking, and triggers with transition tables. **Features** * timescale#3116 Add distributed hypertable compression policies * timescale#3162 Use COPY when executing distributed INSERTs * timescale#3199 Add GENERATED column support on distributed hypertables * timescale#3210 Add trigger support on distributed hypertables * timescale#3230 Support for inserts into compressed chunks **Bugfixes** * timescale#3213 Propagate grants to compressed hypertables * timescale#3229 Use correct lock mode when updating chunk * timescale#3243 Fix assertion failure in decompress_chunk_plan_create * timescale#3250 Fix constraint triggers on hypertables * timescale#3251 Fix segmentation fault due to incorrect call to chunk_scan_internal * timescale#3252 Fix blocking triggers with transition tables **Thanks** * @yyjdelete for reporting a crash with decompress_chunk and identifying the bug in the code
This release adds major new features since the 2.2.1 release. We deem it moderate priority for upgrading. This release adds support for inserting data into compressed chunks and improves performance when inserting data into distributed hypertables. It also adds support for triggers and compression policies on distributed hypertables. The bug fixes in this release address issues related to the handling of privileges on compressed hypertables, locking, and triggers with transition tables. **Features** * timescale#3116 Add distributed hypertable compression policies * timescale#3162 Use COPY when executing distributed INSERTs * timescale#3199 Add GENERATED column support on distributed hypertables * timescale#3210 Add trigger support on distributed hypertables * timescale#3230 Support for inserts into compressed chunks **Bugfixes** * timescale#3213 Propagate grants to compressed hypertables * timescale#3229 Use correct lock mode when updating chunk * timescale#3243 Fix assertion failure in decompress_chunk_plan_create * timescale#3250 Fix constraint triggers on hypertables * timescale#3251 Fix segmentation fault due to incorrect call to chunk_scan_internal * timescale#3252 Fix blocking triggers with transition tables **Thanks** * @yyjdelete for reporting a crash with decompress_chunk and identifying the bug in the code
This release adds major new features since the 2.2.1 release. We deem it moderate priority for upgrading. This release adds support for inserting data into compressed chunks and improves performance when inserting data into distributed hypertables. Distributed hypertables now also support triggers and compression policies. The bug fixes in this release address issues related to the handling of privileges on compressed hypertables, locking, and triggers with transition tables. **Features** * timescale#3116 Add distributed hypertable compression policies * timescale#3162 Use COPY when executing distributed INSERTs * timescale#3199 Add GENERATED column support on distributed hypertables * timescale#3210 Add trigger support on distributed hypertables * timescale#3230 Support for inserts into compressed chunks **Bugfixes** * timescale#3213 Propagate grants to compressed hypertables * timescale#3229 Use correct lock mode when updating chunk * timescale#3243 Fix assertion failure in decompress_chunk_plan_create * timescale#3250 Fix constraint triggers on hypertables * timescale#3251 Fix segmentation fault due to incorrect call to chunk_scan_internal * timescale#3252 Fix blocking triggers with transition tables **Thanks** * @yyjdelete for reporting a crash with decompress_chunk and identifying the bug in the code
This release adds major new features since the 2.2.1 release. We deem it moderate priority for upgrading. This release adds support for inserting data into compressed chunks and improves performance when inserting data into distributed hypertables. Distributed hypertables now also support triggers and compression policies. The bug fixes in this release address issues related to the handling of privileges on compressed hypertables, locking, and triggers with transition tables. **Features** * timescale#3116 Add distributed hypertable compression policies * timescale#3162 Use COPY when executing distributed INSERTs * timescale#3199 Add GENERATED column support on distributed hypertables * timescale#3210 Add trigger support on distributed hypertables * timescale#3230 Support for inserts into compressed chunks **Bugfixes** * timescale#3213 Propagate grants to compressed hypertables * timescale#3229 Use correct lock mode when updating chunk * timescale#3243 Fix assertion failure in decompress_chunk_plan_create * timescale#3250 Fix constraint triggers on hypertables * timescale#3251 Fix segmentation fault due to incorrect call to chunk_scan_internal * timescale#3252 Fix blocking triggers with transition tables **Thanks** * @yyjdelete for reporting a crash with decompress_chunk and identifying the bug in the code
This release adds major new features since the 2.2.1 release. We deem it moderate priority for upgrading. This release adds support for inserting data into compressed chunks and improves performance when inserting data into distributed hypertables. Distributed hypertables now also support triggers and compression policies. The bug fixes in this release address issues related to the handling of privileges on compressed hypertables, locking, and triggers with transition tables. **Features** * timescale#3116 Add distributed hypertable compression policies * timescale#3162 Use COPY when executing distributed INSERTs * timescale#3199 Add GENERATED column support on distributed hypertables * timescale#3210 Add trigger support on distributed hypertables * timescale#3230 Support for inserts into compressed chunks **Bugfixes** * timescale#3213 Propagate grants to compressed hypertables * timescale#3229 Use correct lock mode when updating chunk * timescale#3243 Fix assertion failure in decompress_chunk_plan_create * timescale#3250 Fix constraint triggers on hypertables * timescale#3251 Fix segmentation fault due to incorrect call to chunk_scan_internal * timescale#3252 Fix blocking triggers with transition tables **Thanks** * @yyjdelete for reporting a crash with decompress_chunk and identifying the bug in the code
This release adds major new features since the 2.2.1 release. We deem it moderate priority for upgrading. This release adds support for inserting data into compressed chunks and improves performance when inserting data into distributed hypertables. Distributed hypertables now also support triggers and compression policies. The bug fixes in this release address issues related to the handling of privileges on compressed hypertables, locking, and triggers with transition tables. **Features** * timescale#3116 Add distributed hypertable compression policies * timescale#3162 Use COPY when executing distributed INSERTs * timescale#3199 Add GENERATED column support on distributed hypertables * timescale#3210 Add trigger support on distributed hypertables * timescale#3230 Support for inserts into compressed chunks **Bugfixes** * timescale#3213 Propagate grants to compressed hypertables * timescale#3229 Use correct lock mode when updating chunk * timescale#3243 Fix assertion failure in decompress_chunk_plan_create * timescale#3250 Fix constraint triggers on hypertables * timescale#3251 Fix segmentation fault due to incorrect call to chunk_scan_internal * timescale#3252 Fix blocking triggers with transition tables **Thanks** * @yyjdelete for reporting a crash with decompress_chunk and identifying the bug in the code
This release adds major new features since the 2.2.1 release. We deem it moderate priority for upgrading. This release adds support for inserting data into compressed chunks and improves performance when inserting data into distributed hypertables. Distributed hypertables now also support triggers and compression policies. The bug fixes in this release address issues related to the handling of privileges on compressed hypertables, locking, and triggers with transition tables. **Features** * timescale#3116 Add distributed hypertable compression policies * timescale#3162 Use COPY when executing distributed INSERTs * timescale#3199 Add GENERATED column support on distributed hypertables * timescale#3210 Add trigger support on distributed hypertables * timescale#3230 Support for inserts into compressed chunks **Bugfixes** * timescale#3213 Propagate grants to compressed hypertables * timescale#3229 Use correct lock mode when updating chunk * timescale#3243 Fix assertion failure in decompress_chunk_plan_create * timescale#3250 Fix constraint triggers on hypertables * timescale#3251 Fix segmentation fault due to incorrect call to chunk_scan_internal * timescale#3252 Fix blocking triggers with transition tables **Thanks** * @yyjdelete for reporting a crash with decompress_chunk and identifying the bug in the code
This release adds major new features since the 2.2.1 release. We deem it moderate priority for upgrading. This release adds support for inserting data into compressed chunks and improves performance when inserting data into distributed hypertables. Distributed hypertables now also support triggers and compression policies. The bug fixes in this release address issues related to the handling of privileges on compressed hypertables, locking, and triggers with transition tables. **Features** * timescale#3116 Add distributed hypertable compression policies * timescale#3162 Use COPY when executing distributed INSERTs * timescale#3199 Add GENERATED column support on distributed hypertables * timescale#3210 Add trigger support on distributed hypertables * timescale#3230 Support for inserts into compressed chunks **Bugfixes** * timescale#3213 Propagate grants to compressed hypertables * timescale#3229 Use correct lock mode when updating chunk * timescale#3243 Fix assertion failure in decompress_chunk_plan_create * timescale#3250 Fix constraint triggers on hypertables * timescale#3251 Fix segmentation fault due to incorrect call to chunk_scan_internal * timescale#3252 Fix blocking triggers with transition tables **Thanks** * @yyjdelete for reporting a crash with decompress_chunk and identifying the bug in the code * @fabriziomello for documenting the prerequisites when compiling against PostgreSQL 13
This release adds major new features since the 2.2.1 release. We deem it moderate priority for upgrading. This release adds support for inserting data into compressed chunks and improves performance when inserting data into distributed hypertables. Distributed hypertables now also support triggers and compression policies. The bug fixes in this release address issues related to the handling of privileges on compressed hypertables, locking, and triggers with transition tables. **Features** * timescale#3116 Add distributed hypertable compression policies * timescale#3162 Use COPY when executing distributed INSERTs * timescale#3199 Add GENERATED column support on distributed hypertables * timescale#3210 Add trigger support on distributed hypertables * timescale#3230 Support for inserts into compressed chunks **Bugfixes** * timescale#3213 Propagate grants to compressed hypertables * timescale#3229 Use correct lock mode when updating chunk * timescale#3243 Fix assertion failure in decompress_chunk_plan_create * timescale#3250 Fix constraint triggers on hypertables * timescale#3251 Fix segmentation fault due to incorrect call to chunk_scan_internal * timescale#3252 Fix blocking triggers with transition tables **Thanks** * @yyjdelete for reporting a crash with decompress_chunk and identifying the bug in the code * @fabriziomello for documenting the prerequisites when compiling against PostgreSQL 13
This release adds major new features since the 2.2.1 release. We deem it moderate priority for upgrading. This release adds support for inserting data into compressed chunks and improves performance when inserting data into distributed hypertables. Distributed hypertables now also support triggers and compression policies. The bug fixes in this release address issues related to the handling of privileges on compressed hypertables, locking, and triggers with transition tables. **Features** * timescale#3116 Add distributed hypertable compression policies * timescale#3162 Use COPY when executing distributed INSERTs * timescale#3199 Add GENERATED column support on distributed hypertables * timescale#3210 Add trigger support on distributed hypertables * timescale#3230 Support for inserts into compressed chunks **Bugfixes** * timescale#3213 Propagate grants to compressed hypertables * timescale#3229 Use correct lock mode when updating chunk * timescale#3243 Fix assertion failure in decompress_chunk_plan_create * timescale#3250 Fix constraint triggers on hypertables * timescale#3251 Fix segmentation fault due to incorrect call to chunk_scan_internal * timescale#3252 Fix blocking triggers with transition tables **Thanks** * @yyjdelete for reporting a crash with decompress_chunk and identifying the bug in the code * @fabriziomello for documenting the prerequisites when compiling against PostgreSQL 13
This release adds major new features since the 2.2.1 release. We deem it moderate priority for upgrading. This release adds support for inserting data into compressed chunks and improves performance when inserting data into distributed hypertables. Distributed hypertables now also support triggers and compression policies. The bug fixes in this release address issues related to the handling of privileges on compressed hypertables, locking, and triggers with transition tables. **Features** * #3116 Add distributed hypertable compression policies * #3162 Use COPY when executing distributed INSERTs * #3199 Add GENERATED column support on distributed hypertables * #3210 Add trigger support on distributed hypertables * #3230 Support for inserts into compressed chunks **Bugfixes** * #3213 Propagate grants to compressed hypertables * #3229 Use correct lock mode when updating chunk * #3243 Fix assertion failure in decompress_chunk_plan_create * #3250 Fix constraint triggers on hypertables * #3251 Fix segmentation fault due to incorrect call to chunk_scan_internal * #3252 Fix blocking triggers with transition tables **Thanks** * @yyjdelete for reporting a crash with decompress_chunk and identifying the bug in the code * @fabriziomello for documenting the prerequisites when compiling against PostgreSQL 13
This release adds major new features since the 2.2.1 release. We deem it moderate priority for upgrading. This release adds support for inserting data into compressed chunks and improves performance when inserting data into distributed hypertables. Distributed hypertables now also support triggers and compression policies. The bug fixes in this release address issues related to the handling of privileges on compressed hypertables, locking, and triggers with transition tables. **Features** * #3116 Add distributed hypertable compression policies * #3162 Use COPY when executing distributed INSERTs * #3199 Add GENERATED column support on distributed hypertables * #3210 Add trigger support on distributed hypertables * #3230 Support for inserts into compressed chunks **Bugfixes** * #3213 Propagate grants to compressed hypertables * #3229 Use correct lock mode when updating chunk * #3243 Fix assertion failure in decompress_chunk_plan_create * #3250 Fix constraint triggers on hypertables * #3251 Fix segmentation fault due to incorrect call to chunk_scan_internal * #3252 Fix blocking triggers with transition tables **Thanks** * @yyjdelete for reporting a crash with decompress_chunk and identifying the bug in the code * @fabriziomello for documenting the prerequisites when compiling against PostgreSQL 13
This release adds major new features since the 2.2.1 release. We deem it moderate priority for upgrading. This release adds support for inserting data into compressed chunks and improves performance when inserting data into distributed hypertables. Distributed hypertables now also support triggers and compression policies. The bug fixes in this release address issues related to the handling of privileges on compressed hypertables, locking, and triggers with transition tables. **Features** * #3116 Add distributed hypertable compression policies * #3162 Use COPY when executing distributed INSERTs * #3199 Add GENERATED column support on distributed hypertables * #3210 Add trigger support on distributed hypertables * #3230 Support for inserts into compressed chunks **Bugfixes** * #3213 Propagate grants to compressed hypertables * #3229 Use correct lock mode when updating chunk * #3243 Fix assertion failure in decompress_chunk_plan_create * #3250 Fix constraint triggers on hypertables * #3251 Fix segmentation fault due to incorrect call to chunk_scan_internal * #3252 Fix blocking triggers with transition tables **Thanks** * @yyjdelete for reporting a crash with decompress_chunk and identifying the bug in the code * @fabriziomello for documenting the prerequisites when compiling against PostgreSQL 13
Add distributed hypertable compression policies.