Remove metadata when dropping chunk #6621

fabriziomello · 2024-02-08T18:12:25Z

Historically we preserve chunk metadata because the old format of the
Continuous Aggregate has the chunk_id column in the materialization
hypertable so in order to don't have chunk ids left over there we just
mark it as dropped when dropping chunks.

In #4269 we introduced a new Continuous Aggregate format that don't
store the chunk_id in the materialization hypertable anymore so it's
safe to also remove the metadata when dropping chunk and all associated
Continuous Aggregates are in the new format.

Also added a post-update SQL script to cleanup unnecessary dropped chunk
metadata in our catalog.

Closes #6570

codecov · 2024-02-08T18:21:32Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (59f50f2) 80.06% compared to head (95eb512) 81.46%.
Report is 6 commits behind head on main.

❗ Current head 95eb512 differs from pull request most recent head 30920b4. Consider uploading reports for the commit 30920b4 to get more accurate results

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #6621      +/-   ##
==========================================
+ Coverage   80.06%   81.46%   +1.39%     
==========================================
  Files         190      190              
  Lines       37181    36319     -862     
  Branches     9450     9436      -14     
==========================================
- Hits        29770    29587     -183     
+ Misses       2997     2985      -12     
+ Partials     4414     3747     -667

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

github-actions · 2024-02-09T14:33:34Z

@antekresic, @akuzm: please review this pull request.

Powered by pull-review

test/sql/updates/post.catalog.sql

svenklemm

I dont think you should add this to post-update but use latest-dev instead to fix because otherwise it will constrain changes that can be done in the update script in the future

fabriziomello · 2024-02-09T21:38:10Z

I dont think you should add this to post-update but use latest-dev instead to fix because otherwise it will constrain changes that can be done in the update script in the future

Good that you mentioned it. I had the same feeling but ended up placing this here in order to make sure the future releases we'll don't have chunk metadata leftovers, but I can move it to the latest-dev and in the future when we complete remove the dropped column then write again this cleanup script to make sure we'll not end up with metadata leftovers.

fabriziomello · 2024-02-09T21:46:46Z

@svenklemm moved the post-update.sql cleanup to latest-dev.sql

jnidzwetzki · 2024-02-14T21:37:16Z

@fabriziomello Nit: there is a typo in the commit message unecessary -> unnecessary

jnidzwetzki · 2024-02-14T21:39:49Z

sql/updates/latest-dev.sql

@@ -17,3 +17,52 @@ DROP FUNCTION IF EXISTS _timescaledb_functions.invalidation_process_hypertable_l
 DROP FUNCTION IF EXISTS _timescaledb_functions.invalidation_process_hypertable_log(integer,integer,regtype,integer[],bigint[],bigint[],text[]);
 DROP FUNCTION IF EXISTS _timescaledb_functions.hypertable_invalidation_log_delete(integer);

+-- Cleanup metadata for deleted chunks
+CREATE UNLOGGED TABLE _timescaledb_catalog._chunks_remove AS


Nit: Consider prefixing the table with _tmp

Turned it into a PL/pgSQL function!

jnidzwetzki · 2024-02-15T08:12:34Z

@fabriziomello As discussed, I also did some experiments with compressed hypertables and CAggs. The clean-up works correctly as well.

I am wondering if this cleanup SQL should also be added to the cagg_migration steps and run every time you convert a CAgg into the finalized form.

fabriziomello · 2024-02-15T13:00:15Z

@fabriziomello As discussed, I also did some experiments with compressed hypertables and CAggs. The clean-up works correctly as well.

I am wondering if this cleanup SQL should also be added to the cagg_migration steps and run every time you convert a CAgg into the finalized form.

Very good catch @jnidzwetzki thanks. Will add the cleanup to the migration scripts.

fabriziomello · 2024-02-15T19:20:49Z

@fabriziomello As discussed, I also did some experiments with compressed hypertables and CAggs. The clean-up works correctly as well.
I am wondering if this cleanup SQL should also be added to the cagg_migration steps and run every time you convert a CAgg into the finalized form.

Very good catch @jnidzwetzki thanks. Will add the cleanup to the migration scripts.

Added it also to the cagg_migrate procedure and also added proper regression tests

Historically we preserve chunk metadata because the old format of the Continuous Aggregate has the `chunk_id` column in the materialization hypertable so in order to don't have chunk ids left over there we just mark it as dropped when dropping chunks. In timescale#4269 we introduced a new Continuous Aggregate format that don't store the `chunk_id` in the materialization hypertable anymore so it's safe to also remove the metadata when dropping chunk and all associated Continuous Aggregates are in the new format. Also added a post-update SQL script to cleanup unnecessary dropped chunk metadata in our catalog. Closes timescale#6570

Addressed Sven's concerns

@brasic

This release contains performance improvements and bug fixes since the 2.14.2 release. We recommend that you upgrade at the next available opportunity. In addition, it includes these noteworthy features: * Support `time_bucket` with `origin` and/or `offset` on Continuous Aggregate * Compression improvements: - Improve expression pushdown - Add minmax sparse indexes when compressing columns with btree indexes - Make compression use the defaults functions - Vectorize filters in WHERE clause that contain text equality operators and LIKE expressions **Deprecation warning** * Starting on this release will not be possible to create Continuous Aggregate using `time_bucket_ng` anymore and it will be completely removed on the upcoming releases. * Recommend users to [migrate their old Continuous Aggregate format to the new one](https://docs.timescale.com/use-timescale/latest/continuous-aggregates/migrate/) because it support will be completely removed in next releases prevent them to migrate. * This is the last release supporting PostgreSQL 13. **For on-premise users and this release only**, you will need to run [this SQL script](https://github.com/timescale/timescaledb-extras/blob/master/utils/2.15.X-fix_hypertable_foreign_keys.sql) after running `ALTER EXTENSION`. More details can be found in the pull request [timescale#6786](timescale#6797). **Features** * timescale#6382 Support for time_bucket with origin and offset in CAggs * timescale#6696 Improve defaults for compression segment_by and order_by * timescale#6705 Add sparse minmax indexes for compressed columns that have uncompressed btree indexes * timescale#6754 Allow DROP CONSTRAINT on compressed hypertables * timescale#6767 Add metadata table `_timestaledb_internal.bgw_job_stat_history` for tracking job execution history * timescale#6798 Prevent usage of deprecated time_bucket_ng in CAgg definition * timescale#6810 Add telemetry for access methods * timescale#6811 Remove no longer relevant timescaledb.allow_install_without_preload GUC * timescale#6837 Add migration path for CAggs using time_bucket_ng * timescale#6865 Update the watermark when truncating a CAgg **Bugfixes** * timescale#6617 Fix error in show_chunks * timescale#6621 Remove metadata when dropping chunks * timescale#6677 Fix snapshot usage in CAgg invalidation scanner * timescale#6698 Define meaning of 0 retries for jobs as no retries * timescale#6717 Fix handling of compressed tables with primary or unique index in COPY path * timescale#6726 Fix constify cagg_watermark using window function when querying a CAgg * timescale#6729 Fix NULL start value handling in CAgg refresh * timescale#6732 Fix CAgg migration with custom timezone / date format settings * timescale#6752 Remove custom autovacuum setting from compressed chunks * timescale#6770 Fix plantime chunk exclusion for OSM chunk * timescale#6789 Fix deletes with subqueries and compression * timescale#6796 Fix a crash involving a view on a hypertable * timescale#6797 Fix foreign key constraint handling on compressed hypertables * timescale#6816 Fix handling of chunks with no contraints * timescale#6820 Fix a crash when the ts_hypertable_insert_blocker was called directly * timescale#6849 Use non-orderby compressed metadata in compressed DML * timescale#6867 Clean up compression settings when deleting compressed cagg * timescale#6869 Fix compressed DML with constraints of form value OP column * timescale#6870 Fix bool expression pushdown for queries on compressed chunks **Thanks** * @brasic for reporting a crash when the ts_hypertable_insert_blocker was called directly * @bvanelli for reporting an issue with the jobs retry count * @djzurawsk For reporting error when dropping chunks * @Dzuzepppe for reporting an issue with DELETEs using subquery on compressed chunk working incorrectly. * @hongquan For reporting a 'timestamp out of range' error during CAgg migrations * @kevcenteno for reporting an issue with the show_chunks API showing incorrect output when 'created_before/created_after' was used with time-partitioned columns. * @mahipv For starting working on the job history PR * @rovo89 For reporting constify cagg_watermark not working using window function when querying a CAgg

@brasic

This release contains performance improvements and bug fixes since the 2.14.2 release. We recommend that you upgrade at the next available opportunity. In addition, it includes these noteworthy features: * Support `time_bucket` with `origin` and/or `offset` on Continuous Aggregate * Compression improvements: - Improve expression pushdown - Add minmax sparse indexes when compressing columns with btree indexes - Make compression use the defaults functions - Vectorize filters in WHERE clause that contain text equality operators and LIKE expressions **Deprecation warning** * Starting on this release will not be possible to create Continuous Aggregate using `time_bucket_ng` anymore and it will be completely removed on the upcoming releases. * Recommend users to [migrate their old Continuous Aggregate format to the new one](https://docs.timescale.com/use-timescale/latest/continuous-aggregates/migrate/) because it support will be completely removed in next releases prevent them to migrate. * This is the last release supporting PostgreSQL 13. **For on-premise users and this release only**, you will need to run [this SQL script](https://github.com/timescale/timescaledb-extras/blob/master/utils/2.15.X-fix_hypertable_foreign_keys.sql) after running `ALTER EXTENSION`. More details can be found in the pull request [timescale#6786](timescale#6797). **Features** * timescale#6382 Support for time_bucket with origin and offset in CAggs * timescale#6696 Improve defaults for compression segment_by and order_by * timescale#6705 Add sparse minmax indexes for compressed columns that have uncompressed btree indexes * timescale#6754 Allow DROP CONSTRAINT on compressed hypertables * timescale#6767 Add metadata table `_timestaledb_internal.bgw_job_stat_history` for tracking job execution history * timescale#6798 Prevent usage of deprecated time_bucket_ng in CAgg definition * timescale#6810 Add telemetry for access methods * timescale#6811 Remove no longer relevant timescaledb.allow_install_without_preload GUC * timescale#6837 Add migration path for CAggs using time_bucket_ng * timescale#6865 Update the watermark when truncating a CAgg **Bugfixes** * timescale#6617 Fix error in show_chunks * timescale#6621 Remove metadata when dropping chunks * timescale#6677 Fix snapshot usage in CAgg invalidation scanner * timescale#6698 Define meaning of 0 retries for jobs as no retries * timescale#6717 Fix handling of compressed tables with primary or unique index in COPY path * timescale#6726 Fix constify cagg_watermark using window function when querying a CAgg * timescale#6729 Fix NULL start value handling in CAgg refresh * timescale#6732 Fix CAgg migration with custom timezone / date format settings * timescale#6752 Remove custom autovacuum setting from compressed chunks * timescale#6770 Fix plantime chunk exclusion for OSM chunk * timescale#6789 Fix deletes with subqueries and compression * timescale#6796 Fix a crash involving a view on a hypertable * timescale#6797 Fix foreign key constraint handling on compressed hypertables * timescale#6816 Fix handling of chunks with no contraints * timescale#6820 Fix a crash when the ts_hypertable_insert_blocker was called directly * timescale#6849 Use non-orderby compressed metadata in compressed DML * timescale#6867 Clean up compression settings when deleting compressed cagg * timescale#6869 Fix compressed DML with constraints of form value OP column * timescale#6870 Fix bool expression pushdown for queries on compressed chunks **Thanks** * @brasic for reporting a crash when the ts_hypertable_insert_blocker was called directly * @bvanelli for reporting an issue with the jobs retry count * @djzurawsk For reporting error when dropping chunks * @Dzuzepppe for reporting an issue with DELETEs using subquery on compressed chunk working incorrectly. * @hongquan For reporting a 'timestamp out of range' error during CAgg migrations * @kevcenteno for reporting an issue with the show_chunks API showing incorrect output when 'created_before/created_after' was used with time-partitioned columns. * @mahipv For starting working on the job history PR * @rovo89 For reporting constify cagg_watermark not working using window function when querying a CAgg

@brasic

This release contains performance improvements and bug fixes since the 2.14.2 release. We recommend that you upgrade at the next available opportunity. In addition, it includes these noteworthy features: * Support `time_bucket` with `origin` and/or `offset` on Continuous Aggregate * Compression improvements: - Improve expression pushdown - Add minmax sparse indexes when compressing columns with btree indexes - Make compression use the defaults functions - Vectorize filters in WHERE clause that contain text equality operators and LIKE expressions **Deprecation warning** * Starting on this release will not be possible to create Continuous Aggregate using `time_bucket_ng` anymore and it will be completely removed on the upcoming releases. * Recommend users to [migrate their old Continuous Aggregate format to the new one](https://docs.timescale.com/use-timescale/latest/continuous-aggregates/migrate/) because it support will be completely removed in next releases prevent them to migrate. * This is the last release supporting PostgreSQL 13. **For on-premise users and this release only**, you will need to run [this SQL script](https://github.com/timescale/timescaledb-extras/blob/master/utils/2.15.X-fix_hypertable_foreign_keys.sql) after running `ALTER EXTENSION`. More details can be found in the pull request [timescale#6786](timescale#6797). **Features** * timescale#6382 Support for time_bucket with origin and offset in CAggs * timescale#6696 Improve defaults for compression segment_by and order_by * timescale#6705 Add sparse minmax indexes for compressed columns that have uncompressed btree indexes * timescale#6754 Allow DROP CONSTRAINT on compressed hypertables * timescale#6767 Add metadata table `_timestaledb_internal.bgw_job_stat_history` for tracking job execution history * timescale#6798 Prevent usage of deprecated time_bucket_ng in CAgg definition * timescale#6810 Add telemetry for access methods * timescale#6811 Remove no longer relevant timescaledb.allow_install_without_preload GUC * timescale#6837 Add migration path for CAggs using time_bucket_ng * timescale#6865 Update the watermark when truncating a CAgg **Bugfixes** * timescale#6617 Fix error in show_chunks * timescale#6621 Remove metadata when dropping chunks * timescale#6677 Fix snapshot usage in CAgg invalidation scanner * timescale#6698 Define meaning of 0 retries for jobs as no retries * timescale#6717 Fix handling of compressed tables with primary or unique index in COPY path * timescale#6726 Fix constify cagg_watermark using window function when querying a CAgg * timescale#6729 Fix NULL start value handling in CAgg refresh * timescale#6732 Fix CAgg migration with custom timezone / date format settings * timescale#6752 Remove custom autovacuum setting from compressed chunks * timescale#6770 Fix plantime chunk exclusion for OSM chunk * timescale#6789 Fix deletes with subqueries and compression * timescale#6796 Fix a crash involving a view on a hypertable * timescale#6797 Fix foreign key constraint handling on compressed hypertables * timescale#6816 Fix handling of chunks with no contraints * timescale#6820 Fix a crash when the ts_hypertable_insert_blocker was called directly * timescale#6849 Use non-orderby compressed metadata in compressed DML * timescale#6867 Clean up compression settings when deleting compressed cagg * timescale#6869 Fix compressed DML with constraints of form value OP column * timescale#6870 Fix bool expression pushdown for queries on compressed chunks **Thanks** * @brasic for reporting a crash when the ts_hypertable_insert_blocker was called directly * @bvanelli for reporting an issue with the jobs retry count * @djzurawsk For reporting error when dropping chunks * @Dzuzepppe for reporting an issue with DELETEs using subquery on compressed chunk working incorrectly. * @hongquan For reporting a 'timestamp out of range' error during CAgg migrations * @kevcenteno for reporting an issue with the show_chunks API showing incorrect output when 'created_before/created_after' was used with time-partitioned columns. * @mahipv For starting working on the job history PR * @rovo89 For reporting constify cagg_watermark not working using window function when querying a CAgg

@brasic

This release contains performance improvements and bug fixes since the 2.14.2 release. We recommend that you upgrade at the next available opportunity. In addition, it includes these noteworthy features: * Support `time_bucket` with `origin` and/or `offset` on Continuous Aggregate * Compression improvements: - Improve expression pushdown - Add minmax sparse indexes when compressing columns with btree indexes - Make compression use the defaults functions - Vectorize filters in WHERE clause that contain text equality operators and LIKE expressions **Deprecation warning** * Starting on this release will not be possible to create Continuous Aggregate using `time_bucket_ng` anymore and it will be completely removed on the upcoming releases. * Recommend users to [migrate their old Continuous Aggregate format to the new one](https://docs.timescale.com/use-timescale/latest/continuous-aggregates/migrate/) because it support will be completely removed in next releases prevent them to migrate. * This is the last release supporting PostgreSQL 13. **For on-premise users and this release only**, you will need to run [this SQL script](https://github.com/timescale/timescaledb-extras/blob/master/utils/2.15.X-fix_hypertable_foreign_keys.sql) after running `ALTER EXTENSION`. More details can be found in the pull request [#6786](#6797). **Features** * #6382 Support for time_bucket with origin and offset in CAggs * #6696 Improve defaults for compression segment_by and order_by * #6705 Add sparse minmax indexes for compressed columns that have uncompressed btree indexes * #6754 Allow DROP CONSTRAINT on compressed hypertables * #6767 Add metadata table `_timestaledb_internal.bgw_job_stat_history` for tracking job execution history * #6798 Prevent usage of deprecated time_bucket_ng in CAgg definition * #6810 Add telemetry for access methods * #6811 Remove no longer relevant timescaledb.allow_install_without_preload GUC * #6837 Add migration path for CAggs using time_bucket_ng * #6865 Update the watermark when truncating a CAgg **Bugfixes** * #6617 Fix error in show_chunks * #6621 Remove metadata when dropping chunks * #6677 Fix snapshot usage in CAgg invalidation scanner * #6698 Define meaning of 0 retries for jobs as no retries * #6717 Fix handling of compressed tables with primary or unique index in COPY path * #6726 Fix constify cagg_watermark using window function when querying a CAgg * #6729 Fix NULL start value handling in CAgg refresh * #6732 Fix CAgg migration with custom timezone / date format settings * #6752 Remove custom autovacuum setting from compressed chunks * #6770 Fix plantime chunk exclusion for OSM chunk * #6789 Fix deletes with subqueries and compression * #6796 Fix a crash involving a view on a hypertable * #6797 Fix foreign key constraint handling on compressed hypertables * #6816 Fix handling of chunks with no contraints * #6820 Fix a crash when the ts_hypertable_insert_blocker was called directly * #6849 Use non-orderby compressed metadata in compressed DML * #6867 Clean up compression settings when deleting compressed cagg * #6869 Fix compressed DML with constraints of form value OP column * #6870 Fix bool expression pushdown for queries on compressed chunks **Thanks** * @brasic for reporting a crash when the ts_hypertable_insert_blocker was called directly * @bvanelli for reporting an issue with the jobs retry count * @djzurawsk For reporting error when dropping chunks * @Dzuzepppe for reporting an issue with DELETEs using subquery on compressed chunk working incorrectly. * @hongquan For reporting a 'timestamp out of range' error during CAgg migrations * @kevcenteno for reporting an issue with the show_chunks API showing incorrect output when 'created_before/created_after' was used with time-partitioned columns. * @mahipv For starting working on the job history PR * @rovo89 For reporting constify cagg_watermark not working using window function when querying a CAgg

@brasic

This release contains performance improvements and bug fixes since the 2.14.2 release. We recommend that you upgrade at the next available opportunity. In addition, it includes these noteworthy features: * Support `time_bucket` with `origin` and/or `offset` on Continuous Aggregate * Compression improvements: - Improve expression pushdown - Add minmax sparse indexes when compressing columns with btree indexes - Make compression use the defaults functions - Vectorize filters in WHERE clause that contain text equality operators and LIKE expressions **Deprecation warning** * Starting on this release will not be possible to create Continuous Aggregate using `time_bucket_ng` anymore and it will be completely removed on the upcoming releases. * Recommend users to [migrate their old Continuous Aggregate format to the new one](https://docs.timescale.com/use-timescale/latest/continuous-aggregates/migrate/) because it support will be completely removed in next releases prevent them to migrate. * This is the last release supporting PostgreSQL 13. **For on-premise users and this release only**, you will need to run [this SQL script](https://github.com/timescale/timescaledb-extras/blob/master/utils/2.15.X-fix_hypertable_foreign_keys.sql) after running `ALTER EXTENSION`. More details can be found in the pull request [timescale#6786](timescale#6797). **Features** * timescale#6382 Support for time_bucket with origin and offset in CAggs * timescale#6696 Improve defaults for compression segment_by and order_by * timescale#6705 Add sparse minmax indexes for compressed columns that have uncompressed btree indexes * timescale#6754 Allow DROP CONSTRAINT on compressed hypertables * timescale#6767 Add metadata table `_timestaledb_internal.bgw_job_stat_history` for tracking job execution history * timescale#6798 Prevent usage of deprecated time_bucket_ng in CAgg definition * timescale#6810 Add telemetry for access methods * timescale#6811 Remove no longer relevant timescaledb.allow_install_without_preload GUC * timescale#6837 Add migration path for CAggs using time_bucket_ng * timescale#6865 Update the watermark when truncating a CAgg **Bugfixes** * timescale#6617 Fix error in show_chunks * timescale#6621 Remove metadata when dropping chunks * timescale#6677 Fix snapshot usage in CAgg invalidation scanner * timescale#6698 Define meaning of 0 retries for jobs as no retries * timescale#6717 Fix handling of compressed tables with primary or unique index in COPY path * timescale#6726 Fix constify cagg_watermark using window function when querying a CAgg * timescale#6729 Fix NULL start value handling in CAgg refresh * timescale#6732 Fix CAgg migration with custom timezone / date format settings * timescale#6752 Remove custom autovacuum setting from compressed chunks * timescale#6770 Fix plantime chunk exclusion for OSM chunk * timescale#6789 Fix deletes with subqueries and compression * timescale#6796 Fix a crash involving a view on a hypertable * timescale#6797 Fix foreign key constraint handling on compressed hypertables * timescale#6816 Fix handling of chunks with no contraints * timescale#6820 Fix a crash when the ts_hypertable_insert_blocker was called directly * timescale#6849 Use non-orderby compressed metadata in compressed DML * timescale#6867 Clean up compression settings when deleting compressed cagg * timescale#6869 Fix compressed DML with constraints of form value OP column * timescale#6870 Fix bool expression pushdown for queries on compressed chunks **Thanks** * @brasic for reporting a crash when the ts_hypertable_insert_blocker was called directly * @bvanelli for reporting an issue with the jobs retry count * @djzurawsk For reporting error when dropping chunks * @Dzuzepppe for reporting an issue with DELETEs using subquery on compressed chunk working incorrectly. * @hongquan For reporting a 'timestamp out of range' error during CAgg migrations * @kevcenteno for reporting an issue with the show_chunks API showing incorrect output when 'created_before/created_after' was used with time-partitioned columns. * @mahipv For starting working on the job history PR * @rovo89 For reporting constify cagg_watermark not working using window function when querying a CAgg

@brasic

This release contains performance improvements and bug fixes since the 2.14.2 release. We recommend that you upgrade at the next available opportunity. In addition, it includes these noteworthy features: * Support `time_bucket` with `origin` and/or `offset` on Continuous Aggregate * Compression improvements: - Improve expression pushdown - Add minmax sparse indexes when compressing columns with btree indexes - Make compression use the defaults functions - Vectorize filters in WHERE clause that contain text equality operators and LIKE expressions **Deprecation warning** * Starting on this release will not be possible to create Continuous Aggregate using `time_bucket_ng` anymore and it will be completely removed on the upcoming releases. * Recommend users to [migrate their old Continuous Aggregate format to the new one](https://docs.timescale.com/use-timescale/latest/continuous-aggregates/migrate/) because it support will be completely removed in next releases prevent them to migrate. * This is the last release supporting PostgreSQL 13. **For on-premise users and this release only**, you will need to run [this SQL script](https://github.com/timescale/timescaledb-extras/blob/master/utils/2.15.X-fix_hypertable_foreign_keys.sql) after running `ALTER EXTENSION`. More details can be found in the pull request [#6786](#6797). **Features** * #6382 Support for time_bucket with origin and offset in CAggs * #6696 Improve defaults for compression segment_by and order_by * #6705 Add sparse minmax indexes for compressed columns that have uncompressed btree indexes * #6754 Allow DROP CONSTRAINT on compressed hypertables * #6767 Add metadata table `_timestaledb_internal.bgw_job_stat_history` for tracking job execution history * #6798 Prevent usage of deprecated time_bucket_ng in CAgg definition * #6810 Add telemetry for access methods * #6811 Remove no longer relevant timescaledb.allow_install_without_preload GUC * #6837 Add migration path for CAggs using time_bucket_ng * #6865 Update the watermark when truncating a CAgg **Bugfixes** * #6617 Fix error in show_chunks * #6621 Remove metadata when dropping chunks * #6677 Fix snapshot usage in CAgg invalidation scanner * #6698 Define meaning of 0 retries for jobs as no retries * #6717 Fix handling of compressed tables with primary or unique index in COPY path * #6726 Fix constify cagg_watermark using window function when querying a CAgg * #6729 Fix NULL start value handling in CAgg refresh * #6732 Fix CAgg migration with custom timezone / date format settings * #6752 Remove custom autovacuum setting from compressed chunks * #6770 Fix plantime chunk exclusion for OSM chunk * #6789 Fix deletes with subqueries and compression * #6796 Fix a crash involving a view on a hypertable * #6797 Fix foreign key constraint handling on compressed hypertables * #6816 Fix handling of chunks with no contraints * #6820 Fix a crash when the ts_hypertable_insert_blocker was called directly * #6849 Use non-orderby compressed metadata in compressed DML * #6867 Clean up compression settings when deleting compressed cagg * #6869 Fix compressed DML with constraints of form value OP column * #6870 Fix bool expression pushdown for queries on compressed chunks **Thanks** * @brasic for reporting a crash when the ts_hypertable_insert_blocker was called directly * @bvanelli for reporting an issue with the jobs retry count * @djzurawsk For reporting error when dropping chunks * @Dzuzepppe for reporting an issue with DELETEs using subquery on compressed chunk working incorrectly. * @hongquan For reporting a 'timestamp out of range' error during CAgg migrations * @kevcenteno for reporting an issue with the show_chunks API showing incorrect output when 'created_before/created_after' was used with time-partitioned columns. * @mahipv For starting working on the job history PR * @rovo89 For reporting constify cagg_watermark not working using window function when querying a CAgg

fabriziomello self-assigned this Feb 8, 2024

fabriziomello mentioned this pull request Feb 8, 2024

[Bug]: 'ERROR: tuple already updated by self' after dropping chunks then inserting data #6570

Closed

fabriziomello force-pushed the remove_chunk_metadata_when_dropping_chunk branch 6 times, most recently from 5d47958 to 1b59fe7 Compare February 8, 2024 20:27

fabriziomello force-pushed the remove_chunk_metadata_when_dropping_chunk branch 2 times, most recently from af1c828 to cfaadbe Compare February 9, 2024 13:44

fabriziomello marked this pull request as ready for review February 9, 2024 14:33

github-actions bot requested review from akuzm and antekresic February 9, 2024 14:33

akuzm reviewed Feb 9, 2024

View reviewed changes

test/sql/updates/post.catalog.sql Outdated Show resolved Hide resolved

akuzm approved these changes Feb 9, 2024

View reviewed changes

svenklemm previously requested changes Feb 9, 2024

View reviewed changes

fabriziomello force-pushed the remove_chunk_metadata_when_dropping_chunk branch 2 times, most recently from 44d1389 to 85c2e12 Compare February 9, 2024 21:46

fabriziomello mentioned this pull request Feb 12, 2024

Cleanup invalid chunk metadata #6599

Closed

fabriziomello requested a review from svenklemm February 12, 2024 17:38

fabriziomello force-pushed the remove_chunk_metadata_when_dropping_chunk branch 5 times, most recently from ca17b62 to c11d0a3 Compare February 14, 2024 21:24

jnidzwetzki reviewed Feb 14, 2024

View reviewed changes

fabriziomello force-pushed the remove_chunk_metadata_when_dropping_chunk branch 3 times, most recently from 5d9ce57 to 28ade46 Compare February 15, 2024 19:09

fabriziomello force-pushed the remove_chunk_metadata_when_dropping_chunk branch from 28ade46 to 95eb512 Compare February 15, 2024 21:14

jnidzwetzki approved these changes Feb 16, 2024

View reviewed changes

fabriziomello force-pushed the remove_chunk_metadata_when_dropping_chunk branch from 95eb512 to 30920b4 Compare February 16, 2024 12:20

fabriziomello merged commit 5a359ac into timescale:main Feb 16, 2024
41 checks passed

timescale-automation mentioned this pull request Feb 28, 2024

Backport to 2.14.x: #6621: Remove metadata when dropping chunk #6711

Closed

fabriziomello mentioned this pull request Apr 30, 2024

Release notes header 2.15.0 #6874

Merged

fabriziomello mentioned this pull request May 6, 2024

Release 2.15.0 #6887

Merged

fabriziomello mentioned this pull request May 6, 2024

Release 2.15.0: Release branch #6889

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Remove metadata when dropping chunk #6621

Remove metadata when dropping chunk #6621

fabriziomello commented Feb 8, 2024 •

edited

Loading

codecov bot commented Feb 8, 2024 •

edited

Loading

github-actions bot commented Feb 9, 2024

svenklemm left a comment

fabriziomello commented Feb 9, 2024

fabriziomello commented Feb 9, 2024

jnidzwetzki commented Feb 14, 2024

jnidzwetzki Feb 14, 2024

fabriziomello Feb 15, 2024

jnidzwetzki commented Feb 15, 2024

fabriziomello commented Feb 15, 2024

fabriziomello commented Feb 15, 2024

Remove metadata when dropping chunk #6621

Remove metadata when dropping chunk #6621

Conversation

fabriziomello commented Feb 8, 2024 • edited Loading

codecov bot commented Feb 8, 2024 • edited Loading

Codecov Report

github-actions bot commented Feb 9, 2024

svenklemm left a comment

Choose a reason for hiding this comment

fabriziomello commented Feb 9, 2024

fabriziomello commented Feb 9, 2024

jnidzwetzki commented Feb 14, 2024

jnidzwetzki Feb 14, 2024

Choose a reason for hiding this comment

fabriziomello Feb 15, 2024

Choose a reason for hiding this comment

jnidzwetzki commented Feb 15, 2024

fabriziomello commented Feb 15, 2024

fabriziomello commented Feb 15, 2024

fabriziomello commented Feb 8, 2024 •

edited

Loading

codecov bot commented Feb 8, 2024 •

edited

Loading