Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix gapfill behaviour around dst switches #6908

Merged
merged 1 commit into from
May 13, 2024

Conversation

svenklemm
Copy link
Member

When advancing local time we might not actually advance the timestamp we process e.g. when we switch to DST in spring the local time advances by 1 hour but the UTC time stays the same. Therefore we need to keep advancing until we actually move forward.

Fixes: #6788

@svenklemm svenklemm requested a review from antekresic May 11, 2024 07:18
@svenklemm svenklemm self-assigned this May 11, 2024
Copy link

codecov bot commented May 11, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 80.75%. Comparing base (59f50f2) to head (67dead3).
Report is 153 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #6908      +/-   ##
==========================================
+ Coverage   80.06%   80.75%   +0.69%     
==========================================
  Files         190      199       +9     
  Lines       37181    37231      +50     
  Branches     9450     9732     +282     
==========================================
+ Hits        29770    30067     +297     
- Misses       2997     3202     +205     
+ Partials     4414     3962     -452     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

@antekresic antekresic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved with small nit.

@@ -656,7 +656,8 @@ gapfill_advance_timestamp(GapFillState *state)
* To be consistent with time_bucket we do UTC bucketing unless
* a different timezone got explicitly passed to the function.
*/
if (state->have_timezone)
if (state->have_timezone &&
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment update as to why we do this only for non-fixed intervals.

When advancing local time we might not actually advance the
timestamp we process e.g. when we switch to DST in spring
the local time advances by 1 hour but the UTC time stays the same.
This patch changes the advance function to only advance in local
time when we are dealing with non-fixed intervals, that is intervals
that have a day or month component.
@svenklemm svenklemm enabled auto-merge (rebase) May 13, 2024 11:19
@svenklemm svenklemm merged commit 9c86369 into timescale:main May 13, 2024
41 checks passed
fabriziomello added a commit to fabriziomello/timescaledb that referenced this pull request May 27, 2024
This release contains bug fixes since the 2.15.0 release.
We recommend that you upgrade at the next available opportunity.

**Bugfixes**
* timescale#6540 Segmentation fault when backfilling data with COPY into a compressed chunk
* timescale#6858 Before update trigger not working correctly
* timescale#6908 Fix gapfill with timezone behaviour around dst switches
* timescale#6911 Fix dropped chunk metadata removal in update script
* timescale#6940 Fix `pg_upgrade` failure by removing `regprocedure` from catalog table
* timescale#6957 Fix segfault in UNION queries with ordering on compressed chunks

**Thanks**
* @DiAifU, @kiddhombre and @intermittentnrg for reporting issues with gapfill and daylight saving time
* @edgarzamora for reporting issue with update triggers
* @hongquan for reporting an issue with the update script
* @iliastsa and @SystemParadox for reporting an issue with COPY into a compressed chunk
@fabriziomello fabriziomello mentioned this pull request May 27, 2024
pallavisontakke added a commit to pallavisontakke/timescaledb that referenced this pull request May 28, 2024
the 2.14.2 release. We recommend that you upgrade at the next
available opportunity.

**Bugfixes**
* timescale#6540 Segmentation fault when backfilling data with COPY into a compressed chunk
* timescale#6858 Before update trigger not working correctly
* timescale#6908 Fix gapfill with timezone behaviour around dst switches
* timescale#6911 Fix dropped chunk metadata removal in update script
* timescale#6940 Fix `pg_upgrade` failure by removing `regprocedure` from catalog table
* timescale#6957 Fix segfault in UNION queries with ordering on compressed chunks

**Thanks**
* @DiAifU, @kiddhombre and @intermittentnrg for reporting issues with gapfill and daylight saving time
* @edgarzamora for reporting issue with update triggers
* @hongquan for reporting an issue with the update script
* @iliastsa and @SystemParadox for reporting an issue with COPY into a compressed chunk
@pallavisontakke pallavisontakke mentioned this pull request May 28, 2024
pallavisontakke added a commit that referenced this pull request May 28, 2024
This release contains performance improvements and bug fixes since
the 2.15.0 release. Best practice is to upgrade at the next 
available opportunity.

**Migrating from self-hosted TimescaleDB v2.14.x and earlier**

After you run `ALTER EXTENSION`, you must run [this SQL script](https://github.com/timescale/timescaledb-extras/blob/master/utils/2.15.X-fix_hypertable_foreign_keys.sql). For more details, see the following pull request [#6797](#6797).

If you are migrating from TimescaleDB v2.15.0, no changes are required.

**Bugfixes**
* #6540: Segmentation fault when you backfill data using COPY into a compressed chunk.
* #6858: `BEFORE UPDATE` trigger not working correctly. 
* #6908: Fix `time_bucket_gapfill()` with timezone behaviour around daylight savings time (DST) switches.
* #6911: Fix dropped chunk metadata removal in the update script. 
* #6940: Fix `pg_upgrade` failure by removing `regprocedure` from the catalog table.
* #6957: Fix the `segfault` in UNION queries that contain ordering on compressed chunks.

**Thanks**
* @DiAifU, @kiddhombre and @intermittentnrg for reporting the issues with gapfill and daylight saving time.
* @edgarzamora for reporting the issue with update triggers.
* @hongquan for reporting the issue with the update script.
* @iliastsa and @SystemParadox for reporting the issue with COPY into a compressed chunk.
fabriziomello added a commit to fabriziomello/timescaledb that referenced this pull request May 28, 2024
This release contains bug fixes since the 2.15.0 release.
We recommend that you upgrade at the next available opportunity.

**Bugfixes**
* timescale#6540 Segmentation fault when backfilling data with COPY into a compressed chunk
* timescale#6858 Before update trigger not working correctly
* timescale#6908 Fix gapfill with timezone behaviour around dst switches
* timescale#6911 Fix dropped chunk metadata removal in update script
* timescale#6940 Fix `pg_upgrade` failure by removing `regprocedure` from catalog table
* timescale#6957 Fix segfault in UNION queries with ordering on compressed chunks

**Thanks**
* @DiAifU, @kiddhombre and @intermittentnrg for reporting issues with gapfill and daylight saving time
* @edgarzamora for reporting issue with update triggers
* @hongquan for reporting an issue with the update script
* @iliastsa and @SystemParadox for reporting an issue with COPY into a compressed chunk
fabriziomello added a commit that referenced this pull request May 29, 2024
This release contains bug fixes since the 2.15.0 release.
We recommend that you upgrade at the next available opportunity.

**Bugfixes**
* #6540 Segmentation fault when backfilling data with COPY into a compressed chunk
* #6858 Before update trigger not working correctly
* #6908 Fix gapfill with timezone behaviour around dst switches
* #6911 Fix dropped chunk metadata removal in update script
* #6940 Fix `pg_upgrade` failure by removing `regprocedure` from catalog table
* #6957 Fix segfault in UNION queries with ordering on compressed chunks

**Thanks**
* @DiAifU, @kiddhombre and @intermittentnrg for reporting issues with gapfill and daylight saving time
* @edgarzamora for reporting issue with update triggers
* @hongquan for reporting an issue with the update script
* @iliastsa and @SystemParadox for reporting an issue with COPY into a compressed chunk
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Bug]: time_bucket_gapfill using timezone returns incorrect data for daylight saving
4 participants