Fix partitionwise agg crash due to uninitialized memory #2874
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
When building the partition information for hypertables, the memory for partition bound information wasn't properly initialized. The uninitialized memory could make the planner believe it could plan an ordered append (PostgreSQL's built-in version) when the contents of the memory happened to have the "right" value (specifically the `strategy` char field of the partition bound info). This change fixes the crash by properly initializing the partition bound info so that PostgreSQL's ordered append plans are not triggered on hypertables. Due to the randomness of the uninitialized memory, it isn't possible to reproduce the crash reliably and create a test for it. But the crash was manually reproduced by initializing the `strategy` field of the partition bound info to a value that triggered the crash in the same place reported by the issue below. Fixes timescale#2873
k-rus
approved these changes
Jan 28, 2021
mkindahl
approved these changes
Jan 28, 2021
Codecov Report
@@ Coverage Diff @@
## master #2874 +/- ##
==========================================
+ Coverage 90.07% 90.20% +0.13%
==========================================
Files 212 212
Lines 34772 34725 -47
==========================================
+ Hits 31322 31325 +3
+ Misses 3450 3400 -50
Continue to review full report at Codecov.
|
erimatnor
requested review from
pmwkaa,
svenklemm and
gayyappan
and removed request for
a team
svenklemm
approved these changes
Jan 28, 2021
Merged
svenklemm
added a commit
to svenklemm/timescaledb
that referenced
this pull request
Jan 28, 2021
This maintenance release contains bugfixes since the 2.0.0 release. We deem it high priority for upgrading. In particular the fixes contained in this maintenance release address issues in continuous aggregates, compression, JOINs with hypertables and when upgrading from previous versions. **Bugfixes** * timescale#2772 Always validate existing database and extension * timescale#2780 Fix config enum entries for remote data fetcher * timescale#2806 Add check for dropped chunk on update * timescale#2828 Improve cagg watermark caching * timescale#2842 Do not mark job as started when setting next_start field * timescale#2845 Fix continuous aggregate privileges during upgrade * timescale#2851 Fix nested loop joins that involve compressed chunks * timescale#2860 Fix projection in ChunkAppend nodes * timescale#2861 Remove compression stat update from update script * timescale#2865 Apply volatile function quals at decompresschunk node * timescale#2866 Avoid partitionwise planning of partialize_agg * timescale#2868 Fix corruption in gapfill plan * timescale#2874 Fix partitionwise agg crash due to uninitialized memory **Thanks** * @alex88 for reporting an issue with joined hypertables * @brian-from-quantrocket for reporting an issue with extension update and dropped chunks * @dhodyn for reporting an issue when joining compressed chunks * @markatosi for reporting a segfault with partitionwise aggregates enabled * @PhilippJust for reporting an issue with add_job and initial_start * @sgorsh for reporting an issue when using pgAdmin on windows * @WarriorOfWire for reporting the bug with gapfill queries not being able to find pathkey item to sort
svenklemm
added a commit
to svenklemm/timescaledb
that referenced
this pull request
Jan 28, 2021
This maintenance release contains bugfixes since the 2.0.0 release. We deem it high priority for upgrading. In particular the fixes contained in this maintenance release address issues in continuous aggregates, compression, JOINs with hypertables and when upgrading from previous versions. **Bugfixes** * timescale#2772 Always validate existing database and extension * timescale#2780 Fix config enum entries for remote data fetcher * timescale#2806 Add check for dropped chunk on update * timescale#2828 Improve cagg watermark caching * timescale#2842 Do not mark job as started when setting next_start field * timescale#2845 Fix continuous aggregate privileges during upgrade * timescale#2851 Fix nested loop joins that involve compressed chunks * timescale#2860 Fix projection in ChunkAppend nodes * timescale#2861 Remove compression stat update from update script * timescale#2865 Apply volatile function quals at decompresschunk node * timescale#2866 Avoid partitionwise planning of partialize_agg * timescale#2868 Fix corruption in gapfill plan * timescale#2874 Fix partitionwise agg crash due to uninitialized memory **Thanks** * @alex88 for reporting an issue with joined hypertables * @brian-from-quantrocket for reporting an issue with extension update and dropped chunks * @dhodyn for reporting an issue when joining compressed chunks * @markatosi for reporting a segfault with partitionwise aggregates enabled * @PhilippJust for reporting an issue with add_job and initial_start * @sgorsh for reporting an issue when using pgAdmin on windows * @WarriorOfWire for reporting the bug with gapfill queries not being able to find pathkey item to sort
svenklemm
added a commit
to svenklemm/timescaledb
that referenced
this pull request
Jan 28, 2021
This maintenance release contains bugfixes since the 2.0.0 release. We deem it high priority for upgrading. In particular the fixes contained in this maintenance release address issues in continuous aggregates, compression, JOINs with hypertables and when upgrading from previous versions. **Bugfixes** * timescale#2772 Always validate existing database and extension * timescale#2780 Fix config enum entries for remote data fetcher * timescale#2806 Add check for dropped chunk on update * timescale#2828 Improve cagg watermark caching * timescale#2842 Do not mark job as started when setting next_start field * timescale#2845 Fix continuous aggregate privileges during upgrade * timescale#2851 Fix nested loop joins that involve compressed chunks * timescale#2860 Fix projection in ChunkAppend nodes * timescale#2861 Remove compression stat update from update script * timescale#2865 Apply volatile function quals at decompresschunk node * timescale#2866 Avoid partitionwise planning of partialize_agg * timescale#2868 Fix corruption in gapfill plan * timescale#2874 Fix partitionwise agg crash due to uninitialized memory **Thanks** * @alex88 for reporting an issue with joined hypertables * @brian-from-quantrocket for reporting an issue with extension update and dropped chunks * @dhodyn for reporting an issue when joining compressed chunks * @markatosi for reporting a segfault with partitionwise aggregates enabled * @PhilippJust for reporting an issue with add_job and initial_start * @sgorsh for reporting an issue when using pgAdmin on windows * @WarriorOfWire for reporting the bug with gapfill queries not being able to find pathkey item to sort
svenklemm
added a commit
that referenced
this pull request
Jan 28, 2021
This maintenance release contains bugfixes since the 2.0.0 release. We deem it high priority for upgrading. In particular the fixes contained in this maintenance release address issues in continuous aggregates, compression, JOINs with hypertables and when upgrading from previous versions. **Bugfixes** * #2772 Always validate existing database and extension * #2780 Fix config enum entries for remote data fetcher * #2806 Add check for dropped chunk on update * #2828 Improve cagg watermark caching * #2842 Do not mark job as started when setting next_start field * #2845 Fix continuous aggregate privileges during upgrade * #2851 Fix nested loop joins that involve compressed chunks * #2860 Fix projection in ChunkAppend nodes * #2861 Remove compression stat update from update script * #2865 Apply volatile function quals at decompresschunk node * #2866 Avoid partitionwise planning of partialize_agg * #2868 Fix corruption in gapfill plan * #2874 Fix partitionwise agg crash due to uninitialized memory **Thanks** * @alex88 for reporting an issue with joined hypertables * @brian-from-quantrocket for reporting an issue with extension update and dropped chunks * @dhodyn for reporting an issue when joining compressed chunks * @markatosi for reporting a segfault with partitionwise aggregates enabled * @PhilippJust for reporting an issue with add_job and initial_start * @sgorsh for reporting an issue when using pgAdmin on windows * @WarriorOfWire for reporting the bug with gapfill queries not being able to find pathkey item to sort
svenklemm
added a commit
that referenced
this pull request
Jan 28, 2021
This maintenance release contains bugfixes since the 2.0.0 release. We deem it high priority for upgrading. In particular the fixes contained in this maintenance release address issues in continuous aggregates, compression, JOINs with hypertables and when upgrading from previous versions. **Bugfixes** * #2772 Always validate existing database and extension * #2780 Fix config enum entries for remote data fetcher * #2806 Add check for dropped chunk on update * #2828 Improve cagg watermark caching * #2838 Fix catalog repair in update script * #2842 Do not mark job as started when setting next_start field * #2845 Fix continuous aggregate privileges during upgrade * #2851 Fix nested loop joins that involve compressed chunks * #2860 Fix projection in ChunkAppend nodes * #2861 Remove compression stat update from update script * #2865 Apply volatile function quals at decompresschunk node * #2866 Avoid partitionwise planning of partialize_agg * #2868 Fix corruption in gapfill plan * #2874 Fix partitionwise agg crash due to uninitialized memory **Thanks** * @alex88 for reporting an issue with joined hypertables * @brian-from-quantrocket for reporting an issue with extension update and dropped chunks * @dhodyn for reporting an issue when joining compressed chunks * @markatosi for reporting a segfault with partitionwise aggregates enabled * @PhilippJust for reporting an issue with add_job and initial_start * @sgorsh for reporting an issue when using pgAdmin on windows * @WarriorOfWire for reporting the bug with gapfill queries not being able to find pathkey item to sort
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When building the partition information for hypertables, the memory
for partition bound information wasn't properly initialized. The
uninitialized memory could make the planner believe it could plan an
ordered append (PostgreSQL's built-in version) when the contents of
the memory happened to have the "right" value (specifically the
strategychar field of the partition bound info).This change fixes the crash by properly initializing the partition
bound info so that PostgreSQL's ordered append plans are not triggered
on hypertables.
Due to the randomness of the uninitialized memory, it isn't possible
to reproduce the crash reliably and create a test for it. But the
crash was manually reproduced by initializing the
strategyfield ofthe partition bound info to a value that triggered the crash in the
same place reported by the issue below.
Fixes #2873