New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix continuous aggregate privileges during upgrade #2845
Merged
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
erimatnor
added
bug
update-script
upgrade
Issue is related to upgrading the extension or the PostgreSQL version.
permissions
labels
Jan 20, 2021
erimatnor
force-pushed
the
fix-cagg-permissions
branch
2 times, most recently
from
January 20, 2021 10:47
4a14fd3
to
cac17ff
Compare
Codecov Report
@@ Coverage Diff @@
## master #2845 +/- ##
==========================================
+ Coverage 90.07% 90.20% +0.12%
==========================================
Files 212 212
Lines 34772 34722 -50
==========================================
Hits 31322 31322
+ Misses 3450 3400 -50
Continue to review full report at Codecov.
|
erimatnor
removed
the
upgrade
Issue is related to upgrading the extension or the PostgreSQL version.
label
Jan 21, 2021
erimatnor
force-pushed
the
fix-cagg-permissions
branch
5 times, most recently
from
January 22, 2021 08:51
f21aad0
to
834f1c8
Compare
erimatnor
force-pushed
the
fix-cagg-permissions
branch
from
January 25, 2021 08:34
834f1c8
to
5465f65
Compare
mkindahl
approved these changes
Jan 25, 2021
svenklemm
approved these changes
Jan 25, 2021
erimatnor
force-pushed
the
fix-cagg-permissions
branch
2 times, most recently
from
January 27, 2021 10:27
19e1fee
to
7d29949
Compare
Copy ACL privileges (grants) from the query view (user-facing object) to the internal objects (e.g., materialized hypertable, direct view, and partial view) when updating the extension to the new version. A previous change added such propagation of privileges when executing `GRANT` statements, but didn't apply it retrospectively to the internal objects of existing continuous aggregates. Having the right permissions on internal objects is also necessary for the watermark function used by real-time aggregation since it queries the materialized hypertable directly. The update script copies the ACL information from the user-facing view of every continuous aggregate to its internal objects (including the materialized hypertable and its chunks). This is done by direct insert into `pg_class` instead of executing a `GRANT` statement in the update script, since the latter will record the grant/ACL as an init privilege (i.e., the system believes the GRANT is for an extension object). The init privilege will prevent this ACL from being included in future dump files, since `pg_dump` only includes non-init privileges as it believes such privileges will be recreated with the extension. Fixes timescale#2825
erimatnor
force-pushed
the
fix-cagg-permissions
branch
from
January 27, 2021 10:32
7d29949
to
eaed20c
Compare
mkindahl
approved these changes
Jan 27, 2021
svenklemm
approved these changes
Jan 27, 2021
svenklemm
added a commit
to svenklemm/timescaledb
that referenced
this pull request
Jan 27, 2021
This maintenance release contains bugfixes since the 2.0.0 release. We deem it high priority for upgrading. In particular the fixes contained in this maintenance release address issues in continuous aggregates, compression, JOINs with hypertables and when upgrading from previous versions. **Bugfixes** * timescale#2772 Always validate existing database and extension * timescale#2780 Fix config enum entries for remote data fetcher * timescale#2806 Add check for dropped chunk on update * timescale#2828 Improve cagg watermark caching * timescale#2842 Do not mark job as started when seting next_start field * timescale#2845 Fix continuous aggregate privileges during upgrade * timescale#2860 Fix projection in ChunkAppend nodes * timescale#2861 Remove compression stat update from update script **Thanks** * @sgorsh for reporting an issue when using pgAdmin on windows * @brian-from-quantrocket for reporting an issue with extension update and dropped chunks * @PhilippJust for reporting an issue with add_job and initial_start * @alex88 for reporting an issue with joined hypertables
Merged
svenklemm
added a commit
to svenklemm/timescaledb
that referenced
this pull request
Jan 27, 2021
This maintenance release contains bugfixes since the 2.0.0 release. We deem it high priority for upgrading. In particular the fixes contained in this maintenance release address issues in continuous aggregates, compression, JOINs with hypertables and when upgrading from previous versions. **Bugfixes** * timescale#2772 Always validate existing database and extension * timescale#2780 Fix config enum entries for remote data fetcher * timescale#2806 Add check for dropped chunk on update * timescale#2828 Improve cagg watermark caching * timescale#2842 Do not mark job as started when seting next_start field * timescale#2845 Fix continuous aggregate privileges during upgrade * timescale#2851 Fix nested loop joins that involve compressed chunks * timescale#2860 Fix projection in ChunkAppend nodes * timescale#2861 Remove compression stat update from update script * timescale#2865 Apply volatile function quals at decompresschunk node **Thanks** * @alex88 for reporting an issue with joined hypertables * @brian-from-quantrocket for reporting an issue with extension update and dropped chunks * @dhodyn for reporting an issue when joining compressed chunks * @PhilippJust for reporting an issue with add_job and initial_start * @sgorsh for reporting an issue when using pgAdmin on windows
svenklemm
added a commit
to svenklemm/timescaledb
that referenced
this pull request
Jan 27, 2021
This maintenance release contains bugfixes since the 2.0.0 release. We deem it high priority for upgrading. In particular the fixes contained in this maintenance release address issues in continuous aggregates, compression, JOINs with hypertables and when upgrading from previous versions. **Bugfixes** * timescale#2772 Always validate existing database and extension * timescale#2780 Fix config enum entries for remote data fetcher * timescale#2806 Add check for dropped chunk on update * timescale#2828 Improve cagg watermark caching * timescale#2842 Do not mark job as started when seting next_start field * timescale#2845 Fix continuous aggregate privileges during upgrade * timescale#2851 Fix nested loop joins that involve compressed chunks * timescale#2860 Fix projection in ChunkAppend nodes * timescale#2861 Remove compression stat update from update script * timescale#2865 Apply volatile function quals at decompresschunk node **Thanks** * @alex88 for reporting an issue with joined hypertables * @brian-from-quantrocket for reporting an issue with extension update and dropped chunks * @dhodyn for reporting an issue when joining compressed chunks * @PhilippJust for reporting an issue with add_job and initial_start * @sgorsh for reporting an issue when using pgAdmin on windows
svenklemm
added a commit
to svenklemm/timescaledb
that referenced
this pull request
Jan 27, 2021
This maintenance release contains bugfixes since the 2.0.0 release. We deem it high priority for upgrading. In particular the fixes contained in this maintenance release address issues in continuous aggregates, compression, JOINs with hypertables and when upgrading from previous versions. **Bugfixes** * timescale#2772 Always validate existing database and extension * timescale#2780 Fix config enum entries for remote data fetcher * timescale#2806 Add check for dropped chunk on update * timescale#2828 Improve cagg watermark caching * timescale#2842 Do not mark job as started when seting next_start field * timescale#2845 Fix continuous aggregate privileges during upgrade * timescale#2851 Fix nested loop joins that involve compressed chunks * timescale#2860 Fix projection in ChunkAppend nodes * timescale#2861 Remove compression stat update from update script * timescale#2865 Apply volatile function quals at decompresschunk node **Thanks** * @alex88 for reporting an issue with joined hypertables * @brian-from-quantrocket for reporting an issue with extension update and dropped chunks * @dhodyn for reporting an issue when joining compressed chunks * @PhilippJust for reporting an issue with add_job and initial_start * @sgorsh for reporting an issue when using pgAdmin on windows
svenklemm
added a commit
to svenklemm/timescaledb
that referenced
this pull request
Jan 27, 2021
This maintenance release contains bugfixes since the 2.0.0 release. We deem it high priority for upgrading. In particular the fixes contained in this maintenance release address issues in continuous aggregates, compression, JOINs with hypertables and when upgrading from previous versions. **Bugfixes** * timescale#2772 Always validate existing database and extension * timescale#2780 Fix config enum entries for remote data fetcher * timescale#2806 Add check for dropped chunk on update * timescale#2828 Improve cagg watermark caching * timescale#2842 Do not mark job as started when seting next_start field * timescale#2845 Fix continuous aggregate privileges during upgrade * timescale#2851 Fix nested loop joins that involve compressed chunks * timescale#2860 Fix projection in ChunkAppend nodes * timescale#2861 Remove compression stat update from update script * timescale#2865 Apply volatile function quals at decompresschunk node **Thanks** * @alex88 for reporting an issue with joined hypertables * @brian-from-quantrocket for reporting an issue with extension update and dropped chunks * @dhodyn for reporting an issue when joining compressed chunks * @PhilippJust for reporting an issue with add_job and initial_start * @sgorsh for reporting an issue when using pgAdmin on windows
svenklemm
added a commit
to svenklemm/timescaledb
that referenced
this pull request
Jan 28, 2021
This maintenance release contains bugfixes since the 2.0.0 release. We deem it high priority for upgrading. In particular the fixes contained in this maintenance release address issues in continuous aggregates, compression, JOINs with hypertables and when upgrading from previous versions. **Bugfixes** * timescale#2772 Always validate existing database and extension * timescale#2780 Fix config enum entries for remote data fetcher * timescale#2806 Add check for dropped chunk on update * timescale#2828 Improve cagg watermark caching * timescale#2842 Do not mark job as started when setting next_start field * timescale#2845 Fix continuous aggregate privileges during upgrade * timescale#2851 Fix nested loop joins that involve compressed chunks * timescale#2860 Fix projection in ChunkAppend nodes * timescale#2861 Remove compression stat update from update script * timescale#2865 Apply volatile function quals at decompresschunk node * timescale#2866 Avoid partitionwise planning of partialize_agg * timescale#2868 Fix corruption in gapfill plan * timescale#2874 Fix partitionwise agg crash due to uninitialized memory **Thanks** * @alex88 for reporting an issue with joined hypertables * @brian-from-quantrocket for reporting an issue with extension update and dropped chunks * @dhodyn for reporting an issue when joining compressed chunks * @markatosi for reporting a segfault with partitionwise aggregates enabled * @PhilippJust for reporting an issue with add_job and initial_start * @sgorsh for reporting an issue when using pgAdmin on windows * @WarriorOfWire for reporting the bug with gapfill queries not being able to find pathkey item to sort
svenklemm
added a commit
to svenklemm/timescaledb
that referenced
this pull request
Jan 28, 2021
This maintenance release contains bugfixes since the 2.0.0 release. We deem it high priority for upgrading. In particular the fixes contained in this maintenance release address issues in continuous aggregates, compression, JOINs with hypertables and when upgrading from previous versions. **Bugfixes** * timescale#2772 Always validate existing database and extension * timescale#2780 Fix config enum entries for remote data fetcher * timescale#2806 Add check for dropped chunk on update * timescale#2828 Improve cagg watermark caching * timescale#2842 Do not mark job as started when setting next_start field * timescale#2845 Fix continuous aggregate privileges during upgrade * timescale#2851 Fix nested loop joins that involve compressed chunks * timescale#2860 Fix projection in ChunkAppend nodes * timescale#2861 Remove compression stat update from update script * timescale#2865 Apply volatile function quals at decompresschunk node * timescale#2866 Avoid partitionwise planning of partialize_agg * timescale#2868 Fix corruption in gapfill plan * timescale#2874 Fix partitionwise agg crash due to uninitialized memory **Thanks** * @alex88 for reporting an issue with joined hypertables * @brian-from-quantrocket for reporting an issue with extension update and dropped chunks * @dhodyn for reporting an issue when joining compressed chunks * @markatosi for reporting a segfault with partitionwise aggregates enabled * @PhilippJust for reporting an issue with add_job and initial_start * @sgorsh for reporting an issue when using pgAdmin on windows * @WarriorOfWire for reporting the bug with gapfill queries not being able to find pathkey item to sort
svenklemm
added a commit
to svenklemm/timescaledb
that referenced
this pull request
Jan 28, 2021
This maintenance release contains bugfixes since the 2.0.0 release. We deem it high priority for upgrading. In particular the fixes contained in this maintenance release address issues in continuous aggregates, compression, JOINs with hypertables and when upgrading from previous versions. **Bugfixes** * timescale#2772 Always validate existing database and extension * timescale#2780 Fix config enum entries for remote data fetcher * timescale#2806 Add check for dropped chunk on update * timescale#2828 Improve cagg watermark caching * timescale#2842 Do not mark job as started when setting next_start field * timescale#2845 Fix continuous aggregate privileges during upgrade * timescale#2851 Fix nested loop joins that involve compressed chunks * timescale#2860 Fix projection in ChunkAppend nodes * timescale#2861 Remove compression stat update from update script * timescale#2865 Apply volatile function quals at decompresschunk node * timescale#2866 Avoid partitionwise planning of partialize_agg * timescale#2868 Fix corruption in gapfill plan * timescale#2874 Fix partitionwise agg crash due to uninitialized memory **Thanks** * @alex88 for reporting an issue with joined hypertables * @brian-from-quantrocket for reporting an issue with extension update and dropped chunks * @dhodyn for reporting an issue when joining compressed chunks * @markatosi for reporting a segfault with partitionwise aggregates enabled * @PhilippJust for reporting an issue with add_job and initial_start * @sgorsh for reporting an issue when using pgAdmin on windows * @WarriorOfWire for reporting the bug with gapfill queries not being able to find pathkey item to sort
svenklemm
added a commit
that referenced
this pull request
Jan 28, 2021
This maintenance release contains bugfixes since the 2.0.0 release. We deem it high priority for upgrading. In particular the fixes contained in this maintenance release address issues in continuous aggregates, compression, JOINs with hypertables and when upgrading from previous versions. **Bugfixes** * #2772 Always validate existing database and extension * #2780 Fix config enum entries for remote data fetcher * #2806 Add check for dropped chunk on update * #2828 Improve cagg watermark caching * #2842 Do not mark job as started when setting next_start field * #2845 Fix continuous aggregate privileges during upgrade * #2851 Fix nested loop joins that involve compressed chunks * #2860 Fix projection in ChunkAppend nodes * #2861 Remove compression stat update from update script * #2865 Apply volatile function quals at decompresschunk node * #2866 Avoid partitionwise planning of partialize_agg * #2868 Fix corruption in gapfill plan * #2874 Fix partitionwise agg crash due to uninitialized memory **Thanks** * @alex88 for reporting an issue with joined hypertables * @brian-from-quantrocket for reporting an issue with extension update and dropped chunks * @dhodyn for reporting an issue when joining compressed chunks * @markatosi for reporting a segfault with partitionwise aggregates enabled * @PhilippJust for reporting an issue with add_job and initial_start * @sgorsh for reporting an issue when using pgAdmin on windows * @WarriorOfWire for reporting the bug with gapfill queries not being able to find pathkey item to sort
svenklemm
added a commit
that referenced
this pull request
Jan 28, 2021
This maintenance release contains bugfixes since the 2.0.0 release. We deem it high priority for upgrading. In particular the fixes contained in this maintenance release address issues in continuous aggregates, compression, JOINs with hypertables and when upgrading from previous versions. **Bugfixes** * #2772 Always validate existing database and extension * #2780 Fix config enum entries for remote data fetcher * #2806 Add check for dropped chunk on update * #2828 Improve cagg watermark caching * #2838 Fix catalog repair in update script * #2842 Do not mark job as started when setting next_start field * #2845 Fix continuous aggregate privileges during upgrade * #2851 Fix nested loop joins that involve compressed chunks * #2860 Fix projection in ChunkAppend nodes * #2861 Remove compression stat update from update script * #2865 Apply volatile function quals at decompresschunk node * #2866 Avoid partitionwise planning of partialize_agg * #2868 Fix corruption in gapfill plan * #2874 Fix partitionwise agg crash due to uninitialized memory **Thanks** * @alex88 for reporting an issue with joined hypertables * @brian-from-quantrocket for reporting an issue with extension update and dropped chunks * @dhodyn for reporting an issue when joining compressed chunks * @markatosi for reporting a segfault with partitionwise aggregates enabled * @PhilippJust for reporting an issue with add_job and initial_start * @sgorsh for reporting an issue when using pgAdmin on windows * @WarriorOfWire for reporting the bug with gapfill queries not being able to find pathkey item to sort
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Copy ACL privileges (grants) from the query view (user-facing object)
to the internal objects (e.g., materialized hypertable, direct view,
and partial view) when updating the extension to the new version. A
previous change added such propagation of privileges when executing
GRANT
statements, but didn't apply it retrospectively to theinternal objects of existing continuous aggregates.
Having the right permissions on internal objects is also necessary for
the watermark function used by real-time aggregation since it queries
the materialized hypertable directly.
The update script copies the ACL information from the user-facing view
of every continuous aggregate to its internal objects (including the
materialized hypertable and its chunks). This is done by direct insert
into
pg_class
instead of executing aGRANT
statement in the updatescript, since the latter will record the grant/ACL as an init
privilege (i.e., the system believes the GRANT is for an extension
object). The init privilege will prevent this ACL from being included
in future dump files, since
pg_dump
only includes non-initprivileges as it believes such privileges will be recreated with the
extension.
Fixes #2825