New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support continuous aggregates from multiple hypertables? #3314
Comments
This would really help, we just need a simple join to enrich with additional metadata. Could we ignore this exception maybe with a configuration? |
**Bugfixes** * timescale#3279 Add some more randomness to chunk assignment * timescale#3288 Fix failed update with parallel workers * timescale#3300 Improve trigger handling on distributed hypertables * timescale#3304 Remove paths that reference parent relids for compressed chunks * timescale#3305 Fix pull_varnos miscomputation of relids set * timescale#3310 Generate downgrade script * timescale#3314 Fix heap buffer overflow in hypertable expansion * timescale#3317 Fix heap buffer overflow in remote connection cache. * timescale#3327 Make aggregate in caggs fully qualified * timescale#3327 Make aggregates in caggs fully qualified * timescale#3336 Fix pg_init_privs objsubid handling * timescale#3345 Fix SkipScan distinct column identification * timescale#3355 Fix heap buffer overflow when renaming compressed hypertable columns. * timescale#3367 Improve DecompressChunk qual pushdown * timescale#3377 Fix bad use of repalloc **Thanks** * @db-adrian for reporting an issue when accessing cagg view through postgres_fdw * @fncaldas and @pgwhalen for reporting an issue accessing caggs when public is not in search_path * @fvannee, @mglonnro and @ebreijo for reporting an issue with the upgrade script * @fvannee for reporting a performance regression with SkipScan
**Bugfixes** * timescale#3279 Add some more randomness to chunk assignment * timescale#3288 Fix failed update with parallel workers * timescale#3300 Improve trigger handling on distributed hypertables * timescale#3304 Remove paths that reference parent relids for compressed chunks * timescale#3305 Fix pull_varnos miscomputation of relids set * timescale#3310 Generate downgrade script * timescale#3314 Fix heap buffer overflow in hypertable expansion * timescale#3317 Fix heap buffer overflow in remote connection cache. * timescale#3327 Make aggregate in caggs fully qualified * timescale#3327 Make aggregates in caggs fully qualified * timescale#3336 Fix pg_init_privs objsubid handling * timescale#3345 Fix SkipScan distinct column identification * timescale#3355 Fix heap buffer overflow when renaming compressed hypertable columns. * timescale#3367 Improve DecompressChunk qual pushdown * timescale#3377 Fix bad use of repalloc **Thanks** * @db-adrian for reporting an issue when accessing cagg view through postgres_fdw * @fncaldas and @pgwhalen for reporting an issue accessing caggs when public is not in search_path * @fvannee, @mglonnro and @ebreijo for reporting an issue with the upgrade script * @fvannee for reporting a performance regression with SkipScan
**Bugfixes** * timescale#3279 Add some more randomness to chunk assignment * timescale#3288 Fix failed update with parallel workers * timescale#3300 Improve trigger handling on distributed hypertables * timescale#3304 Remove paths that reference parent relids for compressed chunks * timescale#3305 Fix pull_varnos miscomputation of relids set * timescale#3310 Generate downgrade script * timescale#3314 Fix heap buffer overflow in hypertable expansion * timescale#3317 Fix heap buffer overflow in remote connection cache. * timescale#3327 Make aggregate in caggs fully qualified * timescale#3327 Make aggregates in caggs fully qualified * timescale#3336 Fix pg_init_privs objsubid handling * timescale#3345 Fix SkipScan distinct column identification * timescale#3355 Fix heap buffer overflow when renaming compressed hypertable columns. * timescale#3367 Improve DecompressChunk qual pushdown * timescale#3377 Fix bad use of repalloc **Thanks** * @db-adrian for reporting an issue when accessing cagg view through postgres_fdw * @fncaldas and @pgwhalen for reporting an issue accessing caggs when public is not in search_path * @fvannee, @mglonnro and @ebreijo for reporting an issue with the upgrade script * @fvannee for reporting a performance regression with SkipScan
**Bugfixes** * timescale#3279 Add some more randomness to chunk assignment * timescale#3288 Fix failed update with parallel workers * timescale#3300 Improve trigger handling on distributed hypertables * timescale#3304 Remove paths that reference parent relids for compressed chunks * timescale#3305 Fix pull_varnos miscomputation of relids set * timescale#3310 Generate downgrade script * timescale#3314 Fix heap buffer overflow in hypertable expansion * timescale#3317 Fix heap buffer overflow in remote connection cache. * timescale#3327 Make aggregate in caggs fully qualified * timescale#3327 Make aggregates in caggs fully qualified * timescale#3336 Fix pg_init_privs objsubid handling * timescale#3345 Fix SkipScan distinct column identification * timescale#3355 Fix heap buffer overflow when renaming compressed hypertable columns. * timescale#3367 Improve DecompressChunk qual pushdown * timescale#3377 Fix bad use of repalloc **Thanks** * @db-adrian for reporting an issue when accessing cagg view through postgres_fdw * @fncaldas and @pgwhalen for reporting an issue accessing caggs when public is not in search_path * @fvannee, @mglonnro and @ebreijo for reporting an issue with the upgrade script * @fvannee for reporting a performance regression with SkipScan
**Bugfixes** * timescale#3279 Add some more randomness to chunk assignment * timescale#3288 Fix failed update with parallel workers * timescale#3300 Improve trigger handling on distributed hypertables * timescale#3304 Remove paths that reference parent relids for compressed chunks * timescale#3305 Fix pull_varnos miscomputation of relids set * timescale#3310 Generate downgrade script * timescale#3314 Fix heap buffer overflow in hypertable expansion * timescale#3317 Fix heap buffer overflow in remote connection cache. * timescale#3327 Make aggregate in caggs fully qualified * timescale#3327 Make aggregates in caggs fully qualified * timescale#3336 Fix pg_init_privs objsubid handling * timescale#3345 Fix SkipScan distinct column identification * timescale#3355 Fix heap buffer overflow when renaming compressed hypertable columns. * timescale#3367 Improve DecompressChunk qual pushdown * timescale#3377 Fix bad use of repalloc **Thanks** * @db-adrian for reporting an issue when accessing cagg view through postgres_fdw * @fncaldas and @pgwhalen for reporting an issue accessing caggs when public is not in search_path * @fvannee, @mglonnro and @ebreijo for reporting an issue with the upgrade script * @fvannee for reporting a performance regression with SkipScan
**Bugfixes** * timescale#3279 Add some more randomness to chunk assignment * timescale#3288 Fix failed update with parallel workers * timescale#3300 Improve trigger handling on distributed hypertables * timescale#3304 Remove paths that reference parent relids for compressed chunks * timescale#3305 Fix pull_varnos miscomputation of relids set * timescale#3310 Generate downgrade script * timescale#3314 Fix heap buffer overflow in hypertable expansion * timescale#3317 Fix heap buffer overflow in remote connection cache. * timescale#3327 Make aggregate in caggs fully qualified * timescale#3327 Make aggregates in caggs fully qualified * timescale#3336 Fix pg_init_privs objsubid handling * timescale#3345 Fix SkipScan distinct column identification * timescale#3355 Fix heap buffer overflow when renaming compressed hypertable columns. * timescale#3367 Improve DecompressChunk qual pushdown * timescale#3377 Fix bad use of repalloc **Thanks** * @db-adrian for reporting an issue when accessing cagg view through postgres_fdw * @fncaldas and @pgwhalen for reporting an issue accessing caggs when public is not in search_path * @fvannee, @mglonnro and @ebreijo for reporting an issue with the upgrade script * @fvannee for reporting a performance regression with SkipScan
**Bugfixes** * timescale#3279 Add some more randomness to chunk assignment * timescale#3288 Fix failed update with parallel workers * timescale#3300 Improve trigger handling on distributed hypertables * timescale#3304 Remove paths that reference parent relids for compressed chunks * timescale#3305 Fix pull_varnos miscomputation of relids set * timescale#3310 Generate downgrade script * timescale#3314 Fix heap buffer overflow in hypertable expansion * timescale#3317 Fix heap buffer overflow in remote connection cache. * timescale#3327 Make aggregate in caggs fully qualified * timescale#3327 Make aggregates in caggs fully qualified * timescale#3336 Fix pg_init_privs objsubid handling * timescale#3345 Fix SkipScan distinct column identification * timescale#3355 Fix heap buffer overflow when renaming compressed hypertable columns. * timescale#3367 Improve DecompressChunk qual pushdown * timescale#3377 Fix bad use of repalloc **Thanks** * @db-adrian for reporting an issue when accessing cagg view through postgres_fdw * @fncaldas and @pgwhalen for reporting an issue accessing caggs when public is not in search_path * @fvannee, @mglonnro and @ebreijo for reporting an issue with the upgrade script * @fvannee for reporting a performance regression with SkipScan
**Bugfixes** * timescale#3279 Add some more randomness to chunk assignment * timescale#3288 Fix failed update with parallel workers * timescale#3300 Improve trigger handling on distributed hypertables * timescale#3304 Remove paths that reference parent relids for compressed chunks * timescale#3305 Fix pull_varnos miscomputation of relids set * timescale#3310 Generate downgrade script * timescale#3314 Fix heap buffer overflow in hypertable expansion * timescale#3317 Fix heap buffer overflow in remote connection cache. * timescale#3327 Make aggregate in caggs fully qualified * timescale#3327 Make aggregates in caggs fully qualified * timescale#3336 Fix pg_init_privs objsubid handling * timescale#3345 Fix SkipScan distinct column identification * timescale#3355 Fix heap buffer overflow when renaming compressed hypertable columns. * timescale#3367 Improve DecompressChunk qual pushdown * timescale#3377 Fix bad use of repalloc **Thanks** * @db-adrian for reporting an issue when accessing cagg view through postgres_fdw * @fncaldas and @pgwhalen for reporting an issue accessing caggs when public is not in search_path * @fvannee, @mglonnro and @ebreijo for reporting an issue with the upgrade script * @fvannee for reporting a performance regression with SkipScan
**Bugfixes** * #3279 Add some more randomness to chunk assignment * #3288 Fix failed update with parallel workers * #3300 Improve trigger handling on distributed hypertables * #3304 Remove paths that reference parent relids for compressed chunks * #3305 Fix pull_varnos miscomputation of relids set * #3310 Generate downgrade script * #3314 Fix heap buffer overflow in hypertable expansion * #3317 Fix heap buffer overflow in remote connection cache. * #3327 Make aggregate in caggs fully qualified * #3327 Make aggregates in caggs fully qualified * #3336 Fix pg_init_privs objsubid handling * #3345 Fix SkipScan distinct column identification * #3355 Fix heap buffer overflow when renaming compressed hypertable columns. * #3367 Improve DecompressChunk qual pushdown * #3377 Fix bad use of repalloc **Thanks** * @db-adrian for reporting an issue when accessing cagg view through postgres_fdw * @fncaldas and @pgwhalen for reporting an issue accessing caggs when public is not in search_path * @fvannee, @mglonnro and @ebreijo for reporting an issue with the upgrade script * @fvannee for reporting a performance regression with SkipScan
**Bugfixes** * timescale#3279 Add some more randomness to chunk assignment * timescale#3288 Fix failed update with parallel workers * timescale#3300 Improve trigger handling on distributed hypertables * timescale#3304 Remove paths that reference parent relids for compressed chunks * timescale#3305 Fix pull_varnos miscomputation of relids set * timescale#3310 Generate downgrade script * timescale#3314 Fix heap buffer overflow in hypertable expansion * timescale#3317 Fix heap buffer overflow in remote connection cache. * timescale#3327 Make aggregate in caggs fully qualified * timescale#3327 Make aggregates in caggs fully qualified * timescale#3336 Fix pg_init_privs objsubid handling * timescale#3345 Fix SkipScan distinct column identification * timescale#3355 Fix heap buffer overflow when renaming compressed hypertable columns. * timescale#3367 Improve DecompressChunk qual pushdown * timescale#3377 Fix bad use of repalloc **Thanks** * @db-adrian for reporting an issue when accessing cagg view through postgres_fdw * @fncaldas and @pgwhalen for reporting an issue accessing caggs when public is not in search_path * @fvannee, @mglonnro and @ebreijo for reporting an issue with the upgrade script * @fvannee for reporting a performance regression with SkipScan
**Bugfixes** * timescale#3279 Add some more randomness to chunk assignment * timescale#3288 Fix failed update with parallel workers * timescale#3300 Improve trigger handling on distributed hypertables * timescale#3304 Remove paths that reference parent relids for compressed chunks * timescale#3305 Fix pull_varnos miscomputation of relids set * timescale#3310 Generate downgrade script * timescale#3314 Fix heap buffer overflow in hypertable expansion * timescale#3317 Fix heap buffer overflow in remote connection cache. * timescale#3327 Make aggregate in caggs fully qualified * timescale#3327 Make aggregates in caggs fully qualified * timescale#3336 Fix pg_init_privs objsubid handling * timescale#3345 Fix SkipScan distinct column identification * timescale#3355 Fix heap buffer overflow when renaming compressed hypertable columns. * timescale#3367 Improve DecompressChunk qual pushdown * timescale#3377 Fix bad use of repalloc **Thanks** * @db-adrian for reporting an issue when accessing cagg view through postgres_fdw * @fncaldas and @pgwhalen for reporting an issue accessing caggs when public is not in search_path * @fvannee, @mglonnro and @ebreijo for reporting an issue with the upgrade script * @fvannee for reporting a performance regression with SkipScan
**Bugfixes** * timescale#3279 Add some more randomness to chunk assignment * timescale#3288 Fix failed update with parallel workers * timescale#3300 Improve trigger handling on distributed hypertables * timescale#3304 Remove paths that reference parent relids for compressed chunks * timescale#3305 Fix pull_varnos miscomputation of relids set * timescale#3310 Generate downgrade script * timescale#3314 Fix heap buffer overflow in hypertable expansion * timescale#3317 Fix heap buffer overflow in remote connection cache. * timescale#3327 Make aggregate in caggs fully qualified * timescale#3327 Make aggregates in caggs fully qualified * timescale#3336 Fix pg_init_privs objsubid handling * timescale#3345 Fix SkipScan distinct column identification * timescale#3355 Fix heap buffer overflow when renaming compressed hypertable columns. * timescale#3367 Improve DecompressChunk qual pushdown * timescale#3377 Fix bad use of repalloc **Thanks** * @db-adrian for reporting an issue when accessing cagg view through postgres_fdw * @fncaldas and @pgwhalen for reporting an issue accessing caggs when public is not in search_path * @fvannee, @mglonnro and @ebreijo for reporting an issue with the upgrade script * @fvannee for reporting a performance regression with SkipScan
This maintenance release contains bugfixes since the 2.3.0 release. We deem it moderate priority for upgrading. **Bugfixes** * timescale#3279 Add some more randomness to chunk assignment * timescale#3288 Fix failed update with parallel workers * timescale#3300 Improve trigger handling on distributed hypertables * timescale#3304 Remove paths that reference parent relids for compressed chunks * timescale#3305 Fix pull_varnos miscomputation of relids set * timescale#3310 Generate downgrade script * timescale#3314 Fix heap buffer overflow in hypertable expansion * timescale#3317 Fix heap buffer overflow in remote connection cache. * timescale#3327 Make aggregate in caggs fully qualified * timescale#3327 Make aggregates in caggs fully qualified * timescale#3336 Fix pg_init_privs objsubid handling * timescale#3345 Fix SkipScan distinct column identification * timescale#3355 Fix heap buffer overflow when renaming compressed hypertable columns. * timescale#3367 Improve DecompressChunk qual pushdown * timescale#3377 Fix bad use of repalloc **Thanks** * @db-adrian for reporting an issue when accessing cagg view through postgres_fdw * @fncaldas and @pgwhalen for reporting an issue accessing caggs when public is not in search_path * @fvannee, @mglonnro and @ebreijo for reporting an issue with the upgrade script * @fvannee for reporting a performance regression with SkipScan
This maintenance release contains bugfixes since the 2.3.0 release. We deem it moderate priority for upgrading. **Bugfixes** * timescale#3279 Add some more randomness to chunk assignment * timescale#3288 Fix failed update with parallel workers * timescale#3300 Improve trigger handling on distributed hypertables * timescale#3304 Remove paths that reference parent relids for compressed chunks * timescale#3305 Fix pull_varnos miscomputation of relids set * timescale#3310 Generate downgrade script * timescale#3314 Fix heap buffer overflow in hypertable expansion * timescale#3317 Fix heap buffer overflow in remote connection cache. * timescale#3327 Make aggregate in caggs fully qualified * timescale#3327 Make aggregates in caggs fully qualified * timescale#3336 Fix pg_init_privs objsubid handling * timescale#3345 Fix SkipScan distinct column identification * timescale#3355 Fix heap buffer overflow when renaming compressed hypertable columns. * timescale#3367 Improve DecompressChunk qual pushdown * timescale#3377 Fix bad use of repalloc **Thanks** * @db-adrian for reporting an issue when accessing cagg view through postgres_fdw * @fncaldas and @pgwhalen for reporting an issue accessing caggs when public is not in search_path * @fvannee, @mglonnro and @ebreijo for reporting an issue with the upgrade script * @fvannee for reporting a performance regression with SkipScan
This maintenance release contains bugfixes since the 2.3.0 release. We deem it moderate priority for upgrading. The release introduces the possibility of generating downgrade scripts, improves the trigger handling for distributed hypertables, adds some more randomness to chunk assignment to avoid thundering herd issues in chunk assignment, and fixes some issues in update handling as well as some other bugs. **Bugfixes** * timescale#3279 Add some more randomness to chunk assignment * timescale#3288 Fix failed update with parallel workers * timescale#3300 Improve trigger handling on distributed hypertables * timescale#3304 Remove paths that reference parent relids for compressed chunks * timescale#3305 Fix pull_varnos miscomputation of relids set * timescale#3310 Generate downgrade script * timescale#3314 Fix heap buffer overflow in hypertable expansion * timescale#3317 Fix heap buffer overflow in remote connection cache. * timescale#3327 Make aggregate in caggs fully qualified * timescale#3327 Make aggregates in caggs fully qualified * timescale#3336 Fix pg_init_privs objsubid handling * timescale#3345 Fix SkipScan distinct column identification * timescale#3355 Fix heap buffer overflow when renaming compressed hypertable columns. * timescale#3367 Improve DecompressChunk qual pushdown * timescale#3377 Fix bad use of repalloc **Thanks** * @db-adrian for reporting an issue when accessing cagg view through postgres_fdw * @fncaldas and @pgwhalen for reporting an issue accessing caggs when public is not in search_path * @fvannee, @mglonnro and @ebreijo for reporting an issue with the upgrade script * @fvannee for reporting a performance regression with SkipScan
This maintenance release contains bugfixes since the 2.3.0 release. We deem it moderate priority for upgrading. The release introduces the possibility of generating downgrade scripts, improves the trigger handling for distributed hypertables, adds some more randomness to chunk assignment to avoid thundering herd issues in chunk assignment, and fixes some issues in update handling as well as some other bugs. **Bugfixes** * timescale#3279 Add some more randomness to chunk assignment * timescale#3288 Fix failed update with parallel workers * timescale#3300 Improve trigger handling on distributed hypertables * timescale#3304 Remove paths that reference parent relids for compressed chunks * timescale#3305 Fix pull_varnos miscomputation of relids set * timescale#3310 Generate downgrade script * timescale#3314 Fix heap buffer overflow in hypertable expansion * timescale#3317 Fix heap buffer overflow in remote connection cache. * timescale#3327 Make aggregate in caggs fully qualified * timescale#3327 Make aggregates in caggs fully qualified * timescale#3336 Fix pg_init_privs objsubid handling * timescale#3345 Fix SkipScan distinct column identification * timescale#3355 Fix heap buffer overflow when renaming compressed hypertable columns. * timescale#3367 Improve DecompressChunk qual pushdown * timescale#3377 Fix bad use of repalloc **Thanks** * @db-adrian for reporting an issue when accessing cagg view through postgres_fdw * @fncaldas and @pgwhalen for reporting an issue accessing caggs when public is not in search_path * @fvannee, @mglonnro and @ebreijo for reporting an issue with the upgrade script * @fvannee for reporting a performance regression with SkipScan
This maintenance release contains bugfixes since the 2.3.0 release. We deem it moderate priority for upgrading. The release introduces the possibility of generating downgrade scripts, improves the trigger handling for distributed hypertables, adds some more randomness to chunk assignment to avoid thundering herd issues in chunk assignment, and fixes some issues in update handling as well as some other bugs. **Bugfixes** * #3279 Add some more randomness to chunk assignment * #3288 Fix failed update with parallel workers * #3300 Improve trigger handling on distributed hypertables * #3304 Remove paths that reference parent relids for compressed chunks * #3305 Fix pull_varnos miscomputation of relids set * #3310 Generate downgrade script * #3314 Fix heap buffer overflow in hypertable expansion * #3317 Fix heap buffer overflow in remote connection cache. * #3327 Make aggregate in caggs fully qualified * #3327 Make aggregates in caggs fully qualified * #3336 Fix pg_init_privs objsubid handling * #3345 Fix SkipScan distinct column identification * #3355 Fix heap buffer overflow when renaming compressed hypertable columns. * #3367 Improve DecompressChunk qual pushdown * #3377 Fix bad use of repalloc **Thanks** * @db-adrian for reporting an issue when accessing cagg view through postgres_fdw * @fncaldas and @pgwhalen for reporting an issue accessing caggs when public is not in search_path * @fvannee, @mglonnro and @ebreijo for reporting an issue with the upgrade script * @fvannee for reporting a performance regression with SkipScan
@mkindahl - I was wondering why your commits referenced this issue. Are you guys working on support for multiple hypertables in continuous aggregates? |
@kramer65 There are architectural reasons to why we cannot support multiple hypertables in a query, but we are considering ways to either work around it or re-design it. The reference from the release is a mistake by GitHub: we added a pull request with the same number as this issue and GitHub mistook it for a issue reference. |
@mkindahl - Thanks for your response. I understand it's hard to give a timeline, but how big would you say is the chance that this will be implemented in 2022? I'm currently in the process of designing the basis for a new data platform. A big part of it is the ingress, for which we're going to use timescale. However the part in which we're going to make aggregations on a continuous basis we need to find a solution. If timescale would implement this within half a year we'll probably use that. If it takes longer we need to decide whether we're going find something else or build something homegrown. |
Actually the PR the release notes should reference is |
Aha, that explains it! Thanks for explaining how GitHub implements the references, I was wondering about this. |
I think |
I don't think these issues are the same. This issue is about multiple hypertables. #1446 is about multiple tables (e.g. a lookup table). However the workaround I provided does work for both. |
I know it might not be quite what's asked here, but wanted to share that the team has been working on supporting continuous aggregates that support JOINing a hypertable against a standard PG table. Hope to release this coming quarter (Q1 2023). => #4874 |
We've got a materialized view which joins multiple hypertables. Refreshing that view takes a lot of time, so I wanted to convert it to a continuous aggregate, but got this:
So I was planning on building my own continuous aggregate by simply creating a new table and calculate/append the new records every 10 min or so. A colleague suggested that maybe you guys had good reasons for not permitting a continuous aggregate from multiple hypertables.
So my question; why does timescale not allow for continuous aggregates from multiple hypertables?
The text was updated successfully, but these errors were encountered: