Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: CAGG chunk interval resolves to source_table interval multiplied by 10 #5382

Closed
noctarius opened this issue Mar 2, 2023 · 1 comment · Fixed by #5384
Closed

[Bug]: CAGG chunk interval resolves to source_table interval multiplied by 10 #5382

noctarius opened this issue Mar 2, 2023 · 1 comment · Fixed by #5384

Comments

@noctarius
Copy link
Contributor

What type of bug is this?

Performance issue, Unexpected error

What subsystems and features are affected?

Continuous aggregate

What happened?

When creating an continuous aggregate, the chunk interval taken for it is the chunk interval of the source table multiplied by 10, which increases way to fast. That is true for cagg on hypertable and cagg on cagg.

Example:
raw hypertable chunk interval: 7 days
cagg on hypertable: 70 days
cagg on cagg: 700 days

Screenshot 2023-03-02 at 15 34 30

TimescaleDB version affected

2.9.3

PostgreSQL version used

14.7

What operating system did you use?

Cloud

What installation method did you use?

Not applicable

What platform did you run on?

Timescale Cloud

Relevant log output and stack trace

tsdb=> create table test (
    ts timestamptz primary key,
    value int not null
);
CREATE TABLE

tsdb=> select create_hypertable('test', 'ts');
 create_hypertable
--------------------
 (43,public,test,t)
(1 row)

tsdb=> create materialized view test1
    with (timescaledb.continuous) as
    select
        time_bucket('10 min', ts) bucket,
        avg(value) value
    from test
    group by 1;
NOTICE:  continuous aggregate "test1" is already up-to-date
CREATE MATERIALIZED VIEW

tsdb=> create materialized view test2
    with (timescaledb.continuous) as
    select
        time_bucket('1 h', bucket) bucket,
        avg(value) value
    from test1
    group by 1;
NOTICE:  continuous aggregate "test2" is already up-to-date
CREATE MATERIALIZED VIEW

tsdb=> select h.table_name name, _timescaledb_internal.to_interval(d.interval_length)
from _timescaledb_catalog.hypertable h
left join _timescaledb_catalog.dimension d on d.hypertable_id = h.id
where h.table_name = 'test'
union all
select c.user_view_name name, _timescaledb_internal.to_interval(d.interval_length)
from _timescaledb_catalog.continuous_agg c
left join _timescaledb_catalog.dimension d on d.hypertable_id = c.mat_hypertable_id
where c.user_view_name = 'test1'
union all
select c.user_view_name name, _timescaledb_internal.to_interval(d.interval_length)
from _timescaledb_catalog.continuous_agg c
left join _timescaledb_catalog.dimension d on d.hypertable_id = c.mat_hypertable_id
where c.user_view_name = 'test2';
 name  | to_interval
-------+-------------
 test  | 7 days
 test1 | 70 days
 test2 | 700 days
(3 rows)


### How can we reproduce the bug?

```bash
create table test (
    ts timestamptz primary key,
    value int not null
);

select create_hypertable('test', 'ts');

create materialized view test1
    with (timescaledb.continuous) as
    select
        time_bucket('10 min', ts) bucket,
        avg(value) value
    from test
    group by 1;

create materialized view test2
    with (timescaledb.continuous) as
    select
        time_bucket('1 h', bucket) bucket,
        avg(value) value
    from test1
    group by 1;

select h.table_name name, _timescaledb_internal.to_interval(d.interval_length)
from _timescaledb_catalog.hypertable h
left join _timescaledb_catalog.dimension d on d.hypertable_id = h.id
where h.table_name = 'test'
union all
select c.user_view_name name, _timescaledb_internal.to_interval(d.interval_length)
from _timescaledb_catalog.continuous_agg c
left join _timescaledb_catalog.dimension d on d.hypertable_id = c.mat_hypertable_id
where c.user_view_name = 'test1'
union all
select c.user_view_name name, _timescaledb_internal.to_interval(d.interval_length)
from _timescaledb_catalog.continuous_agg c
left join _timescaledb_catalog.dimension d on d.hypertable_id = c.mat_hypertable_id
where c.user_view_name = 'test2';
@noctarius noctarius added the bug label Mar 2, 2023
@jnidzwetzki
Copy link
Contributor

Hello @noctarius,

Thank you for the detailed steps to reproduce the problem. I could see the same behavior in TimescaleDB 2.10.0 and PostgreSQL 14. The interval_length for the hybertable / caggs is different in the catalog table.

test2=# select * from _timescaledb_catalog.dimension;
 id | hypertable_id | column_name |       column_type        | aligned | num_slices | partitioning_func_schema | partitioning_func | interval_length | compress_interv
al_length | integer_now_func_schema | integer_now_func 
----+---------------+-------------+--------------------------+---------+------------+--------------------------+-------------------+-----------------+----------------
----------+-------------------------+------------------
  1 |             1 | ts          | timestamp with time zone | t       |            |                          |                   |    604800000000 |                
          |                         | 
  2 |             2 | bucket      | timestamp with time zone | t       |            |                          |                   |   6048000000000 |                
          |                         | 
  3 |             3 | bucket      | timestamp with time zone | t       |            |                          |                   |  60480000000000 |                
          |                         | 
(3 rows)

@fabriziomello fabriziomello self-assigned this Mar 2, 2023
fabriziomello added a commit to fabriziomello/timescaledb that referenced this issue Mar 2, 2023
When a Continuous Aggregate is created the `chunk_interval_size` is
defined my the `chunk_interval_size` of the original hypertable
multiplied by a fixed factor of 10.

The problem is currently when we create a Hierarchical Continuous
Aggregate the same factor is applied and it lead to an exponential
`chunk_interval_size`.

Fixed it by just copying the `chunk_interval_size` from the base
Continuous Aggregate for an Hierachical Continuous Aggreagate.

Fixes timescale#5382
fabriziomello added a commit to fabriziomello/timescaledb that referenced this issue Mar 2, 2023
When a Continuous Aggregate is created the `chunk_interval_size` is
defined my the `chunk_interval_size` of the original hypertable
multiplied by a fixed factor of 10.

The problem is currently when we create a Hierarchical Continuous
Aggregate the same factor is applied and it lead to an exponential
`chunk_interval_size`.

Fixed it by just copying the `chunk_interval_size` from the base
Continuous Aggregate for an Hierachical Continuous Aggreagate.

Fixes timescale#5382
@svenklemm svenklemm modified the milestone: TimescaleDB 2.10.1 Mar 3, 2023
fabriziomello added a commit to fabriziomello/timescaledb that referenced this issue Mar 3, 2023
When a Continuous Aggregate is created the `chunk_interval_size` is
defined my the `chunk_interval_size` of the original hypertable
multiplied by a fixed factor of 10.

The problem is currently when we create a Hierarchical Continuous
Aggregate the same factor is applied and it lead to an exponential
`chunk_interval_size`.

Fixed it by just copying the `chunk_interval_size` from the base
Continuous Aggregate for an Hierachical Continuous Aggreagate.

Fixes timescale#5382
fabriziomello added a commit to fabriziomello/timescaledb that referenced this issue Mar 3, 2023
When a Continuous Aggregate is created the `chunk_interval_size` is
defined my the `chunk_interval_size` of the original hypertable
multiplied by a fixed factor of 10.

The problem is currently when we create a Hierarchical Continuous
Aggregate the same factor is applied and it lead to an exponential
`chunk_interval_size`.

Fixed it by just copying the `chunk_interval_size` from the base
Continuous Aggregate for an Hierachical Continuous Aggreagate.

Fixes timescale#5382
fabriziomello added a commit to fabriziomello/timescaledb that referenced this issue Mar 3, 2023
When a Continuous Aggregate is created the `chunk_interval_size` is
defined my the `chunk_interval_size` of the original hypertable
multiplied by a fixed factor of 10.

The problem is currently when we create a Hierarchical Continuous
Aggregate the same factor is applied and it lead to an exponential
`chunk_interval_size`.

Fixed it by just copying the `chunk_interval_size` from the base
Continuous Aggregate for an Hierachical Continuous Aggreagate.

Fixes timescale#5382
fabriziomello added a commit to fabriziomello/timescaledb that referenced this issue Mar 3, 2023
When a Continuous Aggregate is created the `chunk_interval_size` is
defined my the `chunk_interval_size` of the original hypertable
multiplied by a fixed factor of 10.

The problem is currently when we create a Hierarchical Continuous
Aggregate the same factor is applied and it lead to an exponential
`chunk_interval_size`.

Fixed it by just copying the `chunk_interval_size` from the base
Continuous Aggregate for an Hierachical Continuous Aggreagate.

Fixes timescale#5382
fabriziomello added a commit that referenced this issue Mar 3, 2023
When a Continuous Aggregate is created the `chunk_interval_size` is
defined my the `chunk_interval_size` of the original hypertable
multiplied by a fixed factor of 10.

The problem is currently when we create a Hierarchical Continuous
Aggregate the same factor is applied and it lead to an exponential
`chunk_interval_size`.

Fixed it by just copying the `chunk_interval_size` from the base
Continuous Aggregate for an Hierachical Continuous Aggreagate.

Fixes #5382
github-actions bot pushed a commit that referenced this issue Mar 3, 2023
When a Continuous Aggregate is created the `chunk_interval_size` is
defined my the `chunk_interval_size` of the original hypertable
multiplied by a fixed factor of 10.

The problem is currently when we create a Hierarchical Continuous
Aggregate the same factor is applied and it lead to an exponential
`chunk_interval_size`.

Fixed it by just copying the `chunk_interval_size` from the base
Continuous Aggregate for an Hierachical Continuous Aggreagate.

Fixes #5382

(cherry picked from commit 3204683)
timescale-automation pushed a commit that referenced this issue Mar 4, 2023
When a Continuous Aggregate is created the `chunk_interval_size` is
defined my the `chunk_interval_size` of the original hypertable
multiplied by a fixed factor of 10.

The problem is currently when we create a Hierarchical Continuous
Aggregate the same factor is applied and it lead to an exponential
`chunk_interval_size`.

Fixed it by just copying the `chunk_interval_size` from the base
Continuous Aggregate for an Hierachical Continuous Aggreagate.

Fixes #5382

(cherry picked from commit 3204683)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants