[Bug]: INSERT into compressed chunks does not clean dead tuples

[svenklemm]

What type of bug is this?

Performance issue

What subsystems and features are affected?

Compression

What happened?

We have a "storage leak" in compression, more specifically the insert into compressed chunks feature. The leak is due to the way we disable vacuum on these chunks leading to dead tuples never getting cleaned up. To detect these simply checking n_dead_tup is not enough because it will never be updated, you need to explicitly ANALYZE the chunk and check n_dead_tup after. This is also a performance issue as all those dead tuples have to be scanned when scanning the table.

TimescaleDB version affected

2.3+

PostgreSQL version used

14.4

What operating system did you use?

Archlinux 2022.07.01 x64

What installation method did you use?

Source

What platform did you run on?

On prem/Self-hosted

Relevant log output and stack trace

SELECT n_dead_tup from pg_stat_user_tables where relid=<chunk with compression>;
-[ RECORD 1 ]----------
n_dead_tup | 1363992372

[svenklemm]

Seems like this is only a problem when the job errors. Couldn't reproduce it in normal operation.

[antekresic]

Fixed by #5118