Release 2.16.0

[pallavisontakke]

This release contains performance improvements and bug fixes since the 2.15.3 release. We recommend that you upgrade at the next available opportunity.

Features

• Add support for array operations to compressed DML #6880: Add support for the array operators used for compressed DML batch filtering.
• Improve compressed DML expression pushdown #6895: Improve the compressed DML expression pushdown.
• Allow replica identity on compressed hypertables #6897: Add support for replica identity on compressed hypertables.
• Remove PG13 support #6918: Remove support for PG13.
• Rework compression activity wal markers #6920: Rework compression activity wal markers.
• Add support for foreign keys to hypertables #6989: Add support for foreign keys when converting plain tables to hypertables.
• Add support for chunk column statistics tracking #7020: Add support for the chunk column statistics tracking.
• Add index scan to INSERT DML decompression #7048: Add an index scan for INSERT DML decompression.
• Reduce decompression on compressed INSERT #7075: Reduce decompression on the compressed INSERT.
• Reduce decompressions for compressed UPDATE/DELETE #7101: Reduce decompressions for the compressed UPDATE/DELETE.
• Reduce decompression for INSERT with unique constraints #7108 Reduce decompressions for INSERTs with UNIQUE constraints

Bugfixes

• Fix search_path quoting in compression defaults function #7018: Fix search_path quoting in the compression defaults function.
• Prevent locking compressed tuples #7046: Prevent locking for compressed tuples.
• Fix scankey where consttype is different from var #7055: Fix the scankey for segment by columns, where the type constant is different to variable.
• Fix a bug in default orderby calc. in compression #7064: Fix the bug in the default order by calculation in compression.
• Don't use index column names #7069: Fix the index column name usage.
• Fix bug in default segmentby calc. in compression #7074: Fix the bug in the default segment by calculation in compression.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 81.78%. Comparing base (59f50f2) to head (f492bd1).
Report is 257 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #7135      +/-   ##
==========================================
+ Coverage   80.06%   81.78%   +1.71%     
==========================================
  Files         190      203      +13     
  Lines       37181    38035     +854     
  Branches     9450     9872     +422     
==========================================
+ Hits        29770    31106    +1336     
+ Misses       2997     2965      -32     
+ Partials     4414     3964     -450     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[iroussos]

Suggested change

	With this release we introduce a lot of performance focused optimizations for DML over compressed chunks.
	We also speed up those tricky queries that end up accessing more chunks than needed, by extending chunk
	exclusion on compressed hypertables to take into account filters on additional, non partitioning columns.
	We also introduce new options for Timescale users that like their Foreign Keys, allowing for FKs from regular tables towards hypertables, but also removing some really annoying locks in the reverse direction that blocked access to referenced tables while compression was running.
	Finally, big updates on the Continuous Aggregates front - more types of joins supported, additional equality operators on join clauses and support for joining with multiple regular tables.
	**Highlighted features in this release**
	* Improved query performance through chunk exclusion on compressed hypertables by using additional columns:
	You can now define sparse indexes on compressed chunks for any column with
	an integer data type (smallint, int, bigint, serial, bigserial, date, timestamp, timestamptz).
	After enabling this feature for a column using the `enable_column_stats` command, Timescale will track min/max
	values for that column and use that information to exclude chunks for queries that filter by that column and would
	not find any data on those chunks.
	* Improved upsert performance by using index scans to verify constraints during inserts on compressed chunks.
	This improvement can speed up some ON CONFLICT clauses by more than 100x.
	* Improved performance of updates and deletes on all types of compressed chunks, as well as inserts into compressed
	chunks with unique constraints by filtering data while accessing the compressed data and before decompressing.
	This improvement can speedup significantly a lot of types of DML commands by signaling constraint violations without decompressing or decompressing only when matching records are found in the case of updates, deletes and upserts.
	* New ability to add Foreign Keys from regular (non hyper) tables to hypertables, with all types of cascading options supported.
	Really useful for hypertables that partition using sequential IDs, but also for some time series use cases.
	* Advanced handling of Foreign Keys on compressed chunks do eliminate locking of referenced tables when new
	chunks are compressed. No more DML getting blocked on referenced tables while compression runs on a hypertable.
	* Improved support for queries on Continuous Aggregates - more types of joins supported (INNER/LEFT and LATERAL joins),
	more than one equality operator on join clauses allowed and support for joining with multiple regular tables.
	**PostgreSQL 13 support removal announcement**
	Following the deprecation announcement for PostgreSQL 13 in TimescaleDB 2.13,
	PostgreSQL 13 is not supported starting with TimescaleDB 2.16.
	Currently supported PostgreSQL major versions are 14, 15 and 16.

[iroussos]

@svenklemm @antekresic @fabriziomello can you check and let me know if this looks good & whether I missed anything?

@pallavisontakke I am missing more than a few things that are in the milestone from the description and the list that follows. Like for example the PR by Fabrizio on the CAggs improvements and various other PRs above 7100

@sven check what I wrote about FKs - I am assuming that this will make it for the release. Otherwise, we should remove