Skip to content

perf(tuned): activate and configure zswap for postgresql tuned profile#2077

Merged
samrose merged 20 commits intodevelopfrom
INDATA-488
Mar 6, 2026
Merged

perf(tuned): activate and configure zswap for postgresql tuned profile#2077
samrose merged 20 commits intodevelopfrom
INDATA-488

Conversation

@hunleyd
Copy link
Copy Markdown
Contributor

@hunleyd hunleyd commented Mar 2, 2026

This change enables zswap with the zsmalloc allocator and zstd compression within the postgresql tuned profile to improve system responsiveness and database performance under memory pressure.

Technical Overview: What is zswap?

zswap is a Linux kernel feature that provides a compressed write-back cache for swapped pages. Instead of immediately moving pages from RAM to the physical swap device (HDD/SSD) when memory is full, zswap intercepts them, compresses them, and stores them in a dynamically allocated RAM pool. This effectively increases the "perceived" amount of RAM available to the system.

Why zsmalloc over zbud?

We have selected 'zsmalloc' as the zpool allocator instead of 'zbud' for several reasons:

  • Density: zsmalloc is a slab-based allocator that can pack many compressed objects into a single physical page, whereas zbud is hard-coded to a maximum of 2 objects per page (2:1 ratio).
  • Efficiency: On modern kernels (Linux 6.3+), zsmalloc supports proper eviction to disk, removing the primary historical advantage of zbud. It provides significantly better memory utilization and higher compression density.

Why zstd over lz4?

While lz4 offers faster compression/decompression speeds, we have opted for 'zstd' for the following reasons:

  • Compression Ratio: zstd provides a substantially higher compression ratio than lz4. In database workloads where memory density is critical, fitting more pages into the zswap pool is more beneficial than the marginal speed gains of lz4.
  • Balance: zstd offers a superior balance between CPU overhead and compression efficiency, especially for the 4KB-8KB pages typically handled by the kernel and PostgreSQL.

Benefits for PostgreSQL Performance

Implementing zswap is particularly beneficial for PostgreSQL workloads:

  • Mitigation of Swap Thrashing: PostgreSQL performance degrades exponentially when active buffers are swapped to disk. zswap ensures that these pages stay in RAM (compressed), allowing "major page faults" to be resolved at RAM speeds rather than waiting for disk I/O.
  • I/O Latency Stability: By reducing the frequency of physical writes to the swap partition, zswap prevents I/O contention between the kernel's swap subsystem and PostgreSQL's own WAL and data writes.
  • Consistent Query Latency: Queries that access less-frequently used data that has been compressed into zswap will return much faster than if they had to be fetched from physical swap, leading to more predictable p99 latencies.

This change enables zswap with the zsmalloc allocator and zstd compression
within the postgresql tuned profile to improve system responsiveness and
database performance under memory pressure.

### Technical Overview: What is zswap?
zswap is a Linux kernel feature that provides a compressed write-back cache
for swapped pages. Instead of immediately moving pages from RAM to the
physical swap device (HDD/SSD) when memory is full, zswap intercepts them,
compresses them, and stores them in a dynamically allocated RAM pool.
This effectively increases the "perceived" amount of RAM available to
the system.

### Why zsmalloc over zbud?
We have selected 'zsmalloc' as the zpool allocator instead of 'zbud' for several reasons:
- Density: zsmalloc is a slab-based allocator that can pack many
  compressed objects into a single physical page, whereas zbud is
  hard-coded to a maximum of 2 objects per page (2:1 ratio).
- Efficiency: On modern kernels (Linux 6.3+), zsmalloc supports
  proper eviction to disk, removing the primary historical advantage
  of zbud. It provides significantly better memory utilization and
  higher compression density.

### Why zstd over lz4?
While lz4 offers faster compression/decompression speeds, we have
opted for 'zstd' for the following reasons:
- Compression Ratio: zstd provides a substantially higher compression
  ratio than lz4. In database workloads where memory density is
  critical, fitting more pages into the zswap pool is more
  beneficial than the marginal speed gains of lz4.
- Balance: zstd offers a superior balance between CPU overhead and
  compression efficiency, especially for the 4KB-8KB pages
  typically handled by the kernel and PostgreSQL.

### Benefits for PostgreSQL Performance
Implementing zswap is particularly beneficial for PostgreSQL workloads:
- Mitigation of Swap Thrashing: PostgreSQL performance degrades
  exponentially when active buffers are swapped to disk. zswap
  ensures that these pages stay in RAM (compressed), allowing
  "major page faults" to be resolved at RAM speeds rather than
  waiting for disk I/O.
- I/O Latency Stability: By reducing the frequency of physical
  writes to the swap partition, zswap prevents I/O contention
  between the kernel's swap subsystem and PostgreSQL's own WAL
  and data writes.
- Consistent Query Latency: Queries that access less-frequently
  used data that has been compressed into zswap will return much
  faster than if they had to be fetched from physical swap,
  leading to more predictable p99 latencies.
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR enables zswap with the zsmalloc allocator and zstd compressor for the PostgreSQL tuned profile, aiming to improve system responsiveness and reduce swap I/O pressure under memory-intensive PostgreSQL workloads.

Changes:

  • Adds a new community.general.ini_file task to write a zswap configuration entry into the tuned postgresql profile's tuned.conf.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread ansible/tasks/setup-tuned.yml Outdated
Comment thread ansible/tasks/setup-tuned.yml Outdated
hunleyd and others added 4 commits March 2, 2026 12:32
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
…-488

* 'INDATA-488' of github.com:supabase/postgres:
  Update ansible/tasks/setup-tuned.yml
  Update ansible/tasks/setup-tuned.yml
Comment thread ansible/tasks/setup-tuned.yml Outdated
@hunleyd
Copy link
Copy Markdown
Contributor Author

hunleyd commented Mar 2, 2026

@samrose i do not like having to add that when, but the runners don't have the module. open to ideas

@hunleyd
Copy link
Copy Markdown
Contributor Author

hunleyd commented Mar 4, 2026

============================================================
Test Summary
============================================================

ENGINE          RESULT
------          ------
15              PASSED (397s)
17              PASSED (3839s)
17-oriole       PASSED (914s)

============================================================
Total: 3 | Passed: 3 | Failed: 0
============================================================

[INFO] Postgres commit: 420197266c6095c95f198651b50dc944a182a6bc
[INFO] Cache location: /Users/jimchancojr/.cache/supadev/postgres/420197266c6095c95f198651b50dc944a182a6bc
[SUCCESS] All tests passed!

@hunleyd hunleyd marked this pull request as ready for review March 4, 2026 18:08
@hunleyd hunleyd requested review from a team as code owners March 4, 2026 18:08
Copy link
Copy Markdown
Contributor

@tgallacher tgallacher left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/lgtm

Copy link
Copy Markdown
Collaborator

@samrose samrose left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

need to discuss and review before merge

@hunleyd hunleyd requested a review from samrose March 5, 2026 16:14
Copy link
Copy Markdown
Collaborator

@samrose samrose left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just need to address accurate increment of version. Or it will have to be done over again when you merge, and the release will fail

Comment thread ansible/vars.yml Outdated
@samrose
Copy link
Copy Markdown
Collaborator

samrose commented Mar 6, 2026

@samrose samrose enabled auto-merge March 6, 2026 03:36
@samrose samrose added this pull request to the merge queue Mar 6, 2026
Merged via the queue into develop with commit 702f91d Mar 6, 2026
37 checks passed
@samrose samrose deleted the INDATA-488 branch March 6, 2026 04:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants