Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Undo unlogged build v16 #413

Closed
wants to merge 28 commits into from
Closed

Conversation

knizhnik
Copy link
Contributor

Some Postgres indexes (GIN,GIST,SPGIST...) are using two-phase build: at first phase relation pages are constructed and at second phase - all relation is wal-logged. It doesn't work with Neon because if dirty page was thrown away from shared buffer before been wal-logged, then its content will be lost.

We have added support of unlogged builds to SMGR API. But it requires changes in Postgre core. What is even worser some extensions (i.e. pgvector) are also using the same policy and have to be patched.

This PR tries to avoid changes in Postgres core and did it at Neon extension level.

Matthias van de Meent and others added 27 commits February 6, 2024 13:06
This prepares PostgreSQL for compatibility with Neon's storage.
Significant changes compared to the PostgreSQL 15 patchset include:

- Backported changes for users and roles are no longer required
- Use RM_NEON_ID for changes in WAL, instead of modifying core WAL records
Co-authored-by: Konstantin Knizhnik <knizhnik@neon.tech>
* Neon logical replication support for PG16

* Log heap rewrite file after creation.

---------

Co-authored-by: Konstantin Knizhnik <knizhnik@neon.tech>
Co-authored-by: Arseny Sher <sher-ars@yandex.ru>
Co-authored-by: Konstantin Knizhnik <knizhnik@neon.tech>
* Update WAL buffers when restoring WAL at compute needed for LR

* Fix copying data in WAL buffers

---------

Co-authored-by: Konstantin Knizhnik <knizhnik@neon.tech>
PG16 adds new function to SMGR: zeroextend
It's implementation in Neon actually wal-log zero pages of extended relation.
This zero page is wal-logged using XLOG_FPI.
As far as page is zero, the hole optimization (excluding from the image everything between pg_upper and pd_lower) doesn't work.

This PR allows to set hole size to BLCKSZ in case of zero page (PageIsNull() returns true).
---------

Co-authored-by: Konstantin Knizhnik <knizhnik@neon.tech>
* Prevent output callbacks from hearing about neon-file messages
* Load SLRU segments on demand

refer #8763

* Fix errors in downloading SLRU segments

* Fix build problems

* Undo occcasional changes

* Remove unintenmded changes

* Fix smgr_read_slru_segment

* Determine SLRU kind in extension

* Use ctl->PagePrecedes for SLRU page comparison in SimpleLruDownloadSegment to address wraparround

---------

Co-authored-by: Konstantin Knizhnik <knizhnik@neon.tech>
Co-authored-by: Konstantin Knizhnik <knizhnik@neon.tech>
…mary is not alive (#363)

* Set wasShutdown=true during hot-standby replica startup only when primary is not alive
* Report fatal error if hot standaby replica is started with oldestAcriveXid=0

Postgres part of neondatabase/neon#6705
---------

Co-authored-by: Konstantin Knizhnik <knizhnik@neon.tech>
…d for oldestActiveXid while replica startup (#387)

Co-authored-by: Konstantin Knizhnik <knizhnik@neon.tech>
Co-authored-by: Konstantin Knizhnik <knizhnik@neon.tech>
This keeps the walproposer processes alive at shutdown, until after
the shutdown checkpoint has been written. That gives the walproposers
a chance to stream it to the safekeepers.
…on' error message (#390)

Co-authored-by: Konstantin Knizhnik <knizhnik@neon.tech>
This is done in other Postgres headers.
Co-authored-by: Konstantin Knizhnik <knizhnik@neon.tech>
* Revert "Add comment explaining why it is safe to use FirstNormalTransactionXid for oldestActiveXid while replica startup (#387)"

This reverts commit 67240b5.

* Revert "Set wasShutdown=true during hot-standby replica startup only when primary is not alive (#363)"

This reverts commit 61c280c.
* Remember last written LSN when it is first requested

* Return updated LSN in SetLastWrittenLSN

---------

Co-authored-by: Konstantin Knizhnik <knizhnik@neon.tech>
@knizhnik knizhnik changed the base branch from REL_15_STABLE_neon to REL_16_STABLE_neon April 20, 2024 06:29
@knizhnik
Copy link
Contributor Author

knizhnik commented Jun 3, 2024

Replaced by #435

@knizhnik knizhnik closed this Jun 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants