Skip to content

Conversation

@knizhnik
Copy link

@knizhnik knizhnik commented Feb 1, 2025

Make it possible to dump WAL records in format recognised by walredo process.
Intended usage:

pg_waldump -R 1663/5/16396  -B 771727 000000010000000100000034 --save-records=/tmp/walredo.records
postgres --wal-redo < /tmp/walredo.records > /tmp/page.img

Copy link
Contributor

@hlinnaka hlinnaka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add a brief comment explaining the file format

@knizhnik
Copy link
Author

knizhnik commented Feb 5, 2025

Please add a brief comment explaining the file format

Done

Copy link

@problame problame left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't judge the C code, but, as someone who has been commanding the incident during which this was developed: it was a helpful tool.

Would be nice to have a tiny test for this feature so it doesn't bit-rot.

Ideally we'd test that this tool can be used instead of get_values_reconstruct_data in the pageserver / that they always return both results.

Realistically, let's just have a tiny test that points the tool to some Safekeeper WAL and verify that

  1. the exit status is 0 when invoked for some page / LSN range
  2. when piping generated wal stream into walredo process, we get the same page image that we get from the pageserver (use this testing API to get the page image that pageserver returns)

@knizhnik knizhnik force-pushed the pg_waldump_save_records branch from d7514ab to e5fcb13 Compare February 10, 2025 12:53
Co-authored-by: Heikki Linnakangas <heikki@neon.tech>
github-merge-queue bot pushed a commit to neondatabase/neon that referenced this pull request Feb 10, 2025
## Problem

Make it possible to dump WAL records in format recognised by walredo
process.
Intended usage:

```
pg_waldump -R 1663/5/16396  -B 771727 000000010000000100000034 --save-records=/tmp/walredo.records
postgres --wal-redo < /tmp/walredo.records > /tmp/page.img
```

## Summary of changes

Related Postgres PRs:
neondatabase/postgres#575
neondatabase/postgres#572

---------

Co-authored-by: Konstantin Knizhnik <knizhnik@neon.tech>
@knizhnik knizhnik merged commit 4c45d78 into REL_17_STABLE_neon Feb 10, 2025
1 check passed
@knizhnik knizhnik deleted the pg_waldump_save_records branch February 10, 2025 15:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants