Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add catchup commands family #455

Merged
merged 9 commits into from Jan 21, 2020
Merged

Add catchup commands family #455

merged 9 commits into from Jan 21, 2020

Conversation

defntvdm
Copy link
Contributor

@defntvdm defntvdm commented Oct 23, 2019

#363 solution

@defntvdm defntvdm changed the title WIP: catchup #363 WIP: catchup Oct 25, 2019
@defntvdm defntvdm changed the title WIP: catchup Add catchup commands family Dec 18, 2019
Copy link
Collaborator

@g0djan g0djan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually we have the main scenario to test this feature.
First

  1. Run HA cluster - 1 master and 1 replica.
  2. Run pgbench to fill cluster.
  3. After that block streaming WAL to replica by iptables(firstly check taht it's possible)
  4. Run pgbench again, after that master will have more data than replica
  5. Catchup-push from master PGDATA and from replica LSN.
  6. Replica catchup-fetch this incremental-backup
  7. Compare that master and replica has the same data.

P.S. It would be better to create several incremental backups which catchup-fetch will apply

cmd/pg/catchup_fetch.go Outdated Show resolved Hide resolved
cmd/pg/catchup_push.go Outdated Show resolved Hide resolved
docker/pg_tests/Dockerfile Outdated Show resolved Hide resolved
docker/pg_tests/Dockerfile_catchup_perftest Outdated Show resolved Hide resolved
docker/pg_tests/scripts/tests/catchup_perftest.sh Outdated Show resolved Hide resolved
docker/pg_tests/scripts/tests/catchup_perftest.sh Outdated Show resolved Hide resolved
internal/backup.go Outdated Show resolved Hide resolved
internal/backup_fetch_handler.go Outdated Show resolved Hide resolved
internal/catchup_fetch_handler.go Outdated Show resolved Hide resolved
internal/catchup_push_handler.go Outdated Show resolved Hide resolved
docker/pg_tests/scripts/tests/catchup_perftest.sh Outdated Show resolved Hide resolved
internal/backup_push_handler.go Outdated Show resolved Hide resolved
@defntvdm defntvdm requested a review from g0djan January 13, 2020 10:41
@@ -53,7 +53,8 @@ pgbench -i -s 100 -h 127.0.0.1 -p ${ALPHA_PORT} postgres

LSN=`psql -c "SELECT pg_current_wal_lsn() - '0/0'::pg_lsn;" | grep -E '[0-9]+' | head -1`

/tmp/scripts/wait_while_replication_complete.sh postgres pgbench_accounts ${BETA_PORT}
# db table conn_port row_count
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool Idea!

Copy link
Contributor

@dimastark dimastark left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Go imports 🎉, Go fmt 🎉, Go modules 🎉

@g0djan g0djan merged commit d794619 into wal-g:master Jan 21, 2020
@defntvdm defntvdm deleted the pg363 branch May 12, 2020 21:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants