Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DROP MIRROR support for PG, BQ and SF #93

Merged
merged 4 commits into from
Jun 5, 2023
Merged

DROP MIRROR support for PG, BQ and SF #93

merged 4 commits into from
Jun 5, 2023

Conversation

heavycrystal
Copy link
Contributor

This PR adds support for dropping mirrors, via the custom DROP MIRROR SQL command. Dropping a mirror requires handling the following:

  1. Terminating the existing mirror workflow [PeerFlow] by signalling a shutdown via Temporal.
  2. Execute a new workflow [DropFlow] that retrieves connector information, and executes cleanup on both the source and destination peers via new connector functions. From the source side [Postgres], this involves deleting the replication slot and publication. From the destinationn side [Snowflake/BigQuery], this involves deleting the PeerDB specific tables and also deleting job metadata from the PEERDB_MIRROR_JOBS table.
  3. Deleting metadata from the FLOWS table in the PeerDB catalog.

if err != nil {
return fmt.Errorf("error starting transaction for flow cleanup: %w", err)
}
defer pullFlowCleanupTx.Rollback(c.ctx)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚫 [golangci] reported by reviewdog 🐶
Error return value of pullFlowCleanupTx.Rollback is not checked (errcheck)

if err != nil {
return fmt.Errorf("unable to begin transaction for sync flow cleanup: %w", err)
}
defer syncFlowCleanupTx.Rollback()
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚫 [golangci] reported by reviewdog 🐶
Error return value of syncFlowCleanupTx.Rollback is not checked (errcheck)

nexus/catalog/src/lib.rs Outdated Show resolved Hide resolved
nexus/catalog/src/lib.rs Outdated Show resolved Hide resolved
nexus/catalog/src/lib.rs Outdated Show resolved Hide resolved
nexus/catalog/src/lib.rs Outdated Show resolved Hide resolved
nexus/flow-rs/src/lib.rs Show resolved Hide resolved
nexus/flow-rs/src/lib.rs Outdated Show resolved Hide resolved
nexus/server/src/main.rs Outdated Show resolved Hide resolved
nexus/server/src/main.rs Show resolved Hide resolved
nexus/server/src/main.rs Show resolved Hide resolved
@iskakaushik iskakaushik merged commit c9c0178 into main Jun 5, 2023
6 checks passed
@iskakaushik iskakaushik deleted the drop-mirror branch June 7, 2023 20:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants