feat(pg-queue): Migrated secret webhook and replication to postgres #3839
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
🎉 Snyk checks have passed. No issues have been found so far.✅ security/snyk check is complete. No issues have been found. (View Details) |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
There was a problem hiding this comment.
PR Summary
Migrates critical queue operations from Redis to PostgreSQL for secret webhooks and replication, improving persistence and reliability of the secret management system.
- Implements PostgreSQL advisory locks and transaction support in
secret-rotation-v2-service.tsto handle concurrent operations safely - Introduces new
TSecretQueueFactorytype insecret-types.tsto improve TypeScript inference and maintain type safety across queue operations - Adds initialization steps in
routes/index.tsto properly sequence secret replication and queue services during server startup - Enhances secret approval workflow in
secret-approval-request-service.tswith support for encrypted secret values using KMS - Maintains backwards compatibility through dual queue support while transitioning from Redis to PostgreSQL
13 files reviewed, 2 comments
Edit PR Review Bot Settings | Greptile
Comment on lines
+552
to
+559
| syncIntegrations: (dto: { | ||
| secretPath: string; | ||
| projectId: string; | ||
| environment: string; | ||
| isManual?: boolean; | ||
| actorId?: string; | ||
| deDupeQueue?: Record<string, boolean>; | ||
| }) => Promise<void>; |
There was a problem hiding this comment.
style: Consider extracting syncIntegrations DTO into a separate named type for reusability
Comment on lines
+861
to
863
| queueService.start(QueueName.SecretReplication, async (job) => { | ||
| await $secretReplicationQueueTask(job.id as string, job.data); | ||
| }); |
There was a problem hiding this comment.
logic: Potential race condition: Both Redis and Postgres queues are active simultaneously. Need coordination mechanism or migration strategy to prevent duplicate processing.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description 📣
This is part of persistent queue initiative in which the queue that are critical are moved away from redis to postgres. This PR moves secret replication and webhook to postgres queue.
This PR also corrects the ts infer of secret queue service to make ts infer faster.
Type ✨
Tests 🛠️
# Here's some code block to paste some code snippets