-
Notifications
You must be signed in to change notification settings - Fork 281
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
zero: resource bundle reconciler #4445
zero: resource bundle reconciler #4445
Conversation
package reconciler | ||
|
||
// RecordSetBundle is an index of databroker records by type | ||
type RecordSetBundle[T Record[T]] map[string]RecordSet[T] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: I'm not sure why this is a generic when it's only ever used for DatabrokerRecord
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I found it convenient to use generics for a higher level functionality like this one, as it allows writing more concise and easy to read tests compared to using the full data structure as it might be pretty large. See record_tests.go
} | ||
|
||
// MarkForSyncLater marks the bundle with the given ID for syncing later (after all other bundles). | ||
func (b *BundleQueue) MarkForSyncLater(id string) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: this method is O(n)
. If the goal of using the heap package was to achieve O(log n)
performance, shouldn't all the methods be that or better? Otherwise why not just use a loop?
Summary
Bundle Reconciler watches a group of resource bundles, downloads them and reconciles their contents with the databroker.
Bundles are not expected to have any intersection across record types.
For caching purposes, an entry with etag/last-modified is kept in the databroker, and the upload is conditional. That way, there's no need to keep a cache of downloaded bundles.
Related issues
Implements https://github.com/pomerium/internal/issues/1423
User Explanation
Checklist
improvement
/bug
/ etc)