Add usage metrics collector #99
Assignees
Comments
|
added in #125 |
agrski
added a commit
that referenced
this issue
Dec 2, 2022
* Add bus-based event hub Use mustafaturan/bus as a seemingly well-featured bus implementation. Use an atomic counter as its sequence generator, to avoid adding further dependencies for a simple construct. The sequence generator is per hub to minimise the risk of shared state during testing. * Embed event bus in hub for method use * Register topics for hub on creation * Add source field to event hub logger * Add docstring for NewEventHub * Add sync -> async adapter func for model event message handling This function attempts to solve two issues: * Limit leaking implementation details about the use of mustafaturan/bus to the wider application * Provide a simple interface for users with an asynchronous approach * Refactor model event message handler creation This should be slightly more legible, but more importantly gives a name to the actual handle function, to improve logging. * Use bus-based event in scheduler's main package * Use bus-based event hub in in-memory store * Use bus-based event hub in XDS incremental processor * Make XDS processor's sync-handling internal * Use bus-based event hub in agent server + make handling of syncs internal * Use bus-based event hub in version cleaner + make handling of syncs internal * Use bus-based event hub in scheduler server + make handling of events/listeners internal * Add comment re use of Goroutines in agent server * Add comment re (graceful) shutdown of scheduler * Fix typos in event hub implementation * Format of long lines * Add model event publish method for event hub Also add comment on use of goroutines. The only place writing into the event hub previously was memory_status.go. That chose to publish events from goroutines, but that may not be the best policy - we should consider this. * Replace event hub publish calls when updating model state These used to be 'trigger' calls to the old event hub, but are now 'publish' calls to the bus-based event hub. * Remove unused methods for ModelEventHub * Fix typos & build issues * Add comment re closing channels for event hub * Fix store package tests re use of event hub * Fix scheduler server package tests re use of event hub * Fix agent server package tests re use of event hub * Remove ModelEventHub as fully replaced * Refactor event hub types to separate file within package * Use consistent handler keys for event hub handlers Rather than using free-form text as keys, this change uses dot-separated fields. The fields identify the handling type's purpose first, followed by the event's semantics. For example, the scheduler server listens to model events but treats them in two different ways: as model events, but also as events that can affect servers. * Refactor event hub handler names to package-level constants * Add logging for event publishing failures * Refactor event source names to package-level constants * Add mechanism for closing event hub & its handler channels * Stop processing messages of invalid type * Remove exposure of 'bus' library from event hub interface * Fix typos in channel consumption * Refactor to abstract creation of event hub handlers from 'bus' implementation * Make event publishing synchronous This avoids potential reordering of events caused by the use of goroutines. Instead, callers have explicit control over sync/async operation. * Make event hub topic name private * Add test for event hub creation * Use 'require' for assertion on error in test * Add test cases for registering handlers * Simplify event handler test to use single counter with atomic increments * Add test cases for publishing events * Add test cases for closing event hub
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
We can use Spartakus as are kubeflow
The text was updated successfully, but these errors were encountered: