Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE] Expose wadm-internal events on a new stream #251

Open
protochron opened this issue Apr 2, 2024 · 0 comments
Open

[FEATURE] Expose wadm-internal events on a new stream #251

protochron opened this issue Apr 2, 2024 · 0 comments

Comments

@protochron
Copy link
Contributor

Right now the wadm_events stream is defined as a WorkQueue which is great for controlling what Wadm works on but is far less useful for external consumers of wadm. This is because you can only ever have a single consumer for any WorkQueue!

The issue here is that the wadm_events stream is the only place where manifest deploy and undeploy events are logged, so without polling for updates there is not a great way to be able to subscribe for only those events. An alternative which I'm pursuing is creating a stream mirror of the wadm_events stream so that I can create a new consumer so that the wasmcloud-operator can take actions whenever a new manifest is deployed or one is undeployed.

With the new features in NATS 2.10 (similar to #81), we can be much smarter in how we create new consumers and expose a stream for downstream consumers. What I'd like to do is:

  • keep relaying the existing notifications to wadm_events for now.
  • relay all deploy and undeploy notifications to a new stream (wadm_scheduler_events? Not sure of a good name yet). The stream's subject should be something like wadm_scheduler_events.>, and wadm should add the deploy and undeploy notifications on their own prefixed subjects, ex. wadm_scheduler_events.$lattice_id.manifest_deployed and wadm_scheduler_events.$lattice_id.manifest_undeployed. Doing so would allow consumers of the stream to only subscribe to only events they are interested on particular lattices or even only for specific events without needing to parse a CloudEvent and check the payload for the event type before deciding on what to process. This should not be a work queue and should instead be a stream with sensible limits. Downstream consumers can then mirror the stream into a work queue or anything else they want and build out consumers on that stream.
  • Eventually remove the manifest deployed and undeployed events from the wadm_events stream and use a different work queue to provide the same functionality. This work might overlap with Remove mirror stream on update to NATS 2.10 #81 since we wouldn't necessarily need one main mirror stream to drive all work in wadm.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant