new backend system: events service #22344

pjungermann · 2024-01-18T21:24:07Z

Hey, I just made a Pull Request!

✔️ Checklist

A changeset describing the change and affected packages. (more info)
Added or updated documentation
Tests for new functionality and regression tests for bug fixes
Screenshots attached (for UI changes)
All your commits have a Signed-off-by line in the message. (more info)

backstage-goalie · 2024-01-18T21:24:33Z

Changed Packages

Package Name	Package Path	Changeset Bump	Current Version
@backstage/backend-dynamic-feature-service	packages/backend-dynamic-feature-service	patch	`v0.2.0`
example-backend	packages/backend	none	`v0.2.92`
@backstage/plugin-catalog-backend-module-bitbucket-cloud	plugins/catalog-backend-module-bitbucket-cloud	minor	`v0.1.25`
@backstage/plugin-events-backend-module-aws-sqs	plugins/events-backend-module-aws-sqs	minor	`v0.2.13`
@backstage/plugin-events-backend-module-azure	plugins/events-backend-module-azure	minor	`v0.1.20`
@backstage/plugin-events-backend-module-bitbucket-cloud	plugins/events-backend-module-bitbucket-cloud	minor	`v0.1.20`
@backstage/plugin-events-backend-module-gerrit	plugins/events-backend-module-gerrit	minor	`v0.1.20`
@backstage/plugin-events-backend-module-github	plugins/events-backend-module-github	minor	`v0.1.20`
@backstage/plugin-events-backend-module-gitlab	plugins/events-backend-module-gitlab	minor	`v0.1.20`
@backstage/plugin-events-backend-test-utils	plugins/events-backend-test-utils	patch	`v0.1.20`
@backstage/plugin-events-backend	plugins/events-backend	minor	`v0.2.19`
@backstage/plugin-events-node	plugins/events-node	minor	`v0.2.19`

github-actions · 2024-01-18T21:33:24Z

Uffizzi Cluster pr-22344 was deleted.

plugins/events-node/src/api/DefaultEventService.ts

pjungermann · 2024-01-24T10:59:37Z

The "Microsite" check is failing with out of memory. I tried retriggering it a couple of times already.

pjungermann · 2024-02-19T10:00:16Z

@freben I tried to make use of version-bridge. However, it is not permitted to use it at such a package. Maybe, that wasn't really what you meant, though.

Package '@backstage/plugin-events-node' with backend role 'node-library' has a dependency on package '@backstage/version-bridge' with frontend role 'web-library', which is not permitted

pjungermann · 2024-02-19T15:37:27Z

I will remove version-bridge for now.

freben · 2024-02-20T08:48:21Z

...-backend-module-bitbucket-cloud/src/module/catalogModuleBitbucketCloudEntityProvider.test.ts

 import { Duration } from 'luxon';
 import { catalogModuleBitbucketCloudEntityProvider } from './catalogModuleBitbucketCloudEntityProvider';
 import { BitbucketCloudEntityProvider } from '../providers/BitbucketCloudEntityProvider';

 describe('catalogModuleBitbucketCloudEntityProvider', () => {
  it('should register provider at the catalog extension point', async () => {
+    const events = new TestEventsService();
+    const eventsServiceFactory = createServiceFactory({


The test utils package could export a mockEventsServiceFactory in the first place as well, or (structured similarly to mockServices) a mockEventsService.factory(), mockEventsService.service etc or so. No need to look into that at this point though, that can be deferred

I didn't take care of that one.

Currently, the helper functions are not exported at backstage-test-utils. Rebuilding the same seems not right either.

I could put it into backstage-test-utils. However, that would add the dependency to events-node to it.

I could keep them separate and (a) copy or (b) export the helpers to achieve a similar setup.

I could just add a factory implementation similar to how it was done at this test to make it simpler.

...

What do you think?

plugins/events-backend-module-github/README.md

plugins/events-backend-module-gitlab/README.md

freben · 2024-02-20T10:28:16Z

plugins/events-node/src/service.ts

+ */
+export const eventsServiceRef = createServiceRef<EventsService>({
+  id: 'events.service',
+  scope: 'root',


We were looking into merging this for the mainline release, but I'm sorry we think there's one thing that still should be settled.

We were wondering if we could

make this one plugin scoped instead

make it depend on the plugin metadata service and prefixing subscription IDs with <pluginId>.

Then we could possibly introduce a rootEventsServiceRef as well in the future but only if strictly needed (and reimplement the plugin scoped one in terms of the root scoped one). This also makes the naming pattern of this one more aligned.

Does that make sense? We usually add the root ones only if necessary since they kind of by definition are less "powerful". Do you have any particular reason to have a root scoped events service already at this point?

Happy to hear your feedback

@freben The events service is supposed to create an event-based communication across plugin boundaries. Hence, I fail to see how it can be plugin-scoped.

If just the instance itself is plugin-scoped the actual underlying communication is still root-scoped/global, it could work. Even though the benefits might be limited (e.g., the prefixing of subscription IDs).

I guess the version-bridge was an idea to create this shared state/context, despite being plugin-scoped. However, this package is limited to web-library roles.

We could have both, root-scoped and plugin-scoped, and the root-scoped service is used within the plugin-scoped implementations. Not sure if this would help with what you intend to achieve.

Yeah you create the shared state with createRootContext if you need one.

This choice isn't really based on what communication pattern it implements under the hood. It's based on the potential capabilities and whether it might want to contextually be aware of which plugin is using it. And you often do in the end; imagine wanting to emit plugin-tagged metrics based on their events activity for just one example among many.

Thanks for the pointer, forgot about that one. I will have a look at how I can change the setup further.

I switched to plugin-scope. Please let me know if this version works for you.

pjungermann · 2024-02-26T10:46:55Z

I will take care of the merge conflicts

Signed-off-by: Patrick Jungermann <Patrick.Jungermann@gmail.com>

…r` to `EventsService` Signed-off-by: Patrick Jungermann <Patrick.Jungermann@gmail.com>

…gin` to use `EventsService` Signed-off-by: Patrick Jungermann <Patrick.Jungermann@gmail.com>

…ervice` Signed-off-by: Patrick Jungermann <Patrick.Jungermann@gmail.com>

…d module to use `EventsService` Signed-off-by: Patrick Jungermann <Patrick.Jungermann@gmail.com>

Signed-off-by: Patrick Jungermann <Patrick.Jungermann@gmail.com>

…end system Signed-off-by: Patrick Jungermann <Patrick.Jungermann@gmail.com>

Signed-off-by: Patrick Jungermann <Patrick.Jungermann@gmail.com>

… migrates to `EventsService` - Fixes the support for the new backend system that was broken entirely (with and without events support). - Migrates the `BitbucketCloudEntityProvider` to use the `EventsService`. Signed-off-by: Patrick Jungermann <Patrick.Jungermann@gmail.com>

Rugvip

Sweet, I think it's time to! 🎉

Awesome work @pjungermann 👏

github-actions · 2024-02-26T13:33:14Z

Thank you for contributing to Backstage! The changes in this pull request will be part of the 1.24.0 release, scheduled for Tue, 19 Mar 2024.

pjungermann force-pushed the pjungermann/new-backend/events-service branch 3 times, most recently from 6beb1f8 to 21cca84 Compare January 21, 2024 21:54

github-advanced-security bot found potential problems Jan 21, 2024

View reviewed changes

plugins/events-node/src/api/DefaultEventService.ts Fixed Show fixed Hide fixed

pjungermann force-pushed the pjungermann/new-backend/events-service branch 2 times, most recently from fad74bc to 071915e Compare January 21, 2024 23:51

Rugvip mentioned this pull request Jan 22, 2024

💬 RFC: Event Subscription Mechanism for Cross-Plugin Module Integration in the New Backend System #22393

Closed

2 tasks

pjungermann force-pushed the pjungermann/new-backend/events-service branch 7 times, most recently from 4a211f3 to dc41b08 Compare January 23, 2024 20:11

pjungermann changed the title ~~WIP - new backend: event service~~ WIP - new backend system: event service Jan 23, 2024

pjungermann force-pushed the pjungermann/new-backend/events-service branch from dc41b08 to ef78082 Compare January 23, 2024 21:45

github-actions bot added documentation Improvements or additions to documentation area:catalog Related to the Catalog Project Area labels Jan 23, 2024

pjungermann force-pushed the pjungermann/new-backend/events-service branch from ef78082 to 31fc74c Compare January 24, 2024 02:11

pjungermann marked this pull request as ready for review January 24, 2024 10:59

pjungermann requested review from a team and backstage-service as code owners January 24, 2024 10:59

pjungermann requested review from benjdlambert, vinzscam and Edje-C January 24, 2024 10:59

pjungermann force-pushed the pjungermann/new-backend/events-service branch from 31b307a to 4c01b21 Compare February 19, 2024 03:06

pjungermann mentioned this pull request Feb 19, 2024

Added TechDocs backend module for Events #18411

Draft

5 tasks

pjungermann force-pushed the pjungermann/new-backend/events-service branch 4 times, most recently from 8c91817 to 043add7 Compare February 19, 2024 18:28

pjungermann requested review from freben and Rugvip February 19, 2024 18:59

freben reviewed Feb 20, 2024

View reviewed changes

pjungermann force-pushed the pjungermann/new-backend/events-service branch from 043add7 to 155f921 Compare February 21, 2024 14:52

pjungermann requested a review from freben February 22, 2024 00:42

pjungermann added 9 commits February 26, 2024 12:28

feat(events): add new events service

56969b6

Signed-off-by: Patrick Jungermann <Patrick.Jungermann@gmail.com>

feat(events)!: migrate EventRouter implementations from `EventBroke…

eff3ca9

…r` to `EventsService` Signed-off-by: Patrick Jungermann <Patrick.Jungermann@gmail.com>

feat(events)!: migrate HttpPostIngressEventPublisher and `eventsPlu…

c4bd794

…gin` to use `EventsService` Signed-off-by: Patrick Jungermann <Patrick.Jungermann@gmail.com>

chore(events): migrate DemoEventBasedEntityProvider to use `EventsS…

8f6afa9

…ervice` Signed-off-by: Patrick Jungermann <Patrick.Jungermann@gmail.com>

feat(events)!: migrate AwsSqsConsumingEventPublisher and its backen…

132d672

…d module to use `EventsService` Signed-off-by: Patrick Jungermann <Patrick.Jungermann@gmail.com>

feat(events): add events: EventsService to LegacyPluginEnvironment

5247909

Signed-off-by: Patrick Jungermann <Patrick.Jungermann@gmail.com>

docs(events): update docs about events-backend used with the new back…

3c5f586

…end system Signed-off-by: Patrick Jungermann <Patrick.Jungermann@gmail.com>

docs(events): describe the new events setup, update README.md files

1ab76e5

Signed-off-by: Patrick Jungermann <Patrick.Jungermann@gmail.com>

pjungermann force-pushed the pjungermann/new-backend/events-service branch from 155f921 to 9e527c9 Compare February 26, 2024 11:52

Rugvip approved these changes Feb 26, 2024

View reviewed changes

Rugvip merged commit e7a5501 into backstage:master Feb 26, 2024
38 checks passed

pjungermann deleted the pjungermann/new-backend/events-service branch February 26, 2024 14:40

elaine-mattos mentioned this pull request Mar 3, 2024

Add event support for the Catalog Plugin GitLab Discovery Entity providers #23373

Merged

12 tasks

awanlin mentioned this pull request May 2, 2024

🙏 Help Wanted: Documentation for New Backend System Everywhere #24195

Open

46 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

new backend system: events service #22344

new backend system: events service #22344

pjungermann commented Jan 18, 2024 •

edited

backstage-goalie bot commented Jan 18, 2024 •

edited

github-actions bot commented Jan 18, 2024 •

edited

pjungermann commented Jan 24, 2024

pjungermann commented Feb 19, 2024 •

edited

pjungermann commented Feb 19, 2024

freben Feb 20, 2024 •

edited

pjungermann Feb 21, 2024

freben Feb 20, 2024

pjungermann Feb 20, 2024

freben Feb 20, 2024

pjungermann Feb 20, 2024

pjungermann Feb 21, 2024

pjungermann commented Feb 26, 2024

Rugvip left a comment

github-actions bot commented Feb 26, 2024

new backend system: events service #22344

new backend system: events service #22344

Conversation

pjungermann commented Jan 18, 2024 • edited

Hey, I just made a Pull Request!

✔️ Checklist

backstage-goalie bot commented Jan 18, 2024 • edited

Changed Packages

github-actions bot commented Jan 18, 2024 • edited

pjungermann commented Jan 24, 2024

pjungermann commented Feb 19, 2024 • edited

pjungermann commented Feb 19, 2024

freben Feb 20, 2024 • edited

Choose a reason for hiding this comment

pjungermann Feb 21, 2024

Choose a reason for hiding this comment

freben Feb 20, 2024

Choose a reason for hiding this comment

pjungermann Feb 20, 2024

Choose a reason for hiding this comment

freben Feb 20, 2024

Choose a reason for hiding this comment

pjungermann Feb 20, 2024

Choose a reason for hiding this comment

pjungermann Feb 21, 2024

Choose a reason for hiding this comment

pjungermann commented Feb 26, 2024

Rugvip left a comment

Choose a reason for hiding this comment

github-actions bot commented Feb 26, 2024

pjungermann commented Jan 18, 2024 •

edited

backstage-goalie bot commented Jan 18, 2024 •

edited

github-actions bot commented Jan 18, 2024 •

edited

pjungermann commented Feb 19, 2024 •

edited

freben Feb 20, 2024 •

edited