Add event support for the Catalog Plugin GitLab Discovery Entity providers #23373

elaine-mattos · 2024-03-03T12:53:10Z

Hey, I just made a Pull Request!

Add events support for the GitLab Entity Provider modules.
Refs: RFC: Event Subscription Mechanism for Cross-Plugin Module Integration in the New Backend System #22393 and 🚀 Feature: Add Events Support for GitlabDiscoveryEntityProvider and GitlabOrgDiscoveryEntityProvider #21553

✔️ TODO List

Add GitLabOrgDiscoveryEntityProvider module (new Backend)
Add events support for the GitLabDiscoveryEntityProvider
Add events support for the GitLabOrgDiscoveryEntityProvider
Add tests
Update documentation
Remove EventSubscriber interface implementation (feat(GithubMultiOrgEntityProvider): support events #17565 (comment))
Remove the Events Extension Point and add the Events Service

✔️ Checklist

A changeset describing the change and affected packages. (more info)
Added or updated documentation
Tests for new functionality and regression tests for bug fixes
Screenshots attached (for UI changes)
All your commits have a Signed-off-by line in the message. (more info)

backstage-goalie · 2024-03-03T12:53:42Z

Changed Packages

Package Name	Package Path	Changeset Bump	Current Version
@backstage/plugin-catalog-backend-module-gitlab-org	plugins/catalog-backend-module-gitlab-org	patch	`v0.0.0`
@backstage/plugin-catalog-backend-module-gitlab	plugins/catalog-backend-module-gitlab	patch	`v0.3.15-next.1`

github-actions · 2024-03-03T14:10:36Z

Uffizzi Cluster pr-23373 was deleted.

vinzscam

amazing work 👏
Just added a tiny comment about the changesets, otherwise ✨

plugins/catalog-backend-module-gitlab/src/providers/GitlabOrgDiscoveryEntityProvider.ts

.changeset/dull-sloths-clap.md

plugins/catalog-backend-module-gitlab/src/providers/GitlabOrgDiscoveryEntityProvider.ts

plugins/catalog-backend-module-gitlab/src/lib/client.ts

.changeset/eleven-pandas-divide.md

.changeset/shy-students-clap.md

plugins/catalog-backend-module-gitlab/src/providers/GitlabOrgDiscoveryEntityProvider.ts

elaine-mattos · 2024-03-16T17:44:48Z

Hi @vinzscam !

I had some issues with my branch and had to recreate it (I know, I'm such a git newbie! :D ).
I did incorporate your suggestions/remarks. Could you please take a look at them?
Also, the pipeline is throwing a formatting error on a file I did not touch (beps/0004-scaffolder-task-idempotency/README.md). Should I do anything about it?

thanks again in advance!!! :)

freben · 2024-03-19T13:48:18Z

Yeah that prettier error has been fixed since then. Rebasing on a fresh master gets rid of the problem. Something like this

git checkout master
git pull
git checkout feat/add-event-support-gitlab-provider
git pull
git rebase master

elaine-mattos · 2024-03-20T22:29:22Z

Hi guys,

the suggested changes have been made. Could you please take a look?
I appreciate the feedback :)

elaine-mattos · 2024-04-15T20:04:50Z

Hi @elaine-mattos, can you please add details about the fact that you need to define the schedule in your provider config when using the new backend system? This is missing in the current docs and I don't see it here. Thanks!

Hi @awanlin ! I've changed the docu a little bit. Could you please review? :)

elaine-mattos · 2024-04-15T20:05:22Z

also very sorry but a rebase is needed now

Hi @freben , done! :)

awanlin

Thanks for adding in the schedule details to the docs @elaine-mattos, I left two comments but they are the same. As I haven't reviewed the code just the docs I'm only leaving this review as a comment

awanlin · 2024-04-16T13:08:02Z

docs/integrations/gitlab/org.md

        groupPattern: '[\s\S]*' # Optional. Filters found groups based on provided pattern. Defaults to `[\s\S]*`, which means to not filter anything
+        schedule: # optional; same options as in TaskScheduleDefinition


Is this true in all cases that it's optional? If I'm using the New Backend System would that not then be required?

Hi @awanlin !

I changed this line. Could you please check? :)

docs/integrations/gitlab/discovery.md

Signed-off-by: ElaineDeMattosSilvaB <elaine.de-mattos-silva-bezerra@deutschebahn.com>

Signed-off-by: elaine-mattos <elaine.mattos@gmail.com>

Signed-off-by: ElaineDeMattosSilvaB <elaine.de-mattos-silva-bezerra@deutschebahn.com>

elaine-mattos · 2024-04-22T13:12:39Z

Hi @vinzscam, @freben , @awanlin ,

is there anything else I can do to get this one merged?
Please let me know if there is anything missing! :)

freben

OK, phew! I left some comments that can be deferred to a follow-up PR some time down the line. Do check them out. But for now,

freben · 2024-04-22T21:01:39Z

docs/integrations/gitlab/org.md

 As this provider is not one of the default providers, you will first need to install the Gitlab provider plugin:

 ```bash
 # From your Backstage root directory
-yarn --cwd packages/backend add @backstage/plugin-catalog-backend-module-gitlab
+yarn --cwd packages/backend add @backstage/plugin-catalog-backend-module-gitlab @backstage/plugin-catalog-backend-module-gitlab-org


really both?

freben · 2024-04-22T21:04:58Z

plugins/catalog-backend-module-gitlab-org/src/catalogModuleGitlabOrgDiscoveryEntityProvider.ts

+  coreServices,
+  createBackendModule,
+} from '@backstage/backend-plugin-api';
+import { GitlabOrgDiscoveryEntityProvider } from '@backstage/plugin-catalog-backend-module-gitlab';


I do agree that the implementation should really be moved over to the package where it belongs, and deprecated in the old place. If in a follow-up PR - fine. But should probably be done soonish :)

freben · 2024-04-22T21:06:30Z

plugins/catalog-backend-module-gitlab/package.json

    "@backstage/plugin-catalog-node": "workspace:^",
+    "@backstage/plugin-events-node": "workspace:^",
+    "@gitbeaker/rest": "^40.0.3",


Suggested change

"@gitbeaker/rest": "^40.0.3",

"@gitbeaker/rest": "^39.25.0",

if that's okay, to align with the other deps in the repo - this should clean up the yarn.lock diff

freben · 2024-04-22T21:08:23Z

plugins/catalog-backend-module-gitlab/package.json

+      ]
+    }
+  },
+  "version": "0.3.15-next.1"


not sure what happened with the field order sorting here :) version is usually way up at the top etc, but we can sort (hah!) that out some other time

freben · 2024-04-22T21:18:15Z

plugins/catalog-backend-module-gitlab/src/providers/GitlabDiscoveryEntityProvider.ts

 * @public
 */
+// <<< EventSupportChange: implemented EventSubscriber interface


freben · 2024-04-22T21:28:41Z

plugins/catalog-backend-module-gitlab/src/providers/GitlabOrgDiscoveryEntityProvider.ts

@@ -96,7 +133,7 @@ export class GitlabOrgDiscoveryEntityProvider implements EntityProvider {
      const integration = integrations.byHost(providerConfig.host);

      if (!providerConfig.orgEnabled) {
-        return;
+        throw new Error(`Org not enabled for ${providerConfig.id}.`);


Is this really a safe change?

github-actions · 2024-04-22T21:33:27Z

Thank you for contributing to Backstage! The changes in this pull request will be part of the 1.27.0 release, scheduled for Tue, 14 May 2024.

github-actions bot added the area:catalog Related to the Catalog Project Area label Mar 3, 2024

github-actions bot added the documentation Improvements or additions to documentation label Mar 3, 2024

elaine-mattos marked this pull request as ready for review March 3, 2024 17:36

elaine-mattos requested review from a team and backstage-service as code owners March 3, 2024 17:36

elaine-mattos requested review from freben, vinzscam and alde March 3, 2024 17:36

vinzscam requested changes Mar 4, 2024

View reviewed changes

vinzscam reviewed Mar 4, 2024

View reviewed changes

plugins/catalog-backend-module-gitlab/src/lib/client.ts Outdated Show resolved Hide resolved

elaine-mattos requested a review from vinzscam March 5, 2024 17:41

vinzscam requested changes Mar 6, 2024

View reviewed changes

elaine-mattos closed this Mar 11, 2024

elaine-mattos force-pushed the feat/add-event-support-gitlab-provider branch from 4db44fb to 580dcf8 Compare March 11, 2024 21:15

elaine-mattos reopened this Mar 16, 2024

github-actions bot removed the documentation Improvements or additions to documentation label Mar 16, 2024

elaine-mattos marked this pull request as draft March 16, 2024 11:26

elaine-mattos marked this pull request as ready for review March 16, 2024 17:41

elaine-mattos force-pushed the feat/add-event-support-gitlab-provider branch from 093d668 to 24b2bbb Compare March 20, 2024 21:17

elaine-mattos marked this pull request as draft March 20, 2024 21:18

github-actions bot added the documentation Improvements or additions to documentation label Mar 20, 2024

elaine-mattos marked this pull request as ready for review March 20, 2024 22:27

elaine-mattos mentioned this pull request Mar 22, 2024

💬 RFC: Event Subscription Mechanism for Cross-Plugin Module Integration in the New Backend System #22393

Closed

2 tasks

elaine-mattos marked this pull request as draft April 15, 2024 17:01

elaine-mattos force-pushed the feat/add-event-support-gitlab-provider branch from 2bad1cc to 07eb276 Compare April 15, 2024 17:04

elaine-mattos marked this pull request as ready for review April 15, 2024 20:03

awanlin reviewed Apr 16, 2024

View reviewed changes

elaine-mattos and others added 15 commits April 18, 2024 08:54

feat: add events support for gitlab catalog entity provider

a70377d

Signed-off-by: ElaineDeMattosSilvaB <elaine.de-mattos-silva-bezerra@deutschebahn.com>

fix: fix branch

4d87f98

Signed-off-by: ElaineDeMattosSilvaB <elaine.de-mattos-silva-bezerra@deutschebahn.com>

fix: fix imports

36f0081

Signed-off-by: ElaineDeMattosSilvaB <elaine.de-mattos-silva-bezerra@deutschebahn.com>

fix: add api-report

bb3fbd0

Signed-off-by: ElaineDeMattosSilvaB <elaine.de-mattos-silva-bezerra@deutschebahn.com>

fix: merge conflicts

aef7d0e

Signed-off-by: ElaineDeMattosSilvaB <elaine.de-mattos-silva-bezerra@deutschebahn.com>

chore: add documentation and fix package.json

c3c2591

Signed-off-by: ElaineDeMattosSilvaB <elaine.de-mattos-silva-bezerra@deutschebahn.com>

fix: solve uninstalled deps

63336c8

Signed-off-by: ElaineDeMattosSilvaB <elaine.de-mattos-silva-bezerra@deutschebahn.com>

feat: incorporate review changes

c2f20d5

Signed-off-by: ElaineDeMattosSilvaB <elaine.de-mattos-silva-bezerra@deutschebahn.com>

chore: add yarn.lock

d2928ac

Signed-off-by: elaine-mattos <elaine.mattos@gmail.com>

fix: add allowInherited to readGitlabConfig function

0b208f8

Signed-off-by: ElaineDeMattosSilvaB <elaine.de-mattos-silva-bezerra@deutschebahn.com>

feat: improve documentation

38dfefe

Signed-off-by: ElaineDeMattosSilvaB <elaine.de-mattos-silva-bezerra@deutschebahn.com>

fix: add role to package.json

0f394e7

Signed-off-by: ElaineDeMattosSilvaB <elaine.de-mattos-silva-bezerra@deutschebahn.com>

fix: add missing dependencies

72e2339

Signed-off-by: ElaineDeMattosSilvaB <elaine.de-mattos-silva-bezerra@deutschebahn.com>

chore: add api-report

d35b6ba

Signed-off-by: ElaineDeMattosSilvaB <elaine.de-mattos-silva-bezerra@deutschebahn.com>

feat: improve documentation

4594531

Signed-off-by: ElaineDeMattosSilvaB <elaine.de-mattos-silva-bezerra@deutschebahn.com>

elaine-mattos force-pushed the feat/add-event-support-gitlab-provider branch from 282eeb7 to 4594531 Compare April 18, 2024 08:31

elaine-mattos requested a review from awanlin April 18, 2024 11:28

BillKlineVT mentioned this pull request Apr 21, 2024

🐛 Bug Report: Gitlab discovery with SAML users and groups #24361

Closed

2 tasks

freben approved these changes Apr 22, 2024

View reviewed changes

freben merged commit 3778678 into backstage:master Apr 22, 2024
46 checks passed

elaine-mattos deleted the feat/add-event-support-gitlab-provider branch May 20, 2024 17:08

ZF-fredericvanlinthoudt mentioned this pull request May 21, 2024

🐛 Bug Report: cannot use GitLab Org Data and Discovery together anymore #24857

Closed

2 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add event support for the Catalog Plugin GitLab Discovery Entity providers #23373

Add event support for the Catalog Plugin GitLab Discovery Entity providers #23373

elaine-mattos commented Mar 3, 2024 •

edited

backstage-goalie bot commented Mar 3, 2024 •

edited

github-actions bot commented Mar 3, 2024 •

edited

vinzscam left a comment

elaine-mattos commented Mar 16, 2024

freben commented Mar 19, 2024

elaine-mattos commented Mar 20, 2024

elaine-mattos commented Apr 15, 2024

elaine-mattos commented Apr 15, 2024

awanlin left a comment

awanlin Apr 16, 2024

elaine-mattos Apr 18, 2024

elaine-mattos commented Apr 22, 2024

freben left a comment

freben Apr 22, 2024

freben Apr 22, 2024

freben Apr 22, 2024

freben Apr 22, 2024

freben Apr 22, 2024

freben Apr 22, 2024

github-actions bot commented Apr 22, 2024

		groupPattern: '[\s\S]' # Optional. Filters found groups based on provided pattern. Defaults to `[\s\S]`, which means to not filter anything
		schedule: # optional; same options as in TaskScheduleDefinition

Add event support for the Catalog Plugin GitLab Discovery Entity providers #23373

Add event support for the Catalog Plugin GitLab Discovery Entity providers #23373

Conversation

elaine-mattos commented Mar 3, 2024 • edited

Hey, I just made a Pull Request!

✔️ TODO List

✔️ Checklist

backstage-goalie bot commented Mar 3, 2024 • edited

Changed Packages

github-actions bot commented Mar 3, 2024 • edited

vinzscam left a comment

Choose a reason for hiding this comment

elaine-mattos commented Mar 16, 2024

freben commented Mar 19, 2024

elaine-mattos commented Mar 20, 2024

elaine-mattos commented Apr 15, 2024

elaine-mattos commented Apr 15, 2024

awanlin left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

elaine-mattos commented Apr 22, 2024

freben left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Apr 22, 2024

elaine-mattos commented Mar 3, 2024 •

edited

backstage-goalie bot commented Mar 3, 2024 •

edited

github-actions bot commented Mar 3, 2024 •

edited