feat: add new BitbucketCloudEntityProvider and common/shared client lib

[pjungermann]

Hey, I just made a Pull Request!

✔️ Checklist

• A changeset describing the change and affected packages. (more info)
• Added or updated documentation
• Tests for new functionality and regression tests for bug fixes
• Screenshots attached (for UI changes)
• All your commits have a Signed-off-by line in the message. (more info)

[github-actions]

Changed Packages

Package Name	Package Path	Changeset Bump	Current Version
@backstage/plugin-bitbucket-cloud-common	plugins/bitbucket-cloud-common	minor	v0.0.0
@backstage/plugin-catalog-backend-module-bitbucket-cloud	plugins/catalog-backend-module-bitbucket-cloud	minor	v0.0.0
@backstage/plugin-catalog-backend-module-bitbucket	plugins/catalog-backend-module-bitbucket	minor	v0.1.4-next.0

[freben]

This is huge work, thank you. I hope the comments don't deter.

[jhaals]

As already mentioned, great work(as always) Patrick!

The one thing that really complicates this PR is the client generation making the PR gigantic with all the schemas and generated API report.
I'm slightly worried that this ties us too close to the generated code and client. In most cases we try hard not tying ourselves too hard to for other libraries to be able to switch out the client when a new one pops up which is fairly common in javascript land.

Secondly we have very little tooling and common knowledge with schema generation in the repository making this a big risk of technical depth if you decide to work on other things in the future. Is there alternatives to the generation approach proposed in this PR?

I hope you don't take take this as negativity as introducing this method of generation is a big deal for the repository and tooling so it require careful consideration if we go down this path :)

[pjungermann]

@jhaals all fair points!

Let's start with alternatives:

• use an external client library as dependency
• manual written DTOs
• generating models and the client/operations
• other form of generation and/or alignment with the API schema

The one thing that really complicates this PR is the client generation making the PR gigantic with all the schemas and generated API report.

Yes, you are right. It got big due to it being the first time and including all models.
I even decided not to generate the operations. Mostly due influence the design. And start small. Probably, it can be generated, too, by creating a template accordingly. This would make it even bigger, too, though.

Only including the models which are actually used by the client implementation would reduce it considerably, though. Maybe a bit harder to add additional endpoints (without existing models).

I'm slightly worried that this ties us too close to the generated code and client. In most cases we try hard not tying ourselves too hard to for other libraries to be able to switch out the client when a new one pops up which is fairly common in javascript land.

At the end, we control the client and how it looks like by the templates used to generate them. The client lib is inside the backstage project, too. It is easier to change compared to external client libraries.

Secondly we have very little tooling and common knowledge with schema generation in the repository making this a big risk of technical depth if you decide to work on other things in the future.

Picking up the generator didn't took me long. Code generators for OpenAPI schema (client and/or server) are not that uncommon. Still, it takes a moment to get into this, of course.

Is there alternatives to the generation approach proposed in this PR?
Active maintenance, security, etc. are less of concern/more controlled compared to external libraries.

External libraries are likely more feature complete. Extensibility depends on the implementation.
Actually, I even looked for external ones and briefly considered one. Atlassian itself started one, but discontinued the project. Bitbucket.js is quite feature rich and complete considering the API coverage, but has only one maintainer, etc. Implementation regarding loading of data is a bit different, but ok. Also, it didn't work well in Backstage's project env.

Or, we could handwrite all models and APIs. Only those we need at the moment. Manually creating the models vs generating is more error prone though.

The generation is just a means to stay in sync and correct. It should help reduce maintenance efforts.

As you saw, I was doing some improvements to the API schema (has a few bugs; other parts make it easier for generating the right output).

it require careful consideration

Fully agree and open for discussion and open for changing the approach, too.

[pjungermann]

E2E test/cypress is timing out a lot. Trying to retrigger the build by reuploading the last commit

[Rugvip]

Awesome stuff overall 😁 ❤️

Brought up some concerns about the code generation, but I think we should be able to figure out a way to ship this.

I'm feeling that if we do end up stuck in the future where we can't update the model, there is always the option of to drop the generation and continue with hand crafted models, especially if needed for a temporary fix.

Btw, a middle-ground that I want to hear your thoughts on is: what do you think about using the code generation only as a tool for authoring code? So rather using the generated model directly, we copy over the subset that we have the need for in the project right now? That'd reduce the API surface a lot. We could still keep the generated types and modifications so that it's as close to a simple copy-paste as possible, even in source control for easy diffing. Then over time if we eventually feel we just want the entire thing we could start using the generated code directly instead?

Not gonna block this on the failing cypress tests btw, but will keep an eye on that.

[pjungermann]

Btw, a middle-ground that I want to hear your thoughts on is: what do you think about using the code generation only as a tool for authoring code? So rather using the generated model directly, we copy over the subset that we have the need for in the project right now? That'd reduce the API surface a lot. We could still keep the generated types and modifications so that it's as close to a simple copy-paste as possible, even in source control for easy diffing. Then over time if we eventually feel we just want the entire thing we could start using the generated code directly instead?

Sure, we can do that. Manually is easy, automatic a bit more tricky.

..but yeah.. I guess it makes sense. I was a bit in-between as generating the full model makes adding new endpoints a little easier. But smaller API surface is great, too.

[pjungermann]

@Rugvip I took care of all points including reducing the models to have the minimal API surface possible. It is automated (yarn reduce-models) and I wrote it in a way that it is easy to add new methods/endpoints to the client (documented, too).

As this is my first use of ts-morph, let me know if you see any potential improvements. :)

[Rugvip]

Thank you! 🎉

Reduced model does look nicer, and I'm able to verify that it's stable too, at least in my environment. Feeling it's time to

Tiny nit to fix but not a blocker, just giving @backstage/maintainers some extra time to have a look again 😁

[benjdlambert]

Yep - ok. Let's get this in 🚢

[diegomarangoni]

In the bitbucket-discovery plugin I could use parameter rules to limit the allowed entity kinds, is there a way to restrict entity kinds with this entity provider?

[pjungermann]

@diegomarangoni Unfortunately, at the moment only default rules apply as these rules need to be added to the Location entities which is managed by the static config entity provider reading the locations from the app-config.yaml.

I mentioned this case at another PR, too
#10480 (comment)

I plan to add a feature for it, but it still does not exist. And there is no issue yet.

[pjungermann]

@diegomarangoni I created issue #12880 as follow up related to the "allow" rule