🚀 Feature: Remove entities from old EntityProviders by default

[jhaals]

🔖 Feature description

Deleting(uninstalling) an entityProvider or changing the EntityProvider name cause Entities to remain in the catalog unless manually removed. This causes hard to debug issues for users that for example move to a new Entity Provider or change their Entity Provider name leading to reference issues as the source_key(EntityProvider name) in the refresh_state_references table still remain. This manifest itself as deletion/sync issues as the Entity is "owned" by a provider which is in reality no longer in use so the entity will not be managed any longer.

✌️ Possible Implementation

Change the default startup behaviour of the Catalog to remove entities from EntityProviders that are no longer installed.
This will work for most users but will be undesirable if you shard your ingestion and have a subset of EntityProviders installed on some backends only. This could be mitigated by having a config flag to disable this behaviour.

Initially we could add said config flag but default it to false to retain the old behaviour and potentially avoid having to introduce a breaking change in the catalog.

👀 Have you spent some time to check if this feature request has been raised before?

• I checked and didn't find similar issue

🏢 Have you read the Code of Conduct?

• I have read the Code of Conduct

Are you willing to submit PR?

No, I don't have time to work on this right now

[efenner-cambia]

I would also love to see this change. We recently changed the name of one of our providers and ended up getting conflicting location warnings on every refresh cycle

[Manish3323]

+1

[benjdlambert]

Discussed with @backstage/catalog-maintainers and we're thinking that this is a good idea. Going to look at picking this up soon.

[SonilPro]

@benjdlambert any news on this?