[Feature Request]: Metadata Injection keeps obsolete mappings after changing the source or template pipeline

[CarlosJuncher03]

What would you like to happen?

When using ETL Metadata Injection, if the source pipeline or template pipeline is modified or replaced, existing metadata injection mappings are not automatically cleaned up.

As a result, mappings can continue referencing transforms that no longer exist in the selected pipelines. This leads to errors such as:

Source transform 'METADADOS JSON' is not available in 'raw_etl_metadata' pipeline!
Target transform 'JSON DADOS' is not defined in 'raw_template' pipeline!
Target transform 'SEL. FORMATACAO' is not defined in 'raw_template' pipeline!

Apache Hop is able to detect these invalid mappings and displays the “Invalid mapping” dialog asking whether the missing mappings should be removed. However, the obsolete mappings should ideally be cleared or revalidated automatically when the source/template pipeline changes, preventing invalid references from remaining in the configuration.

Issue Priority

Priority: 3

Issue Component

Component: Transforms

[hansva]

I'm not really sure if I'm a fan of this. How should this happen? When should this happen?
I can't really imagine a non-intrusive, user-friendly way to do this. Should a rename of a transform just throw away all your old configuration? No, you probably want to remap those manually in the injection pipeline.

I fear that what you are suggestion will just result in an annoying pop-up that will be ignored by the user anyway

[CarlosJuncher03]

Perhaps in the transformation view, we could highlight in red what no longer exists, making it possible to remove it. Or something along those lines. We could look into interface improvements for this scenario. It’s an idea.