Can we add fallback behaviour for automatically pre-fixed namespace datasets?

[ankatiyar]

Description

Following discussion #4506

Currently a namespaced pipeline also expects the inputs, outputs and parameters to be namespaced. Users can either:

• Add namespaced datasets/parameters to configuration
• Specify mappings of inputs and outputs in the pipeline() function

The goal for this ticket is to explore if this behaviour can be updated so:

• The input, output, parameter mapping can be automatic (eg. use the non-namespaced entry if the namespaced dataset is not available)
• This behaviour is clear and unambiguous

Context

#4506 (comment)

[ankatiyar]

I've opened a PR to demo the solution for this #4700
Would like to get opinions on the solution.

Proposal:
Add a rename (or rename_datasets) flag to the pipeline() method.
When namespacing their pipelines, currently users have to either:

• namespace datasets in catalog
• specify mapping for inputs outputs and parameters

This will allow users to disable the renaming of the datasets when they are using the namespacing feature for deployment purposes.
The value of adding an argument over automatic renaming is so that this setting is deliberate and doesn't confuse the users. The automatic disabling would also add overhead for checking if the datasets are available in the catalog and will couple Pipeline and the DataCatalog class.

[ankatiyar]

Closed by #4700