import export improvements

[mvi-eea-sk]

Pipeline import from zip file

• resolves sto_82v1: Import pipeline from zip file containing no DPU JARs OpenDataNode/open-data-node#210
• resolves Export/Import v2 #356 (partially)
• resolved Importing Child Templates does not work #503
• resolves Pipeline import doesn't work #395 (partially)

Implemented features / changes:

• checking permission if the DPU template is visible to the user
• checking version and comparing them
• doesn't allow to import pipeline with DPU tamplates of higher version as in the system
• pipeline with DPU templates with version older or the same version can be imported
• DPU instance of 3rd lvl template (with unchecked "Use template configuration") are changed to use 2lvl DPU template
• 3rd lvl DPU templates are matched by name and configuration, so:
• DPU instance of 3rd lvl template (with checked "Use template configuration") and if DPU template with matching name is found:
  • is automatically imported if the configuration match too
  • can choose different import strategies if the configurations are different

Import strategies for 3rd lvl DPU templates with matching name but different configuration:

• user can choose to use the DPU template of the system it is imported to and lose the original configuration (the pipeline may not work the way as in the original system)
• user can choose to replace the DPU instance configuration so it will work exactly as in the system it is imported from (the "Use template configuration" will be unchecked)

Minor fixes:

• when searching for DPU template, its searched by name and directory so it will not find templates of different parent
• fixed that user had to relog if he wanted to see template in dpu tree, that was imported by another user

NOT implemented yet:

• upgrading dpu jar to newer version if its in the zip when importing pipeline from zip
• installing DPU templates that are not present in the system when importing pipeline from zip
• importing DPU templates from zip that can update old versions jars

Note:

• jar directory and version of template are parsed from jar name and will be not read from jar manifest. This allows to compare versions for pipeline zip that doesn't have jars exported.
• sto_82v2 contains the not implemented features

[tomas-knap]

Will review today

[tomas-knap]

Major issue: Imported pipelines are different then the exported pipelines

UV A has DPU templates T1 and T1.1, which is child of T1. UV A has pipeline P with DPU X, which is an instance of DPU template T1.1. (3rd level), which is instance of DPU template T1 (2nd level). UV B has DPU the same set of DPUs. If I export pipeline P from UV A and import to UV B, the imported pipeline will have DPU Y, which is instance of T1 directly. So basically export and import changes DATA and also the work done by pipeline designer on UV A is simply lost by importing to UV B (the imported pipeline is using different DPU templates).

As I discussed with @mvi-eea-sk , he was doing the matching of 3rd level DPU templates anyway (for the case when DPU template config is checked), so this should be used for all templates.

Solution: DPU instances of 3rd lvl templates (with unchecked "Use template configuration") are also mapped to 3rd lvl templates in the target system. When importing, it is simply checked that the 3rd level template is in the target system. If not, it cannot be imported and the missing DPU is among missing DPUs (in the import dialog). The 3rd lvl template on the target system is matched by the name and type (JAR file). IT is NOT checked that the configuration of the target system's 3rd level template is the same as the configuration of the source system's 3rd level template, because the imported DPU instance anyway uses its instance configuration.

[tomas-knap]

I prepared pipeline with instance of t-filesRenamer 2.2.0. When I click on Export from pipeline detail, I got:

Why the version is unknown?

The same for import dialog:

Sometimes it also happens that some versions are parsed and some not (see the comment below)

[tomas-knap]

When I try to import a pipeline
which contains newer version of the DPU template (2.3.0) then is available in the target system (there is 2.2.0), I got rather brutal and confusing error message (see below):

This is really confusing.

I would expect that in this case, I DO NOT get any big error message, but, there will be information that certain DPU template missing in the system (better would be to inform that required version of certain DPU template is missing), which is more or less shown in the lower table (see below).

Instead of the big and confusing warning message, there should be some red text below the table with missing DPU templates saying "Import is not possible due to missing DPU templates".

[tomas-knap]

One further note: Is it really so difficult to implement "installing DPU templates that are not present in the system when importing pipeline from zip"?

It is really confusing to be start by importing pipeline, then closing the dialog, switching to dpu templates, importing dpu templates, then re-import pipelines.

It seems to me that the only thing which must be developed is one button on pipeline import dialog, which calls "dpuTemplate.import()". Or am I not correct?

It should work in this way:

1. I would like to import pipeline, click on pipelines->import.
2. I upload the zip with pipeline import.
3. it shows which DPUs templates are missing
4. there is a button import missing DPU templates (new). By clicking this button, dpu templates are imported in the same way as if the user would close this dialog and click on button dpu templates->import.
5. dialog with missing DPU templates is refreshed. (new)
6. I can import pipeline

[tomas-knap]

@mvi-eea-sk Please call me on Monday - morning.

[tomas-knap]

TODO @tomas-knap Labels for import strategy

[tomas-knap]

Minor issue: As I adjusted labels in EN version, please update a bit layout of the dialog so that there is enough space for 3-line description of the table:

[tomas-knap]

I checked the previous bugs above and these are solved

[mvi-eea-sk]

@tomas-knap layout adjusted

• missing localization of table columns added
• updated SK localization