Add support for pipeline parameters

[kiersten-stokes]

This PR, once out of draft, will address all bullet points in #2994 with the exception of the CLI support.

Related: discussion #2970

What changes were proposed in this pull request?

This PR includes both the frontend and backend changes required to support pipeline parameters.

• a new API endpoint and handler to fetch pipeline parameter schema /pipeline/{processor_type}/parameters
• a new class to represent a well-defined PipelineParameter object (one for each runtime type that supports parameters) and supply the schema with runtime-specific input types
• add plumbing to support a fourth "input type" (aka "widget type") called parameter for relevant custom component properties
• add an extra property pipeline_parameters to the generic component properties
• add support for converting and passing parameters to the Pipeline object after submission
• add support for adding pipeline parameters to the generated DSL
• add support for passing pipeline parameters as arguments to the KFP bootstrapper and setting them as environment variables in the bootstrapped container

NOTE: After some discussion, it was decided that generic pipelines should support the set of properties/parameters/etc that are supported by all runtimes (ie an intersection). While changes will be required to make this true of how properties are handled, this pipeline parameter support PR does follow this directive.

TODO

• backend: rename variables and files that include the substring parameter to use property instead (where applicable)
• backend: implement processing and validation logic for custom and generic components
• backend: fix KFP icon rendering
• backend: add backend tests
• backend: address all TODOs
• backend: add validation and/or trap error during compile for mismatched KFP types
• backend/frontend: add description to individual fields for each parameter
• backend/frontend: only allow selection of parameter type(s) that match the given component-defined type - may be better in a follow-up PR
• frontend: add support for new third "Pipeline Parameters" panel tab
• frontend: fix style for tooltip
• frontend: fix validation issues
• frontend: add types to submit/export dialog
• frontend: (nice to have) display custom message in above tab for runtimes that do not support parameters
• frontend: add params to the appropriate location in the pipeline JSON
• frontend: add new widget type named parameter; component properties will appear as below in the pipeline JSON

  • "url": {
      "widget": "parameter",
      "value": "user_defined_param_1"
    },

• frontend: add ability to change pipeline parameter default values on pipeline submit
• frontend: add ability to change pipeline parameter default values on pipeline export
• frontend: implement parameter widget selection for custom components
• frontend: implement parameter property for generic components
• documentation: add pipeline parameters content to docs or open another issue/PR to address -> Add documentation for pipeline parameters #3046

How was this pull request tested?

Manual testing.
Added new backend tests.

NOTE: To test this PR as it relates to generic components, you will have to update the ELYRA_BOOTSTRAP_SCRIPT_URL env var to point to this branch https://raw.githubusercontent.com/kiersten-stokes/elyra/pipeline-params/elyra/kfp/bootstrapper.py

Developer's Certificate of Origin 1.1

   By making a contribution to this project, I certify that:

   (a) The contribution was created in whole or in part by me and I
       have the right to submit it under the Apache License 2.0; or

   (b) The contribution is based upon previous work that, to the best
       of my knowledge, is covered under an appropriate open source
       license and I have the right under that license to submit that
       work with modifications, whether created in whole or in part
       by me, under the same open source license (unless I am
       permitted to submit under a different license), as indicated
       in the file; or

   (c) The contribution was provided directly to me by some other
       person who certified (a), (b) or (c) and I have not modified
       it.

   (d) I understand and agree that this project and the contribution
       are public and that a record of the contribution (including all
       personal information I submit with it, including my sign-off) is
       maintained indefinitely and may be redistributed consistent with
       this project or the open source license(s) involved.

[elyra-bot]

Thanks for making a pull request to Elyra!

To try out this branch on binder, follow this link:

[github-advanced-security]

CodeQL found more than 10 potential problems in the proposed changes. Check the Files changed tab for more details.

[kiersten-stokes]

Note: the CodeQL warnings on this PR are 'the usual suspects' and have not been introduced by any changes made in this PR. (I think they may be triggered by the renaming of the component_parameter.py file to properties.py.)

[ptitzler]

Note: the CodeQL warnings on this PR are 'the usual suspects' and have not been introduced by any changes made in this PR. (I think they may be triggered by the renaming of the component_parameter.py file to properties.py.)

I dismissed them.

[ptitzler]

LGTM. Thanks to you both for an awesome new feature!