Tekton hub integration #9771

karthikjeeyar · 2021-08-10T09:40:15Z

Fixes:

https://issues.redhat.com/browse/ODC-6134
https://issues.redhat.com/browse/ODC-6135

Analysis / Root cause:
Integrate tektonHub tasks in the pipeline builder, where users can find/install and update a tekton hub tasks in a namespace.

Solution Description:

Pull the tasks from tekton hub api endpoint and add it to the pipeline task catalog
Install / update a tekton hub task to the namespace.
Show the installing decorator when the API call is in progress.
Added access review checks for the tekton hub CTA button.
Update the limited permission flow to have only the installed tasks listed in the modal.

Todo:

Update the Alert message with finalized content (will be provided by UX)

Screen shots / Gifs for design review:

Add already Installed Task flow:

Install task flow:

Update task flow:

Installing and Delete Add task Decorators:

Unit test coverage report:

Test setup:

pipeline-quicksearch-utils
    isSelectedVersionInstalled
      ✓ should return false when the attributes is not present in catalogItem (1ms)
      ✓ should return false when the selected version is not installed
      ✓ should return true when the selected version is installed (1ms)
    isTaskVersionInstalled
      ✓ should return false when the attributes is not present in catalogItem
      ✓ should return false when the installed attribute is not present in catalogItem (1ms)
      ✓ should return true when the installed attribute is set in catalogItem
    isOneVersionInstalled
      ✓ should return false when the attributes is not present in catalogItem (1ms)
      ✓ should return false when the none of the version is installed in the cluster
      ✓ should return true when the one version is installed in the cluster
    isSelectedVersionUpgradable
      ✓ should return false when the attributes is not present in catalogItem (1ms)
      ✓ should return false when the passed version is not upgradable
      ✓ should return false when the selected version is already installed in the cluster
      ✓ should return true when the selected version is upgradable in catalogItem (1ms)
    getTaskCtaType
      ✓ should return the Install button type for the uninstalled task
      ✓ should return the Add button type for the installed version
      ✓ should return the Upgrade button type if selected version is upgradable (1ms)
    isInstalledNamespaceTask
      ✓ should return false if the tekton hub task is passed
      ✓ should return false if the namespace task is not installed through pipeline builder
      ✓ should return true if the namespace task is installed through pipeline builder (1ms)
    isExternalTask
      ✓ should return true for the tekton hub task
      ✓ should return false if clusterTask and namespaceTask is passed
    isTaskSearchable
      ✓ should return true for all the tekton hub tasks are searchable (1ms)
      ✓ should return true for all the cluster and namespace tasks not added through pipeline builder
      ✓ should return false for all the namespace tasks added through pipeline builder
    getSelectedVersionUrl
      ✓ should return null if the attirbutes is not present in the catalogItem (1ms)
      ✓ should return the download url when the item and version id is passed
    findInstalledTask
      ✓ should return undefined the catalogItem is not installed through pipeline builder
      ✓ should return the installed Task resource through pipeline builder (1ms)

Browser conformance:

Chrome
Firefox
Safari
Edge

cc: @serenamarie125 @bgliwa01 @andrewballantyne

frontend/packages/console-shared/src/components/quick-search/QuickSearchList.tsx

invincibleJai · 2021-08-10T10:34:15Z

frontend/packages/pipelines-plugin/src/components/catalog/const.ts

@@ -0,0 +1 @@
+export const TEKTON_HUB_API_ENDPOINT = 'https://api.hub.tekton.dev';


isn't this endpoint id there within operator, so that we can read and could be dynamic

Backend support for reading the endpoints is not there yet, so for now we are going with tektonhub API

frontend/packages/pipelines-plugin/src/components/catalog/hooks/useTaskCategories.ts

frontend/packages/pipelines-plugin/src/components/catalog/hooks/useTaskResources.ts

...tend/packages/pipelines-plugin/src/components/catalog/providers/useTekonHubTasksProvider.tsx

.../packages/pipelines-plugin/src/components/pipelines/pipeline-builder/PipelineBuilderForm.tsx

frontend/packages/pipelines-plugin/src/components/quicksearch/pipeline-quicksearch-utils.ts

invincibleJai · 2021-08-10T10:51:48Z

frontend/packages/pipelines-plugin/src/components/quicksearch/PipelineQuickSearch.tsx

+            }
+          } else {
+            resolve(savedCallback.current({ metadata: { name: item.data.name } }));
+            createTask(selectedVersionUrl, namespace).catch(() =>


no action needed on then here only on catch?

Yes, In case of successful task creation, the resources will be created and pulled in automatically by K8swatchers.

frontend/packages/pipelines-plugin/src/components/quicksearch/PipelineQuickSearchDetails.scss

andrewballantyne

Also we talked about the storage of the loading tasks, please make those changes as discussed.

frontend/packages/console-shared/src/components/quick-search/QuickSearchDetails.scss

andrewballantyne · 2021-08-10T13:53:40Z

frontend/packages/console-shared/src/components/quick-search/QuickSearchDetails.tsx

 }

-const QuickSearchDetails: React.FC<QuickSearchDetailsProps> = ({ selectedItem, closeModal }) => {
+const DefaultContent = ({ selectedItem, closeModal }) => {


We should type this with the props, it's a stand-alone function.

(props: QuickSearchDetailsRendererProps) => React.ReactNode maybe can be it's own type and we just reuse it here?

Also, maybe we make this lower case. It's not a component, it's a defaultContentRenderer. Perhaps the rename and go with camelCase as it's a function.

frontend/packages/console-shared/src/components/quick-search/QuickSearchList.tsx

...tend/packages/pipelines-plugin/src/components/catalog/providers/useTekonHubTasksProvider.tsx

...gin/src/components/pipelines/detail-page-tabs/pipeline-details/PipelineVisualizationTask.tsx

frontend/packages/pipelines-plugin/src/components/quicksearch/const.ts

frontend/packages/pipelines-plugin/src/components/quicksearch/PiplineQuickSearchDetails.tsx

andrewballantyne · 2021-08-10T16:12:46Z

frontend/packages/pipelines-plugin/src/components/quicksearch/PipelineQuickSearch.tsx

  React.useEffect(() => {
    savedCallback.current = callback;
  }, [callback]);


Just move this next to the useRef... you're just holding onto it and not doing any computation, no reason to use an effect.

andrewballantyne

Couple other things...

Somehow the Sidebar styles got mucked up. I think it was in your previous PR (or another PR that I'm unaware of that touched the Pipeline Builder):

Also not sure how it happened... I was clicking around in the modal and I got this:

I also got a ton of "Failed to fetch" errors from the useTekonHubTasksProvider.tsx file.

Your implementation also is not what we discussed. I'm not sure if that's because it didn't work or because you misunderstood. I don't want to hold up your PR beyond this sprint, if you can fix the UID issue above and other issues I mentioned, we can merge this as-is.

Seek another review for the LGTM when you get a chance, hopefully we can merge this before EOD tomorrow.

andrewballantyne · 2021-08-12T01:27:59Z

frontend/packages/pipelines-plugin/src/components/catalog/providers/useTasksProvider.tsx

+      const tags = annotations[TektonTaskAnnotation.tags]?.split(/\s*,\s*/) || [];
+      const categories = annotations[TektonTaskAnnotation.categories]?.split(/\s*,\s*/) || [];


Sorry I didn't see this before... but I wonder if we don't want to use parseJSONAnnotation... not sure if that's the goal of what you're trying to do or not.

I'm unsure what format these annotations are in...

annotations are strings, could be comma separated, so splitting the string into array.

andrewballantyne · 2021-08-12T02:20:14Z

frontend/packages/pipelines-plugin/src/types/pipeline.ts

@@ -42,6 +42,7 @@ export type PipelineResult = {
 };

 export type PipelineTask = {
+  metadata?: { installing?: boolean };


This is no longer needed, right?

Oops, missed this spot.

karthikjeeyar · 2021-08-12T07:29:40Z

Fixed TaskSidebar alignment issue caused by this removing bootstrap close styles commit 70a267f

Not able to reproduce the uid of undefined error, It seems like some flake happened in catalogServiceProvider while loading extensions.

I also got a ton of "Failed to fetch" errors from the useTekonHubTasksProvider.tsx file.

There were a lot of failures for /api/check-updates call, the errors are no longer appearing after building the backeend and re-sourcing it.

invincibleJai · 2021-08-12T10:59:53Z

frontend/packages/pipelines-plugin/src/components/pipelines/pipeline-builder/hooks.ts

+
+  React.useEffect(() => {
+    const { loadingTasks } = values.formData;
+    loadingTasks.map((task) => {


nit: will forEach be a better fit here

frontend/packages/pipelines-plugin/src/components/pipelines/pipeline-builder/utils.ts

invincibleJai · 2021-08-12T11:20:34Z

frontend/packages/pipelines-plugin/src/components/quicksearch/pipeline-quicksearch-utils.ts

+export const isTaskVersionInstalled = (item: CatalogItem): boolean => {
+  if (item.attributes?.installed) {
+    return item.attributes.installed !== '';
+  }
+  return false;
+};


nit: can we have it like this?

Suggested change

export const isTaskVersionInstalled = (item: CatalogItem): boolean => {

if (item.attributes?.installed) {

return item.attributes.installed !== '';

}

return false;

};

export const isTaskVersionInstalled = (item: CatalogItem): boolean => !!item.attributes?.installed;

invincibleJai · 2021-08-12T11:24:42Z

frontend/packages/pipelines-plugin/src/components/quicksearch/pipeline-quicksearch-utils.ts

+export const isInstalledNamespaceTask = (item: CatalogItem) => {
+  return (
+    item.data.kind === TaskModel.kind &&
+    item.data?.metadata?.annotations[TektonTaskAnnotation.installedFrom] === TEKTONHUB


Suggested change

item.data?.metadata?.annotations[TektonTaskAnnotation.installedFrom] === TEKTONHUB

item.data.metadata?.annotations[TektonTaskAnnotation.installedFrom] === TEKTONHUB

invincibleJai · 2021-08-12T11:27:20Z

frontend/packages/pipelines-plugin/src/components/quicksearch/pipeline-quicksearch-utils.ts

+  if (!item?.attributes?.versions) {
+    return null;
+  }
+  return item?.attributes?.versions.find((v) => v.id.toString() === version)?.rawURL;


Suggested change

return item?.attributes?.versions.find((v) => v.id.toString() === version)?.rawURL;

return item.attributes.versions.find((v) => v.id.toString() === version)?.rawURL;

invincibleJai · 2021-08-12T11:27:53Z

frontend/packages/pipelines-plugin/src/components/quicksearch/pipeline-quicksearch-utils.ts

+      i.name === item.name &&
+      item.data.kind !== ClusterTaskModel.kind &&
+      i.data.kind === TaskModel.kind &&
+      i.data?.metadata?.annotations[TektonTaskAnnotation.installedFrom] === TEKTONHUB,


same as above

invincibleJai · 2021-08-12T11:57:32Z

/approve
/label qe-approved

Verified the changes, works as expected cc @andrewballantyne

karthikjeeyar · 2021-08-12T12:19:12Z

/retest

andrewballantyne

/lgtm

karthikjeeyar · 2021-08-12T13:13:56Z

Adding on behalf of the docs & px team as the Epic already contains the acks.
/label px-approved
/label docs-approved

karthikjeeyar · 2021-08-13T15:20:17Z

Rebased the PR to resolve conflicts.

invincibleJai · 2021-08-14T04:43:22Z

/lgtm

openshift-ci · 2021-08-14T04:44:06Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: andrewballantyne, invincibleJai, karthikjeeyar

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~frontend/OWNERS~~ [andrewballantyne,invincibleJai]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

openshift-ci bot added the component/dev-console Related to dev-console label Aug 10, 2021

openshift-ci bot requested review from abhinandan13jan and dtaylor113 August 10, 2021 09:40

openshift-ci bot added component/pipelines Related to pipelines-plugin component/sdk Related to console-plugin-sdk component/shared Related to console-shared kind/i18n Indicates issue or PR relates to internationalization or has content that needs to be translated labels Aug 10, 2021

karthikjeeyar force-pushed the tekton-hub-integration branch from e938cfe to f83430b Compare August 10, 2021 10:02

invincibleJai reviewed Aug 10, 2021

View reviewed changes

karthikjeeyar force-pushed the tekton-hub-integration branch 2 times, most recently from 1ac673f to 040c056 Compare August 10, 2021 11:52

andrewballantyne suggested changes Aug 10, 2021

View reviewed changes

andrewballantyne suggested changes Aug 12, 2021

View reviewed changes

karthikjeeyar force-pushed the tekton-hub-integration branch from d0c5e77 to 8b6994a Compare August 12, 2021 07:21

karthikjeeyar force-pushed the tekton-hub-integration branch from 30045fb to 126ace2 Compare August 12, 2021 10:24

invincibleJai reviewed Aug 12, 2021

View reviewed changes

karthikjeeyar force-pushed the tekton-hub-integration branch from 126ace2 to 4296e7b Compare August 12, 2021 11:54

openshift-ci bot added qe-approved Signifies that QE has signed off on this PR approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Aug 12, 2021

andrewballantyne reviewed Aug 12, 2021

View reviewed changes

openshift-ci bot assigned andrewballantyne Aug 12, 2021

openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Aug 12, 2021

openshift-ci bot added px-approved Signifies that Product Support has signed off on this PR docs-approved Signifies that Docs has signed off on this PR needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Aug 12, 2021

karthikjeeyar added 4 commits August 13, 2021 20:42

Add tekton hub tasks to pipeline builder

6313f05

Add access review checks

3bb71a1

Add new loading task type in pipeline builder

f6f706d

Add unit test for pipeline quick search

12878fa

karthikjeeyar force-pushed the tekton-hub-integration branch from 4296e7b to 12878fa Compare August 13, 2021 15:18

openshift-ci bot removed lgtm Indicates that a PR is ready to be merged. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Aug 13, 2021

openshift-ci bot assigned invincibleJai Aug 14, 2021

openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Aug 14, 2021

openshift-ci bot merged commit 0a20908 into openshift:master Aug 14, 2021

spadgett added this to the v4.9 milestone Aug 30, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Tekton hub integration #9771

Tekton hub integration #9771

karthikjeeyar commented Aug 10, 2021 •

edited

invincibleJai Aug 10, 2021

karthikjeeyar Aug 10, 2021

invincibleJai Aug 10, 2021

karthikjeeyar Aug 10, 2021

andrewballantyne left a comment

andrewballantyne Aug 10, 2021

andrewballantyne Aug 10, 2021

andrewballantyne left a comment

andrewballantyne Aug 12, 2021

karthikjeeyar Aug 12, 2021

andrewballantyne Aug 12, 2021

karthikjeeyar Aug 12, 2021

karthikjeeyar commented Aug 12, 2021

invincibleJai Aug 12, 2021

invincibleJai Aug 12, 2021

invincibleJai Aug 12, 2021

invincibleJai Aug 12, 2021

invincibleJai Aug 12, 2021

invincibleJai commented Aug 12, 2021

karthikjeeyar commented Aug 12, 2021

andrewballantyne left a comment

karthikjeeyar commented Aug 12, 2021

karthikjeeyar commented Aug 13, 2021

invincibleJai commented Aug 14, 2021

openshift-ci bot commented Aug 14, 2021

		@@ -0,0 +1 @@
		export const TEKTON_HUB_API_ENDPOINT = 'https://api.hub.tekton.dev';

		const tags = annotations[TektonTaskAnnotation.tags]?.split(/\s,\s/) \|\| [];
		const categories = annotations[TektonTaskAnnotation.categories]?.split(/\s,\s/) \|\| [];

	item.data?.metadata?.annotations[TektonTaskAnnotation.installedFrom] === TEKTONHUB
	item.data.metadata?.annotations[TektonTaskAnnotation.installedFrom] === TEKTONHUB

	return item?.attributes?.versions.find((v) => v.id.toString() === version)?.rawURL;
	return item.attributes.versions.find((v) => v.id.toString() === version)?.rawURL;

Tekton hub integration #9771

Tekton hub integration #9771

Conversation

karthikjeeyar commented Aug 10, 2021 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

andrewballantyne left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

andrewballantyne left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

karthikjeeyar commented Aug 12, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

invincibleJai commented Aug 12, 2021

karthikjeeyar commented Aug 12, 2021

andrewballantyne left a comment

Choose a reason for hiding this comment

karthikjeeyar commented Aug 12, 2021

karthikjeeyar commented Aug 13, 2021

invincibleJai commented Aug 14, 2021

openshift-ci bot commented Aug 14, 2021

karthikjeeyar commented Aug 10, 2021 •

edited