Switch backend plugins to dynamic, in a way compatible with upcoming …

…frontend dynamic plugins. (janus-idp#631) * Switch to dynamic plugin: @roadiehq/scaffolder-backend-module-utils Signed-off-by: David Festal <dfestal@redhat.com> * Switch to dynamic plugin: @roadiehq/scaffolder-backend-argocd * Switch to dynamic plugin: @backstage/plugin-catalog-backend-module-gitlab * Switch to dynamic plugin: @backstage/plugin-scaffolder-backend-module-gitlab * Switch to dynamic plugin: @immobiliarelabs/backstage-plugin-gitlab-backend Signed-off-by: David Festal <dfestal@redhat.com> * Switch to dynamic plugin: @backstage/plugin-catalog-backend-module-github Signed-off-by: David Festal <dfestal@redhat.com> * Switch to dynamic plugin: @janus-idp/backstage-plugin-keycloak-backend Signed-off-by: David Festal <dfestal@redhat.com> * Switch to dynamic plugin: @janus-idp/backstage-plugin-ocm-backend * Switch to dynamic plugin: @janus-idp/backstage-plugin-aap-backend * Switch to dynamic plugin: @backstage/plugin-techdocs-backend * Switch to dynamic plugin: @roadiehq/backstage-plugin-argo-cd-backend Signed-off-by: David Festal <dfestal@redhat.com> * Switch to dynamic plugin: @backstage/plugin-azure-devops-backend Signed-off-by: David Festal <dfestal@redhat.com> * Switch to dynamic plugin: @backstage/plugin-kubernetes-backend Signed-off-by: David Festal <dfestal@redhat.com> * Switch to dynamic plugin: @backstage/plugin-jenkins-backend Signed-off-by: David Festal <dfestal@redhat.com> * Switch to dynamic plugin: @backstage/plugin-sonarqube-backend Signed-off-by: David Festal <dfestal@redhat.com> * Complete dynamic-plugins.default.yaml Signed-off-by: David Festal <dfestal@redhat.com> * Workaround for the watching bug Signed-off-by: David Festal <dfestal@redhat.com> * Support imported (non-wrapped) dynamic plugins... ...in the Showcase build and docker build in a consistent way. This involves moving (and renaming) all the wrapped dynamic plugins from the `dynamic-plugins` directory to the `dynamic-plugins/wrappers`. Signed-off-by: David Festal <dfestal@redhat.com> * Add changeset Signed-off-by: David Festal <dfestal@redhat.com> --------- Signed-off-by: David Festal <dfestal@redhat.com>
RedHatInsights · Oct 20, 2023 · ce85b16 · ce85b16
1 parent 7188844
commit ce85b16
Show file tree

Hide file tree

Showing 166 changed files with 35,608 additions and 521 deletions.
diff --git a/.changeset/spotty-cups-hear.md b/.changeset/spotty-cups-hear.md
@@ -0,0 +1,20 @@
+---
+'backstage-plugin-scaffolder-backend-module-gitlab': patch
+'backstage-plugin-catalog-backend-module-github': patch
+'backstage-plugin-catalog-backend-module-gitlab': patch
+'janus-idp-backstage-plugin-keycloak-backend': patch
+'roadiehq-backstage-plugin-argo-cd-backend': patch
+'immobiliarelabs-backstage-plugin-gitlab-backend': patch
+'roadiehq-scaffolder-backend-module-utils': patch
+'janus-idp-backstage-plugin-aap-backend': patch
+'janus-idp-backstage-plugin-ocm-backend': patch
+'backstage-plugin-azure-devops-backend': patch
+'backstage-plugin-kubernetes-backend': patch
+'roadiehq-scaffolder-backend-argocd': patch
+'backstage-plugin-sonarqube-backend': patch
+'backstage-plugin-techdocs-backend': patch
+'backstage-plugin-jenkins-backend': patch
+'backend': patch
+---
+
+Switch static backend plugins to dynamic plugins
diff --git a/.gitignore b/.gitignore
@@ -56,4 +56,5 @@ site
 # Dynamic plugins root content
 dynamic-plugins-root/*
 !dynamic-plugins-root/.gitkeep
-dynamic-plugins/*/dist-dynamic/src
+dynamic-plugins/wrappers/*/dist-dynamic/src
+dynamic-plugins/imports/*/
diff --git a/docker/Dockerfile b/docker/Dockerfile
@@ -68,17 +68,7 @@ RUN $YARN build --filter=backend
 
 # Build dynamic plugins
 RUN $YARN export-dynamic
-RUN $YARN clean-dynamic-sources
-RUN mkdir -p dynamic-plugins-root && \
-    cd dynamic-plugins-root && \
-    rm -Rf * && \
-    for pkg in $CONTAINER_SOURCE/dynamic-plugins/*/dist-dynamic; do \
-      if [ -d $pkg ]; then \
-        archive=$(npm pack $pkg) && \
-        tar -xzf "$archive" && rm "$archive" && \
-        mv package $(echo $archive | sed -e 's:\.tgz$::'); \
-      fi; \
-    done
+RUN $YARN copy-dynamic-plugins dist
 
 # Stage 4 - Build the actual backend image and install production dependencies
 FROM skeleton AS cleanup
@@ -128,12 +118,11 @@ COPY docker/install-dynamic-plugins.py ./
 RUN chmod a+r ./install-dynamic-plugins.py
 
 # Copy embedded dynamic plugins
-COPY --from=build $CONTAINER_SOURCE/dynamic-plugins/ ./dynamic-plugins/
-RUN chmod -R a+r ./dynamic-plugins/
+COPY --from=build $CONTAINER_SOURCE/dynamic-plugins/dist/ ./dynamic-plugins/dist/
+RUN chmod -R a+r ./dynamic-plugins/dist/
 
-# Copy default dynamic plugins root
-COPY --from=build $CONTAINER_SOURCE/dynamic-plugins-root/ ./dynamic-plugins-root/
-RUN chmod -R a+r ./dynamic-plugins-root/
+# Copy embedded dynamic plugins to default dynamic plugins root
+RUN rm -f dynamic-plugins-root && cp -R dynamic-plugins/dist/ dynamic-plugins-root
 
 # The fix-permissions script is important when operating in environments that dynamically use a random UID at runtime, such as OpenShift.
 # The upstream backstage image does not account for this and it causes the container to fail at runtime.
@@ -142,6 +131,10 @@ RUN fix-permissions ./
 # Switch to nodejs user
 USER 1001
 
+# Temporary workaround to avoid triggering issue
+# https://github.com/backstage/backstage/issues/20644
+ENV CHOKIDAR_USEPOLLING='1' CHOKIDAR_INTERVAL='10000'
+
 ENTRYPOINT ["node", "packages/backend", "--config", "app-config.yaml", "--config", "app-config.example.yaml", "--config", "app-config.example.production.yaml"]
 
 # append Brew metadata here
diff --git a/docker/brew.Dockerfile b/docker/brew.Dockerfile
@@ -110,17 +110,11 @@ RUN $YARN build --filter=backend
 
 # Build dynamic plugins
 RUN $YARN export-dynamic
-RUN $YARN clean-dynamic-sources
-RUN mkdir -p dynamic-plugins-root && \
-    cd dynamic-plugins-root && \
-    rm -Rf * && \
-    for pkg in $CONTAINER_SOURCE/dynamic-plugins/*/dist-dynamic; do \
-      if [ -d $pkg ]; then \
-        archive=$(npm pack $pkg) && \
-        tar -xzf "$archive" && rm "$archive" && \
-        mv package $(echo $archive | sed -e 's:\.tgz$::'); \
-      fi; \
-    done
+RUN $YARN copy-dynamic-plugins dist
+
+# Cleanup dynamic plugins sources
+# Downstream only
+RUN find dynamic-plugins -type f -not -name 'dist' -delete
 
 # Stage 4 - Build the actual backend image and install production dependencies
 
@@ -181,12 +175,11 @@ COPY docker/install-dynamic-plugins.py ./
 RUN chmod a+r ./install-dynamic-plugins.py
 
 # Copy embedded dynamic plugins
-COPY --from=builder $CONTAINER_SOURCE/dynamic-plugins/ ./dynamic-plugins/
+COPY --from=builder $CONTAINER_SOURCE/dynamic-plugins/dist ./dynamic-plugins/dist
 RUN chmod -R a+r ./dynamic-plugins/
 
-# Copy default dynamic plugins root
-COPY --from=build $CONTAINER_SOURCE/dynamic-plugins-root/ ./dynamic-plugins-root/
-RUN chmod -R a+r ./dynamic-plugins-root/
+# Copy embedded dynamic plugins to default dynamic plugins root
+RUN rm -f dynamic-plugins-root && cp -R dynamic-plugins/dist/ dynamic-plugins-root
 
 # The fix-permissions script is important when operating in environments that dynamically use a random UID at runtime, such as OpenShift.
 # The upstream backstage image does not account for this and it causes the container to fail at runtime.
@@ -195,6 +188,10 @@ RUN fix-permissions ./
 # Switch to nodejs user
 USER 1001
 
+# Temporary workaround to avoid triggering issue
+# https://github.com/backstage/backstage/issues/20644
+ENV CHOKIDAR_USEPOLLING='1' CHOKIDAR_INTERVAL='10000'
+
 ENTRYPOINT ["node", "packages/backend", "--config", "app-config.yaml", "--config", "app-config.example.yaml", "--config", "app-config.example.production.yaml"]
 
 # append Brew metadata here
diff --git a/dynamic-plugins.default.yaml b/dynamic-plugins.default.yaml
@@ -1 +1,185 @@
-- package: ./dynamic-plugins/scaffolder-backend-module-utils-wrapped/dist-dynamic
+- package: ./dynamic-plugins/dist/roadiehq-scaffolder-backend-module-utils-dynamic
+
+- package: ./dynamic-plugins/dist/janus-idp-backstage-plugin-aap-backend-dynamic
+  pluginConfig:
+    enabled:
+      aap: true
+    catalog:
+      providers:
+        aap:
+          prod:
+            baseUrl: '${AAP_BASE_URL}'
+            authorization: "${AAP_AUTH_TOKEN}"
+
+- package: ./dynamic-plugins/dist/immobiliarelabs-backstage-plugin-gitlab-backend-dynamic
+  pluginConfig:
+    enabled:
+      gitlab: true
+    gitlab:
+      host: ${GITLAB_HOST}
+      token: ${GITLAB_TOKEN}
+
+- package: ./dynamic-plugins/dist/janus-idp-backstage-plugin-keycloak-backend-dynamic
+  pluginConfig:
+    enabled:
+      keycloak: true
+    catalog:
+      providers:
+        keycloakOrg:
+          default:
+            baseUrl: "${KEYCLOAK_BASE_URL}"
+            loginRealm: "${KEYCLOAK_LOGIN_REALM}"
+            realm: "${KEYCLOAK_REALM}"
+            clientId: "${KEYCLOAK_CLIENT_ID}"
+            clientSecret: "${KEYCLOAK_CLIENT_SECRET}"
+
+- package: ./dynamic-plugins/dist/janus-idp-backstage-plugin-ocm-backend-dynamic
+  pluginConfig:
+    enabled:
+      ocm: true
+    catalog:
+      providers:
+        ocm:
+          default:
+            name: "${OCM_HUB_NAME}"
+            url: "${OCM_HUB_URL}"
+            serviceAccountToken: "${moc_infra_token}"
+            owner: janus-authors
+
+- package: ./dynamic-plugins/dist/backstage-plugin-catalog-backend-module-github-dynamic
+  pluginConfig:
+    enabled:
+      github: true
+      githubOrg: true
+    catalog:
+      providers:
+        githubOrg:
+          default:
+            id: production
+            orgUrl: "${GITHUB_ORG_URL}"
+
+        # Plugin: GitHub Discovery
+        # uncomment following lines and set GITHUB_ORG to enable GitHub Discovery
+        # github:
+        #   providerId:
+        #     organization: "${GITHUB_ORG}"
+
+- package: ./dynamic-plugins/dist/backstage-plugin-catalog-backend-module-gitlab-dynamic
+  pluginConfig:
+    enabled:
+      gitlab: true
+    catalog:
+      providers:
+        gitlab: {}
+
+- package: ./dynamic-plugins/dist/backstage-plugin-scaffolder-backend-module-gitlab-dynamic
+
+- package: ./dynamic-plugins/dist/roadiehq-backstage-plugin-argo-cd-backend-dynamic
+  pluginConfig:
+    enabled:
+      argocd: true
+    argocd:
+      username: "${ARGOCD_USERNAME}"
+      password: "${ARGOCD_PASSWORD}"
+      appLocatorMethods:
+        - type: 'config'
+          instances:
+            - name: argoInstance1
+              url: "${ARGOCD_INSTANCE1_URL}"
+              token: "${ARGOCD_AUTH_TOKEN}"
+            - name: argoInstance2
+              url: "${ARGOCD_INSTANCE2_URL}"
+              token: "${ARGOCD_AUTH_TOKEN2}"
+
+- package: ./dynamic-plugins/dist/roadiehq-scaffolder-backend-argocd-dynamic
+  pluginConfig:
+    enabled:
+      argocd: true
+    argocd:
+      username: "${ARGOCD_USERNAME}"
+      password: "${ARGOCD_PASSWORD}"
+      appLocatorMethods:
+        - type: 'config'
+          instances:
+            - name: argoInstance1
+              url: "${ARGOCD_INSTANCE1_URL}"
+              token: "${ARGOCD_AUTH_TOKEN}"
+            - name: argoInstance2
+              url: "${ARGOCD_INSTANCE2_URL}"
+              token: "${ARGOCD_AUTH_TOKEN2}"
+
+- package: ./dynamic-plugins/dist/backstage-plugin-azure-devops-backend-dynamic
+  pluginConfig:
+    enabled:
+      azureDevOps: true
+    azureDevOps:
+      host: dev.azure.com
+      token: ${AZURE_TOKEN}
+      organization: ${AZURE_ORG}
+
+- package: ./dynamic-plugins/dist/backstage-plugin-jenkins-backend-dynamic
+  pluginConfig:
+    enabled:
+      jenkins: true
+    jenkins:
+      instances:
+        - name: default
+          baseUrl: ${JENKINS_URL}
+          username: ${JENKINS_USERNAME}
+          apiKey: ${JENKINS_TOKEN}
+
+- package: ./dynamic-plugins/dist/backstage-plugin-kubernetes-backend-dynamic
+  pluginConfig:
+    enabled:
+      kubernetes: true
+    kubernetes:
+      customResources:
+        - group: 'tekton.dev'
+          apiVersion: 'v1beta1'
+          plural: 'pipelines'
+        - group: 'tekton.dev'
+          apiVersion: 'v1beta1'
+          plural: 'pipelineruns'
+        - group: 'tekton.dev'
+          apiVersion: 'v1beta1'
+          plural: 'taskruns'
+        - group: 'route.openshift.io'
+          apiVersion: 'v1'
+          plural: 'routes'
+      serviceLocatorMethod:
+        type: 'multiTenant'
+      clusterLocatorMethods:
+        - type: 'config'
+          clusters:
+            - name: ${K8S_CLUSTER_NAME}
+              url: ${K8S_CLUSTER_URL}
+              authProvider: 'serviceAccount'
+              skipTLSVerify: true
+              serviceAccountToken: ${K8S_CLUSTER_TOKEN}
+
+- package: ./dynamic-plugins/dist/backstage-plugin-sonarqube-backend-dynamic
+  pluginConfig:
+    enabled:
+      sonarqube: true
+    sonarqube:
+      baseUrl: ${SONARQUBE_URL}
+      apiKey: ${SONARQUBE_TOKEN}
+
+- package: ./dynamic-plugins/dist/backstage-plugin-techdocs-backend-dynamic
+  pluginConfig:
+    enabled:
+      techdocs: true
+    techdocs:
+      builder: ${TECHDOCS_BUILDER_TYPE}
+      generator:
+        runIn: ${TECHDOCS_GENERATOR_TYPE}
+      publisher:
+        type: ${TECHDOCS_PUBLISHER_TYPE}
+        awsS3:
+          bucketName: ${BUCKET_NAME}
+          region: ${BUCKET_REGION_VAULT}
+          endpoint: ${BUCKET_URL}
+          s3ForcePathStyle: true
+          credentials:
+            accessKeyId: ${AWS_ACCESS_KEY_ID}
+            secretAccessKey: ${AWS_SECRET_ACCESS_KEY}
diff --git a/dynamic-plugins/_utils/.eslintrc.js b/dynamic-plugins/_utils/.eslintrc.js
@@ -0,0 +1 @@
+module.exports = require('@backstage/cli/config/eslint-factory')(__dirname);
diff --git a/dynamic-plugins/_utils/copy-plugins.js b/dynamic-plugins/_utils/copy-plugins.js
@@ -0,0 +1,59 @@
+const fs = require('fs');
+const path = require('path');
+const process = require('process');
+
+process.chdir(path.join(__dirname, '..'));
+
+if (!process.argv[2]) {
+  console.log('Usage: node copy-plugins.js <destination directory>');
+  process.exit(1);
+}
+
+const destination = process.argv[2];
+fs.mkdirSync(destination, { recursive: true });
+
+const wrappersDir = path.join('.', 'wrappers');
+const importsDir = path.join('.', 'imports');
+
+const wrappers = fs
+  .readdirSync(wrappersDir, {
+    withFileTypes: true,
+  })
+  .filter(dirent => dirent.isDirectory())
+  .map(dirent => path.join(wrappersDir, dirent.name));
+
+const imports = fs
+  .readdirSync(importsDir, {
+    withFileTypes: true,
+  })
+  .filter(dirent => dirent.isDirectory())
+  .map(dirent => path.join(importsDir, dirent.name));
+
+for (const directory of [...imports, ...wrappers]) {
+  const distDynamicDir = path.join(directory, 'dist-dynamic');
+
+  const packageToCopy = fs.existsSync(distDynamicDir)
+    ? distDynamicDir
+    : directory;
+
+  const pkgJsonPath = path.join(packageToCopy, 'package.json');
+  if (!fs.existsSync(pkgJsonPath)) {
+    console.error(`No 'package.json' in '${directory}': skipping`);
+    continue;
+  }
+  const pkgJson = require(fs.realpathSync(pkgJsonPath));
+  if (!pkgJson || !pkgJson.name) {
+    console.error(
+      `Package '${directory}' is missing 'package.json' or 'name' attribute.`,
+    );
+    continue;
+  }
+  const copyName = pkgJson.name.replace(/^@/, '').replace(/\//, '-');
+
+  console.log(`Copying ${packageToCopy} to ${destination}`);
+  const destinationDir = path.join(destination, copyName);
+  fs.rmSync(destinationDir, { recursive: true, force: true });
+  fs.cpSync(fs.realpathSync(packageToCopy), destinationDir, {
+    recursive: true,
+  });
+}
diff --git a/dynamic-plugins/_utils/package.json b/dynamic-plugins/_utils/package.json
@@ -0,0 +1,16 @@
+{
+  "name": "dynamic-plugins-utils",
+  "version": "0.0.0",
+  "license": "Apache-2.0",
+  "private": true,
+  "backstage": {
+    "role": "cli"
+  },
+  "scripts": {
+    "copy-dynamic-plugins": "node ./copy-plugins.js",
+    "lint": "backstage-cli package lint"
+  },
+  "devDependencies": {
+    "@backstage/cli": "0.22.13"
+  }
+}
diff --git a/dynamic-plugins/imports/.eslintignore b/dynamic-plugins/imports/.eslintignore
@@ -0,0 +1 @@
+/*/*
diff --git a/dynamic-plugins/imports/.eslintrc.js b/dynamic-plugins/imports/.eslintrc.js
@@ -0,0 +1 @@
+module.exports = require('@backstage/cli/config/eslint-factory')(__dirname);