Feature: add build caching #14

jacobfogolyan · 2024-04-22T07:49:00Z

Here is what my continuous delivery file looks like


name: Deployment

env:
  CUSTOMER_NAME: jacob-pipeline
  REGION: europe-west1 # australia-southeast1 | us-central1
  PROD_BRANCH: main
  STAGE_BRANCH: stage
  DEV_BRANCH: dev
  PROD_INSTANCE: prod
  STAGE_INSTANCE: stage
  DEV_INSTANCE: dev
  DOCKER_REGISTRY_URL: registry.storefrontcloud.io

on:
  workflow_dispatch:
  push:
    branches:
      - dev
      - stage
      - main

concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

jobs:
  setup:
    runs-on: ubuntu-latest
    outputs:
      instance: ${{ steps.instance_step.outputs.instance }}
      gh_environment: ${{ steps.instance_step.outputs.gh_environment }}
      date: ${{ steps.instance_step.outputs.date }}
    steps:
      # Function to associate branch with environment
      - uses: actions/checkout@v4
      - name: Set Instance variable
        id: instance_step
        run: |
          if [ ${{ github.ref }} == 'refs/heads/${{ env.PROD_BRANCH }}' ]; then
            echo "instance=${{ env.PROD_INSTANCE }}" >> $GITHUB_OUTPUT
            echo "gh_environment=Production" >> $GITHUB_OUTPUT
          elif [ ${{ github.ref }} == 'refs/heads/${{ env.STAGE_BRANCH }}' ]; then
            echo "instance=${{ env.STAGE_INSTANCE }}" >> $GITHUB_OUTPUT
            echo "gh_environment=Staging" >> $GITHUB_OUTPUT
          elif [ ${{ github.ref }} == 'refs/heads/${{ env.DEV_BRANCH }}' ]; then
            echo "instance=${{ env.DEV_INSTANCE }}" >> $GITHUB_OUTPUT
            echo "gh_environment=Develop" >> $GITHUB_OUTPUT
          else
            echo "instance=${{ env.DEV_INSTANCE }}" >> $GITHUB_OUTPUT
            echo "gh_environment=Develop" >> $GITHUB_OUTPUT
          fi
          echo "date=${{ github.sha }}-$(date -u +%Y-%m-%dt%H%M%S)" >> $GITHUB_ENV
        shell: bash

  build-frontend:
    name: Build Frontend
    runs-on: ubuntu-latest
    needs: setup
    environment: ${{ needs.setup.outputs.gh_environment }}
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Build
        # remember to put @1.2.3 version number at end
        uses: ./.github/actions/build-frontend
        with:
          # Change frontend to desired one
          frontend: ${{ secrets.FRONTEND_FRAMEWORK || 'next' }}
          docker_registry_url: ${{ env.DOCKER_REGISTRY_URL }}
          docker_registry_ref: ${{ env.DOCKER_REGISTRY_URL }}/${{ env.CUSTOMER_NAME }}-storefrontcloud-io
          project_name: ${{ env.CUSTOMER_NAME }}
          cloud_username: ${{ secrets.CLOUD_USERNAME }}
          cloud_password: ${{ secrets.CLOUD_PASSWORD }}
          cloud_region: ${{ secrets.CLOUD_REGION }}
          npm_email: ${{ secrets.NPM_EMAIL }}
          npm_user: ${{ secrets.NPM_USER }}
          npm_pass: ${{ secrets.NPM_PASS }}
          version: ${{ github.sha }}-${{ needs.setup.ouputs.date }}

  build-middleware:
    name: Build Middleware
    runs-on: ubuntu-latest
    needs: setup
    environment: ${{ needs.setup.outputs.gh_environment }}
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Build
        uses: ./.github/actions/build-middleware
        with:
          docker_registry_url: ${{ env.DOCKER_REGISTRY_URL }}
          docker_registry_ref: ${{ env.DOCKER_REGISTRY_URL }}/${{ env.CUSTOMER_NAME }}-storefrontcloud-io
          project_name: ${{ env.CUSTOMER_NAME }}
          cloud_username: ${{ secrets.CLOUD_USERNAME }}
          cloud_password: ${{ secrets.CLOUD_PASSWORD }}
          npm_email: ${{ secrets.NPM_EMAIL }}
          npm_user: ${{ secrets.NPM_USER }}
          npm_pass: ${{ secrets.NPM_PASS }}
          version: ${{ github.sha }}-${{ needs.setup.ouputs.date }}

  deploy:
    name: Deploy
    runs-on: ubuntu-latest
    permissions:
      contents: read
      deployments: write
    needs: [build-frontend, build-middleware]
    environment: ${{ needs.setup.outputs.gh_environment }}
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Deploy
        uses: ./.github/actions/deploy
        with:
          # review object payloads here: https://api.console.alokai.com/api#/cloud/CloudController_deploy
          console_api_url: ${{ secrets.CONSOLE_API_URL }}
          docker_registry_url: ${{ env.DOCKER_REGISTRY_URL }}
          project_name: ${{ env.CUSTOMER_NAME }}
          cloud_username: ${{ secrets.CLOUD_USERNAME }}
          cloud_password: ${{ secrets.CLOUD_PASSWORD }}
          cloud_region: ${{ secrets.CLOUD_REGION }}
          version: ${{ github.sha }}-${{ needs.setup.ouputs.date }}

🔗 Linked issue

❓ Type of change

📖 Documentation (updates to the documentation, readme or JSDoc annotations)
🐞 Bug fix (a non-breaking change that fixes an issue)
👌 Enhancement (improving an existing functionality like performance)
✨ New feature (a non-breaking change that adds functionality)
🧹 Chore (updates to the build process or auxiliary tools and libraries)
⚠️ Breaking change (fix or feature that would cause existing functionality to change)

📚 Description

📝 Checklist

I have linked an issue or discussion.
I have updated the documentation accordingly.

jacobfogolyan · 2024-04-22T07:54:50Z

build-middleware/action.yml

@@ -33,23 +36,29 @@ inputs:
 runs:
  using: "composite"
  steps:
+    # Need buildx for caching
+    - name: Set up Docker Buildx


need to define buildx before caching happens

jacobfogolyan · 2024-04-22T07:57:02Z

deploy/action.yml

-        customHeaders: '{"Content-Type":"application/json"}'
-        data: '{ "middleware":true, "cloudUserId":"${{ inputs.cloud_username }}", "cloudUserPassword":"${{ inputs.cloud_password }}", "dockerImageHash":"${{ inputs.version || github.sha }}", "middlewareData":{ "path":"/api/", "port":4000, "has_base_path":false } }'
-        timeout: 60000
+    - name: Deploy on ${{ inputs.project_name }}.${{ inputs.cloud_region }}.gcp.storefrontcloud.io


i changed this because this code is what the SA's are familiar with. What do you think?

Deployments must be triggered via the Console API. The firing it through the Farmer is a deprecated way.

Triggering it via Farmer, erases the environment variables and there's lack of deployment markers in the Console — so part of their functionality is broken.

grixu · 2024-04-22T12:55:49Z

build-frontend/action.yml

-        NUXT_PUBLIC_MULTISTORE_ENABLED: ${{ inputs.multistore_enabled }}
-        NUXT_IMAGE_PROVIDER: ${{ inputs.image_provider }}
-        NUXT_PUBLIC_IMAGE_LOADER_UPLOAD_URL: ${{ inputs.image_provider_upload_url }}
-        NUXT_PUBLIC_IMAGE_LOADER_FETCH_URL: ${{ inputs.image_provider_fetch_url }}
-        NUXT_PUBLIC_COVEO_ORGANIZATION_ID: ${{ inputs.coveo_organization_id }}
-        NUXT_PUBLIC_COVEO_ACCESS_TOKEN: ${{ inputs.coveo_access_token }}


Until we release the upgrade to the Next 14 & App Router, those env variables must stay for both of the frontends - we would rather not introduce the differences between them in the deployment

grixu · 2024-04-22T12:57:03Z

build-frontend/action.yml

+          NPM_EMAIL=${{ inputs.npm_email }}
+          NPM_PASS=${{ inputs.npm_pass }}
+          NPM_USER=${{ inputs.npm_user }}
+          NPM_REGISTRY=https://registrynpm.storefrontcloud.io
+          NUXT_PUBLIC_API_BASE_URL: https://${{ inputs.project_name }}.${{ inputs.cloud_region }}.gcp.storefrontcloud.io/api/


If we're passing the build args this way, I guess that we don't have to declare the envs key below.

grixu · 2024-04-22T12:58:25Z

build-frontend/action.yml

+        context: .
+        file: .vuestorefrontcloud/docker/nuxtjs/Dockerfile-frontend
+        push: true
+        tags: ${{ inputs.docker_registry_ref }}/vue-storefront:${{ github.sha }}


Here we need a space for custom versioning: ${{ inputs.version || github.sha }}

build-frontend/action.yml

grixu · 2024-04-22T13:01:22Z

build-frontend/action.yml

    required: false
-    default: 'registry.vuestorefront.cloud'
+    default: "registry.storefrontcloud.io"
+  docker_registry_ref:


Can't we by default compute the value? It should be like:
${{ inputs.project_name }}-storefrontcloud-io

it won't build if i use this change unfortunately. i can't seem to reference inputs within the inputs section of this file. it has to be hard coded.

However within the codebase the continuous-delivery.yml file looks like this

docker_registry_ref: ${{ env.DOCKER_REGISTRY_URL }}/${{ env.CUSTOMER_NAME }}-storefrontcloud-io

grixu · 2024-04-22T13:03:45Z

deploy/action.yml

-        customHeaders: '{"Content-Type":"application/json"}'
-        data: '{ "middleware":true, "cloudUserId":"${{ inputs.cloud_username }}", "cloudUserPassword":"${{ inputs.cloud_password }}", "dockerImageHash":"${{ inputs.version || github.sha }}", "middlewareData":{ "path":"/api/", "port":4000, "has_base_path":false } }'
-        timeout: 60000
+    - name: Deploy on ${{ inputs.project_name }}.${{ inputs.cloud_region }}.gcp.storefrontcloud.io


Deployments must be triggered via the Console API. The firing it through the Farmer is a deprecated way.

grixu · 2024-04-22T13:05:14Z

deploy/action.yml

-        customHeaders: '{"Content-Type":"application/json"}'
-        data: '{ "middleware":true, "cloudUserId":"${{ inputs.cloud_username }}", "cloudUserPassword":"${{ inputs.cloud_password }}", "dockerImageHash":"${{ inputs.version || github.sha }}", "middlewareData":{ "path":"/api/", "port":4000, "has_base_path":false } }'
-        timeout: 60000
+    - name: Deploy on ${{ inputs.project_name }}.${{ inputs.cloud_region }}.gcp.storefrontcloud.io


Triggering it via Farmer, erases the environment variables and there's lack of deployment markers in the Console — so part of their functionality is broken.

build-middleware/action.yml

chore: add frontend caching

15d0b29

jacobfogolyan requested review from skirianov and a team as code owners April 22, 2024 07:49

jacobfogolyan added 2 commits April 22, 2024 17:52

feat: add middleware caching

77c72ce

chore: update deploy script

99408c8

jacobfogolyan commented Apr 22, 2024

View reviewed changes

grixu requested changes Apr 22, 2024

View reviewed changes

chore: change tag name to version and fix curl req

ebc2d54

jacobfogolyan changed the title ~~chore: add frontend caching~~ Feature: add build caching Apr 24, 2024

jacobfogolyan added 4 commits April 24, 2024 11:42

chore: re-add nuxt env vars

e80c2cd

chore: build caching for middleware action

3799929

chore: modify deploy script to be more legible

81b509e

chore: set npm registry as inputs

fd3987d

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feature: add build caching #14

Feature: add build caching #14

jacobfogolyan commented Apr 22, 2024 •

edited

jacobfogolyan Apr 22, 2024

jacobfogolyan Apr 22, 2024

grixu Apr 22, 2024

grixu Apr 22, 2024

grixu Apr 22, 2024

grixu Apr 22, 2024 •

edited

grixu Apr 22, 2024

grixu Apr 22, 2024

jacobfogolyan Jun 13, 2024

grixu Apr 22, 2024

grixu Apr 22, 2024

Feature: add build caching #14

Are you sure you want to change the base?

Feature: add build caching #14

Conversation

jacobfogolyan commented Apr 22, 2024 • edited

🔗 Linked issue

❓ Type of change

📚 Description

📝 Checklist

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

grixu Apr 22, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jacobfogolyan commented Apr 22, 2024 •

edited

grixu Apr 22, 2024 •

edited