Skip to content

[#1918] Updated CI config to export and restore exported codebase only for artifact deployment. #1924

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
AlexSkrypnyk merged 1 commit into from
Aug 22, 2025
Merged

[#1918] Updated CI config to export and restore exported codebase only for artifact deployment. #1924

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 11 additions & 2 deletions .circleci/config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,6 @@ aliases:
# Change to 'large' for faster builds.
resource_class: medium

# Set up remote Docker.
- &step_setup_remote_docker
setup_remote_docker:
# Docker Layer Caching allows to significantly speed up builds by caching
Expand All @@ -91,14 +90,18 @@ aliases:
docker_layer_caching: false
version: default

# Process the codebase to be run in CI environment.
- &step_process_codebase_for_ci
run:
name: Process codebase to run in CI
command: |
find . -name "docker-compose.yml" -print0 | xargs -0 -I {} sh -c "sed -i -e ''/###/d'' {} && sed -i -e ''s/##//'' {}"
mkdir -p /tmp/workspace/code

- &load_variables_from_dotenv
run:
name: Load environment variables from .env file
# Load variables from .env file, respecting existing values, and make them available for the next steps.
command: t=$(mktemp) && export -p >"${t}" && set -a && . ./.env && set +a && . "${t}" && export -p >> "$BASH_ENV"

Comment on lines +100 to 105
Copy link

@coderabbitai coderabbitai bot Aug 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Fix unsafe/ineffective .env loader anchor; export only .env keys to $BASH_ENV

Current anchor:

  • Re-sources the pre-.env snapshot, undoing .env values.
  • Appends the entire exported environment to $BASH_ENV, risking pollution and quoting issues.
  • Triggers SC1090 due to sourcing .env.

Use a safe parser that writes only .env variables to $BASH_ENV with proper quoting. This also removes the need to ignore SC1090 elsewhere.

Apply:

   - &load_variables_from_dotenv
     run:
-      name: Load environment variables from .env file
-      # Load variables from .env file, respecting existing values, and make them available for the next steps.
-      command: t=$(mktemp) && export -p >"${t}" && set -a && . ./.env && set +a && . "${t}" && export -p >> "$BASH_ENV"
+      name: Load environment variables from .env file (safe)
+      command: |
+        if [ -f .env ]; then
+          # Export only keys from .env; ignore comments/blank lines.
+          while IFS='=' read -r key rest; do
+            [ -z "${key}" ] && continue
+            case "${key}" in \#* ) continue;; esac
+            key="${key%%[[:space:]]*}"
+            # Safely quote for BASH_ENV
+            printf 'export %s=%q\n' "$key" "$rest" >> "$BASH_ENV"
+          done < .env
+        fi
+        # Optional: names-only debug
+        [ "${VORTEX_DEBUG:-}" = "1" ] && awk -F= '!/^($|#)/{print "Loaded: "$1}' .env || true
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
- &load_variables_from_dotenv
run:
name: Load environment variables from .env file
# Load variables from .env file, respecting existing values, and make them available for the next steps.
command: t=$(mktemp) && export -p >"${t}" && set -a && . ./.env && set +a && . "${t}" && export -p >> "$BASH_ENV"
- &load_variables_from_dotenv
run:
name: Load environment variables from .env file (safe)
command: |
if [ -f .env ]; then
# Export only keys from .env; ignore comments/blank lines.
while IFS='=' read -r key rest; do
[ -z "${key}" ] && continue
case "${key}" in \#* ) continue;; esac
key="${key%%[[:space:]]*}"
# Safely quote for BASH_ENV
printf 'export %s=%q\n' "$key" "$rest" >> "$BASH_ENV"
done < .env
fi
# Optional: names-only debug
[ "${VORTEX_DEBUG:-}" = "1" ] && awk -F= '!/^($|#)/{print "Loaded: "$1}' .env || true
🤖 Prompt for AI Agents 
In .circleci/config.yml around lines 100 to 105, replace the unsafe anchor that
snapshots the environment, sources .env, then appends the entire exported
environment to $BASH_ENV; instead remove the tmp snapshot and the export -p
append and implement a safe .env writer: parse .env without sourcing (ignore
comments/blank lines), validate KEY=VALUE lines, shell-quote values correctly,
and append only those KEY="quoted value" export statements to $BASH_ENV so
existing environment is respected, no pre-resourcing is undone, and SC1090 is
avoided.

################################################################################
# PARAMETERS
Expand Down Expand Up @@ -133,6 +136,7 @@ jobs:

- checkout
- *step_process_codebase_for_ci
- *load_variables_from_dotenv
- *step_setup_remote_docker

- run:
Expand Down Expand Up @@ -214,6 +218,7 @@ jobs:

- checkout
- *step_process_codebase_for_ci
- *load_variables_from_dotenv

- run:
name: Validate Composer configuration
Expand Down Expand Up @@ -262,6 +267,7 @@ jobs:
- run:
name: Export built codebase
command: |
echo "${VORTEX_DEPLOY_TYPES:-}" | grep -vq "artifact" && exit 0 || true
mkdir -p "/tmp/workspace/code"
docker compose cp -L cli:"/app/." "/tmp/workspace/code"
du -sh "/tmp/workspace/code"
Expand Down Expand Up @@ -398,6 +404,7 @@ jobs:

- checkout
- *step_process_codebase_for_ci
- *load_variables_from_dotenv

- run:
name: Deploy
Expand All @@ -424,6 +431,7 @@ jobs:

- checkout
- *step_process_codebase_for_ci
- *load_variables_from_dotenv

- run:
name: Deploy
Expand Down Expand Up @@ -495,6 +503,7 @@ jobs:
steps:
- checkout
- *step_process_codebase_for_ci
- *load_variables_from_dotenv
- *step_setup_remote_docker

- run:
Expand Down
11 changes: 9 additions & 2 deletions .github/workflows/build-test-deploy.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -199,6 +199,9 @@ jobs:
- name: Process the codebase to run in CI
run: find . -name "docker-compose.yml" -print0 | xargs -0 -I {} sh -c "sed -i -e '/###/d' {} && sed -i -e 's/##//' {}"

- name: Load environment variables from .env
run: t=$(mktemp) && export -p >"${t}" && set -a && . ./.env && set +a && . "${t}" && env >> "$GITHUB_ENV"

- name: Validate Composer configuration
run: composer validate --strict
continue-on-error: ${{ vars.VORTEX_CI_COMPOSER_VALIDATE_IGNORE_FAILURE == '1' }}
Expand Down Expand Up @@ -248,7 +251,7 @@ jobs:
run: docker compose up -d

- name: Export built codebase
if: matrix.instance == 0
if: matrix.instance == 0 && contains(env.VORTEX_DEPLOY_TYPES, 'artifact')
run: |
mkdir -p "/tmp/workspace/code"
docker compose cp -L cli:"/app/." "/tmp/workspace/code"
Expand Down Expand Up @@ -362,7 +365,7 @@ jobs:

- name: Upload exported codebase as artifact
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
if: ${{ matrix.instance == 0 && !startsWith(github.head_ref || github.ref_name, 'deps/') }}
if: ${{ matrix.instance == 0 && !startsWith(github.head_ref || github.ref_name, 'deps/') && contains(env.VORTEX_DEPLOY_TYPES, 'artifact') }}
with:
name: code-artifact
path: "/tmp/workspace/code"
Expand Down Expand Up @@ -407,8 +410,12 @@ jobs:
persist-credentials: false
ref: ${{ github.head_ref || github.ref_name }}

- name: Load environment variables from .env
run: t=$(mktemp) && export -p >"${t}" && set -a && . ./.env && set +a && . "${t}" && env >> "$GITHUB_ENV"

Comment on lines +413 to +415
Copy link

@coderabbitai coderabbitai bot Aug 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

⚠️ Potential issue

Repeat the safe .env loader here as well

Same concerns as in the build job: avoid restoring the snapshot after sourcing .env and avoid dumping the whole environment to GITHUB_ENV.

-      - name: Load environment variables from .env
-        run: t=$(mktemp) && export -p >"${t}" && set -a && . ./.env && set +a && . "${t}" && env >> "$GITHUB_ENV"
+      - name: Load environment variables from .env (safe)
+        run: |
+          while IFS='=' read -r key rest; do
+            [ -z "${key}" ] && continue
+            case "${key}" in \#* ) continue;; esac
+            key="${key%%[[:space:]]*}"
+            {
+              echo "${key}<<__ENV__"
+              printf '%s\n' "${rest}"
+              echo "__ENV__"
+            } >> "$GITHUB_ENV"
+          done < .env
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
- name: Load environment variables from .env
run: t=$(mktemp) && export -p >"${t}" && set -a && . ./.env && set +a && . "${t}" && env >> "$GITHUB_ENV"
- name: Load environment variables from .env (safe)
run: |
while IFS='=' read -r key rest; do
[ -z "${key}" ] && continue
case "${key}" in \#* ) continue;; esac
key="${key%%[[:space:]]*}"
{
echo "${key}<<__ENV__"
printf '%s\n' "${rest}"
echo "__ENV__"
} >> "$GITHUB_ENV"
done < .env

- name: Download exported codebase as an artifact
uses: actions/download-artifact@634f93cb2916e3fdff6788551b99b062d0335ce0 # v5
if: ${{ contains(env.VORTEX_DEPLOY_TYPES, 'artifact') }}
with:
name: code-artifact
path: "/tmp/workspace/code"
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/vortex-test-common.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -273,5 +273,5 @@ jobs:
continue-on-error: ${{ vars.VORTEX_CI_YAMLLINT_IGNORE_FAILURE == '1' }}

- name: Check coding standards with actionlint
run: docker run --rm -v "${GITHUB_WORKSPACE:-.}":/app --workdir /app rhysd/actionlint:1.7.2 -ignore 'SC2002:' -ignore 'SC2155:' -ignore 'SC2015:' -ignore 'SC2046:'
run: docker run --rm -v "${GITHUB_WORKSPACE:-.}":/app --workdir /app rhysd/actionlint:1.7.2 -ignore 'SC2002:' -ignore 'SC2155:' -ignore 'SC2015:' -ignore 'SC2046:' -ignore 'SC1090:'
Copy link

@coderabbitai coderabbitai bot Aug 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick (assertive)

Avoid blanket-ignoring SC1090; eliminate the root cause instead

SC1090 is triggered by sourcing .env in other workflows. Prefer switching those steps to a non-sourcing .env loader (parsing key=value and writing to GITHUB_ENV/BASH_ENV) which removes SC1090 entirely. Then drop this global ignore to keep signal from shellcheck high.

If you keep SC1090 ignored here, scope it with a regex to only the specific jobs/files that need it, rather than every workflow.

Once you adopt the safe loader in build-test-deploy.yml and CircleCI, you can safely apply:

-        run: docker run --rm -v "${GITHUB_WORKSPACE:-.}":/app --workdir /app rhysd/actionlint:1.7.2 -ignore 'SC2002:' -ignore 'SC2155:' -ignore 'SC2015:' -ignore 'SC2046:' -ignore 'SC1090:'
+        run: docker run --rm -v "${GITHUB_WORKSPACE:-.}":/app --workdir /app rhysd/actionlint:1.7.2 -ignore 'SC2002:' -ignore 'SC2155:' -ignore 'SC2015:' -ignore 'SC2046:'
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
run: docker run --rm -v "${GITHUB_WORKSPACE:-.}":/app --workdir /app rhysd/actionlint:1.7.2 -ignore 'SC2002:' -ignore 'SC2155:' -ignore 'SC2015:' -ignore 'SC2046:' -ignore 'SC1090:'
run: docker run --rm -v "${GITHUB_WORKSPACE:-.}":/app --workdir /app rhysd/actionlint:1.7.2 -ignore 'SC2002:' -ignore 'SC2155:' -ignore 'SC2015:' -ignore 'SC2046:'
🤖 Prompt for AI Agents 
In .github/workflows/vortex-test-common.yml around line 276 the actionlint
invocation is blanket-ignoring SC1090; remove SC1090 from the global -ignore
list and instead fix the root cause by changing any workflow steps that "source"
.env to a safe loader that parses key=value and writes to GITHUB_ENV or BASH_ENV
(update build-test-deploy.yml and CircleCI accordingly); if you cannot update
all callers immediately, narrow the ignore to a regex that targets only the
specific workflow files/jobs that legitimately need SC1090 until the safe loader
is adopted, then drop the ignore entirely.

continue-on-error: ${{ vars.VORTEX_CI_ACTIONLINT_IGNORE_FAILURE == '1' }}
11 changes: 9 additions & 2 deletions .vortex/installer/tests/Fixtures/install/_baseline/.github/workflows/build-test-deploy.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -188,6 +188,9 @@ jobs:
- name: Process the codebase to run in CI
run: find . -name "docker-compose.yml" -print0 | xargs -0 -I {} sh -c "sed -i -e '/###/d' {} && sed -i -e 's/##//' {}"

- name: Load environment variables from .env
run: t=$(mktemp) && export -p >"${t}" && set -a && . ./.env && set +a && . "${t}" && env >> "$GITHUB_ENV"

- name: Validate Composer configuration
run: composer validate --strict
continue-on-error: ${{ vars.VORTEX_CI_COMPOSER_VALIDATE_IGNORE_FAILURE == '1' }}
Expand Down Expand Up @@ -235,7 +238,7 @@ jobs:
run: docker compose up -d

- name: Export built codebase
if: matrix.instance == 0
if: matrix.instance == 0 && contains(env.VORTEX_DEPLOY_TYPES, 'artifact')
run: |
mkdir -p "/tmp/workspace/code"
docker compose cp -L cli:"/app/." "/tmp/workspace/code"
Expand Down Expand Up @@ -347,7 +350,7 @@ jobs:

- name: Upload exported codebase as artifact
uses: actions/upload-artifact@__HASH__ # __VERSION__
if: ${{ matrix.instance == 0 && !startsWith(github.head_ref || github.ref_name, 'deps/') }}
if: ${{ matrix.instance == 0 && !startsWith(github.head_ref || github.ref_name, 'deps/') && contains(env.VORTEX_DEPLOY_TYPES, 'artifact') }}
with:
name: code-artifact
path: "/tmp/workspace/code"
Expand Down Expand Up @@ -389,8 +392,12 @@ jobs:
persist-credentials: false
ref: ${{ github.head_ref || github.ref_name }}

- name: Load environment variables from .env
run: t=$(mktemp) && export -p >"${t}" && set -a && . ./.env && set +a && . "${t}" && env >> "$GITHUB_ENV"

- name: Download exported codebase as an artifact
uses: actions/download-artifact@__HASH__ # __VERSION__
if: ${{ contains(env.VORTEX_DEPLOY_TYPES, 'artifact') }}
with:
name: code-artifact
path: "/tmp/workspace/code"
Expand Down
13 changes: 11 additions & 2 deletions .vortex/installer/tests/Fixtures/install/ciprovider_circleci/.circleci/config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,6 @@ aliases:
# Change to 'large' for faster builds.
resource_class: medium

# Set up remote Docker.
- &step_setup_remote_docker
setup_remote_docker:
# Docker Layer Caching allows to significantly speed up builds by caching
Expand All @@ -80,14 +79,19 @@ aliases:
docker_layer_caching: false
version: default

# Process the codebase to be run in CI environment.
- &step_process_codebase_for_ci
run:
name: Process codebase to run in CI
command: |
find . -name "docker-compose.yml" -print0 | xargs -0 -I {} sh -c "sed -i -e ''/###/d'' {} && sed -i -e ''s/##//'' {}"
mkdir -p /tmp/workspace/code

- &load_variables_from_dotenv
run:
name: Load environment variables from .env file
# Load variables from .env file, respecting existing values, and make them available for the next steps.
command: t=$(mktemp) && export -p >"${t}" && set -a && . ./.env && set +a && . "${t}" && export -p >> "$BASH_ENV"

################################################################################
# PARAMETERS
################################################################################
Expand Down Expand Up @@ -120,6 +124,7 @@ jobs:

- checkout
- *step_process_codebase_for_ci
- *load_variables_from_dotenv
- *step_setup_remote_docker

- run:
Expand Down Expand Up @@ -200,6 +205,7 @@ jobs:

- checkout
- *step_process_codebase_for_ci
- *load_variables_from_dotenv

- run:
name: Validate Composer configuration
Expand Down Expand Up @@ -246,6 +252,7 @@ jobs:
- run:
name: Export built codebase
command: |
echo "${VORTEX_DEPLOY_TYPES:-}" | grep -vq "artifact" && exit 0 || true
mkdir -p "/tmp/workspace/code"
docker compose cp -L cli:"/app/." "/tmp/workspace/code"
du -sh "/tmp/workspace/code"
Expand Down Expand Up @@ -379,6 +386,7 @@ jobs:

- checkout
- *step_process_codebase_for_ci
- *load_variables_from_dotenv

- run:
name: Deploy
Expand All @@ -405,6 +413,7 @@ jobs:

- checkout
- *step_process_codebase_for_ci
- *load_variables_from_dotenv

- run:
name: Deploy
Expand Down
13 changes: 11 additions & 2 deletions .vortex/installer/tests/Fixtures/install/deploy_type_all_circleci/.circleci/config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,6 @@ aliases:
# Change to 'large' for faster builds.
resource_class: medium

# Set up remote Docker.
- &step_setup_remote_docker
setup_remote_docker:
# Docker Layer Caching allows to significantly speed up builds by caching
Expand All @@ -80,14 +79,19 @@ aliases:
docker_layer_caching: false
version: default

# Process the codebase to be run in CI environment.
- &step_process_codebase_for_ci
run:
name: Process codebase to run in CI
command: |
find . -name "docker-compose.yml" -print0 | xargs -0 -I {} sh -c "sed -i -e ''/###/d'' {} && sed -i -e ''s/##//'' {}"
mkdir -p /tmp/workspace/code

- &load_variables_from_dotenv
run:
name: Load environment variables from .env file
# Load variables from .env file, respecting existing values, and make them available for the next steps.
command: t=$(mktemp) && export -p >"${t}" && set -a && . ./.env && set +a && . "${t}" && export -p >> "$BASH_ENV"

################################################################################
# PARAMETERS
################################################################################
Expand Down Expand Up @@ -120,6 +124,7 @@ jobs:

- checkout
- *step_process_codebase_for_ci
- *load_variables_from_dotenv
- *step_setup_remote_docker

- run:
Expand Down Expand Up @@ -200,6 +205,7 @@ jobs:

- checkout
- *step_process_codebase_for_ci
- *load_variables_from_dotenv

- run:
name: Validate Composer configuration
Expand Down Expand Up @@ -246,6 +252,7 @@ jobs:
- run:
name: Export built codebase
command: |
echo "${VORTEX_DEPLOY_TYPES:-}" | grep -vq "artifact" && exit 0 || true
mkdir -p "/tmp/workspace/code"
docker compose cp -L cli:"/app/." "/tmp/workspace/code"
du -sh "/tmp/workspace/code"
Expand Down Expand Up @@ -379,6 +386,7 @@ jobs:

- checkout
- *step_process_codebase_for_ci
- *load_variables_from_dotenv

- run:
name: Deploy
Expand All @@ -405,6 +413,7 @@ jobs:

- checkout
- *step_process_codebase_for_ci
- *load_variables_from_dotenv

- run:
name: Deploy
Expand Down
11 changes: 9 additions & 2 deletions .vortex/installer/tests/Fixtures/install/deploy_type_none_circleci/.circleci/config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,6 @@ aliases:
# Change to 'large' for faster builds.
resource_class: medium

# Set up remote Docker.
- &step_setup_remote_docker
setup_remote_docker:
# Docker Layer Caching allows to significantly speed up builds by caching
Expand All @@ -80,14 +79,19 @@ aliases:
docker_layer_caching: false
version: default

# Process the codebase to be run in CI environment.
- &step_process_codebase_for_ci
run:
name: Process codebase to run in CI
command: |
find . -name "docker-compose.yml" -print0 | xargs -0 -I {} sh -c "sed -i -e ''/###/d'' {} && sed -i -e ''s/##//'' {}"
mkdir -p /tmp/workspace/code

- &load_variables_from_dotenv
run:
name: Load environment variables from .env file
# Load variables from .env file, respecting existing values, and make them available for the next steps.
command: t=$(mktemp) && export -p >"${t}" && set -a && . ./.env && set +a && . "${t}" && export -p >> "$BASH_ENV"

################################################################################
# PARAMETERS
################################################################################
Expand Down Expand Up @@ -120,6 +124,7 @@ jobs:

- checkout
- *step_process_codebase_for_ci
- *load_variables_from_dotenv
- *step_setup_remote_docker

- run:
Expand Down Expand Up @@ -200,6 +205,7 @@ jobs:

- checkout
- *step_process_codebase_for_ci
- *load_variables_from_dotenv

- run:
name: Validate Composer configuration
Expand Down Expand Up @@ -246,6 +252,7 @@ jobs:
- run:
name: Export built codebase
command: |
echo "${VORTEX_DEPLOY_TYPES:-}" | grep -vq "artifact" && exit 0 || true
mkdir -p "/tmp/workspace/code"
docker compose cp -L cli:"/app/." "/tmp/workspace/code"
du -sh "/tmp/workspace/code"
Expand Down
6 changes: 5 additions & 1 deletion ...aller/tests/Fixtures/install/deploy_type_none_gha/.github/workflows/build-test-deploy.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
@@ -361,61 +361,3 @@
@@ -364,65 +364,3 @@
timeout-minutes: 120 # Cancel the action after 15 minutes, regardless of whether a connection has been established.
with:
detached: true
Expand Down Expand Up @@ -30,8 +30,12 @@
- persist-credentials: false
- ref: ${{ github.head_ref || github.ref_name }}
-
- - name: Load environment variables from .env
- run: t=$(mktemp) && export -p >"${t}" && set -a && . ./.env && set +a && . "${t}" && env >> "$GITHUB_ENV"
-
- - name: Download exported codebase as an artifact
- uses: actions/download-artifact@__HASH__ # __VERSION__
- if: ${{ contains(env.VORTEX_DEPLOY_TYPES, 'artifact') }}
- with:
- name: code-artifact
- path: "/tmp/workspace/code"
Expand Down
Loading