Skip to content

Fix Gitlab-CI #441

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Jun 24, 2024
Merged

Fix Gitlab-CI #441

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
df1736b
Added Job-Token as needed by Gitlab
jakob-fritz Jun 17, 2024
d7443c0
Moved schedule towards nighttime
jakob-fritz Jun 19, 2024
383c70d
Schedule gitlab-ci workflow at the same time as github-workflow
jakob-fritz Jun 24, 2024
d18df59
Moved installation of more packages into preparation to avoid paralle…
jakob-fritz Jun 24, 2024
94f03c6
First try to use virtual envs from python
jakob-fritz Jun 24, 2024
2a0f7fa
Store venv on scratch
jakob-fritz Jun 24, 2024
9143651
Activate project to use SCRATCH
jakob-fritz Jun 24, 2024
3b0f7e8
Wrong usage of variables
jakob-fritz Jun 24, 2024
30ecb30
Try installing mpi4py-fft for later usage
jakob-fritz Jun 24, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/gitlab_ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ on:
pull_request_target:
types: [opened, synchronize, reopened, labeled]
schedule:
- cron: '1 5 2 * *'
- cron: '1 5 * * 1'

jobs:
check_permission:
Expand Down
31 changes: 26 additions & 5 deletions .gitlab-ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,12 +8,18 @@ stages:

variables:
JUWELS_ACCOUNT: "cstma"
JUWELS_PROJECT: "ccstma"


prepare_JUWELS:
stage: benchmark
rules:
- if: $CI_COMMIT_MESSAGE !~ /.*\[CI-no-benchmarks\]/
id_tokens:
CI_JOB_JWT:
aud: https://gitlab.jsc.fz-juelich.de
SITE_ID_TOKEN:
aud: https://gitlab.jsc.fz-juelich.de
tags:
- jacamar
- juwels
Expand All @@ -29,8 +35,16 @@ prepare_JUWELS:
- module load mpi4py
- module load SciPy-Stack
- module load CuPy
- jutil env activate -p ${JUWELS_PROJECT}
- python -m venv --clear $SCRATCH/.venv/pySDC
- source $SCRATCH/.venv/pySDC/bin/activate
- pip install -e .
- pip install pytest-benchmark coverage
- git submodule add -f https://github.com/brownbaerchen/mpi4py-fft.git
- cd mpi4py-fft
- git checkout cupy_implementation
- FFTW_LIBRARY_DIR="/p/software/juwels/stages/2024/software/FFTW/3.3.10-GCC-12.3.0/lib64" pip install --force-reinstall .
- cd ../


test_JUWELS:
Expand All @@ -39,6 +53,11 @@ test_JUWELS:
- prepare_JUWELS
rules:
- if: $CI_COMMIT_MESSAGE !~ /.*\[CI-no-benchmarks\]/
id_tokens:
CI_JOB_JWT:
aud: https://gitlab.jsc.fz-juelich.de
SITE_ID_TOKEN:
aud: https://gitlab.jsc.fz-juelich.de
tags:
- jacamar
- juwels
Expand All @@ -63,11 +82,8 @@ test_JUWELS:
- module load mpi4py
- module load SciPy-Stack
- module load CuPy
- git submodule add -f https://github.com/brownbaerchen/mpi4py-fft.git
- cd mpi4py-fft
- git checkout cupy_implementation
- FFTW_LIBRARY_DIR="/p/software/juwels/stages/2024/software/FFTW/3.3.10-GCC-12.3.0/lib64" pip install --force-reinstall -e .
- cd ../
- jutil env activate -p ${JUWELS_PROJECT}
- source $SCRATCH/.venv/pySDC/bin/activate
script:
# - touch benchmarks/output.json
- echo $SYSTEMNAME
Expand All @@ -88,6 +104,11 @@ benchmark:
when: manual
tags:
- docker
id_tokens:
CI_JOB_JWT:
aud: https://gitlab.jsc.fz-juelich.de
SITE_ID_TOKEN:
aud: https://gitlab.jsc.fz-juelich.de
rules:
- if: $CI_COMMIT_MESSAGE !~ /.*\[CI-no-benchmarks\]/
artifacts:
Expand Down