-
Notifications
You must be signed in to change notification settings - Fork 77
[OpenMP] Print loaded envar when debug flag is enabled #520
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
| if: github.event.pull_request.draft == false | ||
| runs-on: | ||
| group: compiler-generic-runners | ||
| env: | ||
| svc_acc_org_secret: ${{secrets.CI_GITHUB_TOKEN}} | ||
| input_sha: ${{ github.event.pull_request.head.sha != '' && github.event.pull_request.head.sha || github.sha }} | ||
| input_pr_num: ${{ github.event.pull_request.number != '' && github.event.pull_request.number || 0 }} | ||
| input_pr_url: ${{ github.event.pull_request.html_url != '' && github.event.pull_request.html_url || '' }} | ||
| input_pr_title: ${{ github.event.pull_request.title != '' && github.event.pull_request.title || '' }} | ||
| # set the pipeline name here based on branch name | ||
| pipeline_name: ${{secrets.CI_JENKINS_JOB_NAME}} | ||
| JENKINS_URL: ${{secrets.CI_JENKINS_URL}} | ||
| CONTAINER_IMAGE: ${{ secrets.JENKINS_TRIGGER_DOCKER_IMAGE }} | ||
|
|
||
| # Steps represent a sequence of tasks that will be executed as part of the job | ||
| steps: | ||
|
|
||
| # Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it | ||
| - name: Set environment variable for container image | ||
| run: | | ||
| echo "CONTAINER_IMAGE=${{ secrets.JENKINS_TRIGGER_DOCKER_IMAGE }}" >> $GITHUB_ENV | ||
| echo "CONTAINER_NAME=my_container_${{ github.run_id }}" >> $GITHUB_ENV | ||
|
|
||
|
|
||
| - name: Pull container image | ||
| run: docker pull "${{env.CONTAINER_IMAGE}}" | ||
|
|
||
|
|
||
| - name: Run container | ||
| run: | | ||
| docker run -d --name "${{env.CONTAINER_NAME}}" $CONTAINER_IMAGE sleep infinity | ||
| #docker exec "${{env.CONTAINER_NAME}}" /bin/bash -c "git clone ${{secrets.CI_UTILS_REPO}} ." | ||
| docker exec "${{env.CONTAINER_NAME}}" /bin/bash -c "echo 'Running commands inside the container'" | ||
|
|
||
| - name: Escape pull request title | ||
| run: | | ||
| import json | ||
| import os | ||
| import shlex | ||
| with open('${{ github.event_path }}') as fh: | ||
| event = json.load(fh) | ||
| escaped = event['pull_request']['title'] | ||
| with open(os.environ['GITHUB_ENV'], 'a') as fh: | ||
| print(f'PR_TITLE={escaped}', file=fh) | ||
| shell: python3 {0} | ||
|
|
||
| - name: Run Jenkins Cancel Script | ||
| env: | ||
| JENKINS_URL: ${{secrets.CI_JENKINS_URL}} | ||
| JENKINS_USER: ${{secrets.CI_JENKINS_USER}} | ||
| JENKINS_API_TOKEN: ${{secrets.CI_JENKINS_TOKEN}} | ||
| JENKINS_JOB_NAME: ${{secrets.CI_JENKINS_JOB_NAME}} | ||
| PR_NUMBER: ${{ github.event.pull_request.number }} | ||
| COMMIT_HASH: ${{ github.event.after }} | ||
| run: | | ||
| docker exec -e JENKINS_JOB_NAME=${{secrets.CI_JENKINS_JOB_NAME}} -e PR_NUMBER=${{ github.event.pull_request.number }} -e COMMIT_HASH=${{ github.event.after }} -e JENKINS_URL=${{secrets.CI_JENKINS_URL}} -e JENKINS_USER=${{secrets.CI_JENKINS_USER}} -e JENKINS_API_TOKEN=${{secrets.CI_JENKINS_TOKEN}} "${{env.CONTAINER_NAME}}" /bin/bash -c "PYTHONHTTPSVERIFY=0 python3 cancel_previous_build.py" | ||
|
|
||
|
|
||
| # Runs a set of commands using the runners shell | ||
| - name: Getting Event Details | ||
| run: | | ||
| echo $(pwd) | ||
| echo $GITHUB_ENV | ||
| echo $GITHUB_REPOSITORY | ||
| echo $GITHUB_SERVER_URL | ||
| echo "GITHUB_SHA is: $GITHUB_SHA" | ||
| echo "GITHUB_WORKFLOW_SHA is: $GITHUB_WORKFLOW_SHA" | ||
| echo "GITHUB_BASE_REF is: $GITHUB_BASE_REF" | ||
| echo "GITHUB_REF_NAME is: $GITHUB_REF_NAME" | ||
| echo "github.event.pull_request.id is: ${{github.event.pull_request.id}}" | ||
| echo "github.event.pull_request.html_url is: ${{github.event.pull_request.html_url}}" | ||
| echo "github.event.pull_request.number is: ${{github.event.pull_request.number}}" | ||
| echo "github.event.pull_request.url is: ${{github.event.pull_request.url}}" | ||
| echo "github.event.pull_request.issue_url is: ${{github.event.pull_request.issue_url}}" | ||
| echo "github.event.pull_request.head.sha is: ${{github.event.pull_request.head.sha}}" | ||
| echo "github.event.pull_request.base.ref is: ${{github.event.pull_request.base.ref}}" | ||
| echo "github.event.pull_request.merge_commit_sha is: ${{github.event.pull_request.merge_commit_sha}}" | ||
| echo "github.event.pull_request is: ${{github.event.pull_request}}" | ||
|
|
||
|
|
||
| - name: Trigger Jenkins Pipeline | ||
| if: steps.check_changes.outcome != 'failure' | ||
| run: | | ||
| echo "--Running jenkins_api.py with input sha - $input_sha for pull request - $input_pr_url" | ||
| docker exec -e GITHUB_REPOSITORY="$GITHUB_REPOSITORY" -e svc_acc_org_secret="$svc_acc_org_secret" -e input_sha="$input_sha" -e input_pr_url="$input_pr_url" -e pipeline_name="$pipeline_name" \ | ||
| -e input_pr_num="$input_pr_num" -e PR_TITLE="$PR_TITLE" -e JENKINS_URL="$JENKINS_URL" -e GITHUB_PAT="$svc_acc_org_secret" "${{env.CONTAINER_NAME}}" \ | ||
| /bin/bash -c 'echo \"PR NUM: "$input_pr_num"\" && PYTHONHTTPSVERIFY=0 python3 jenkins_api.py -s \"${JENKINS_URL}\" -jn "$pipeline_name" -ghr "$GITHUB_REPOSITORY" -ghsha "$input_sha" -ghprn "$input_pr_num" -ghpru "$input_pr_url" -ghprt "$PR_TITLE" -ghpat="$svc_acc_org_secret"' | ||
|
|
||
| - name: Stop and remove container | ||
| if: always() | ||
| run: | | ||
| docker stop "${{env.CONTAINER_NAME}}" | ||
| docker rm "${{env.CONTAINER_NAME}}" |
Check warning
Code scanning / CodeQL
Workflow does not contain permissions
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 21 days ago
To resolve the issue, you should add an explicit permissions: block to the workflow to scope the permissions of the GITHUB_TOKEN to the minimum required set for the jobs being run. This can be done at the root level (so that it applies to all jobs) or at the job level if finer granularity is needed. For this workflow, where there is only one job, the simplest way is to add the block at the root level, immediately after the workflow name and before the on: block. As a starting point, permissions: contents: read is recommended unless more access is needed (for example, to write to pull requests, in which case pull-requests: write can be added). Add the following block:
permissions:
contents: readMake this change at the top of the file .github/workflows/PSDB-amd-staging.yml, right after the workflow name.
-
Copy modified lines R2-R3
| @@ -1,4 +1,6 @@ | ||
| name: Compiler CI PSDB trigger on amd-staging branch | ||
| permissions: | ||
| contents: read | ||
|
|
||
| # Controls when the workflow will run | ||
| on: |
| if: github.event.pull_request.draft == false | ||
| runs-on: | ||
| group: compiler-generic-runners | ||
| env: | ||
| PR_SHA: ${{ github.event.pull_request.head.sha != '' && github.event.pull_request.head.sha || github.sha }} | ||
| PR_NUMBER: ${{ github.event.pull_request.number != '' && github.event.pull_request.number || 0 }} | ||
| PR_URL: ${{ github.event.pull_request.html_url != '' && github.event.pull_request.html_url || '' }} | ||
| PR_TITLE: ${{ github.event.pull_request.title != '' && github.event.pull_request.title || '' }} | ||
| BASE_BRANCH: ${{ github.event.pull_request.base.ref != '' && github.event.pull_request.base.ref || '' }} | ||
| GITHUB_TOKEN: ${{secrets.CI_GITHUB_TOKEN}} | ||
|
|
||
| steps: | ||
| # Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it | ||
| - name: Set environment variable for container image | ||
| run: | | ||
| echo "CONTAINER_IMAGE=${{ secrets.BUILDBOT_DOCKER_IMAGE }}" >> $GITHUB_ENV | ||
| echo "CONTAINER_NAME=my_container_${{ github.run_id }}" >> $GITHUB_ENV | ||
|
|
||
| - name: Pull container image | ||
| run: docker pull "${{env.CONTAINER_IMAGE}}" | ||
|
|
||
| - name: Run container | ||
| run: | | ||
| docker run -d --name "${{env.CONTAINER_NAME}}" $CONTAINER_IMAGE sleep infinity | ||
| docker exec "${{env.CONTAINER_NAME}}" /bin/bash -c "echo 'Running commands inside the container'" | ||
|
|
||
| - name: Escape pull request title | ||
| run: | | ||
| import json | ||
| import os | ||
| import shlex | ||
| with open('${{ github.event_path }}') as fh: | ||
| event = json.load(fh) | ||
| escaped = event['pull_request']['title'] | ||
| with open(os.environ['GITHUB_ENV'], 'a') as fh: | ||
| print(f'PR_TITLE={escaped}', file=fh) | ||
| shell: python3 {0} | ||
|
|
||
| - name: Trigger Buildbot Build | ||
| run: | | ||
| echo "${{ secrets.BUILDBOT_HOST }}:${{ secrets.BUILDBOT_WORKER_PORT }}" | ||
| docker exec -e PR_TITLE="$PR_TITLE" "${{env.CONTAINER_NAME}}" /bin/bash -c 'buildbot sendchange -W ${{ secrets.BUILDBOT_USER }} -a ${{secrets.BUILDBOT_USER}}:${{secrets.BUILDBOT_PWD}} --master="${{ secrets.BUILDBOT_HOST }}:${{ secrets.BUILDBOT_WORKER_PORT }}" --branch=${{ env.BASE_BRANCH }} --revision=${{ env.PR_SHA }} -p PR_NUMBER:${{ env.PR_NUMBER }} -p PR_TITLE:"$PR_TITLE" -p PR_URL:${{ env.PR_URL }} -p SHA:${{ env.PR_SHA }}' | ||
|
|
||
| - name: Set Initial Status to Pending | ||
| run: | | ||
| docker exec -e PR_SHA=$PR_SHA -e GITHUB_TOKEN=$GITHUB_TOKEN "${{env.CONTAINER_NAME}}" /bin/bash -c "python3 -c \" | ||
| import os | ||
| import requests | ||
| GITHUB_TOKEN = os.getenv('GITHUB_TOKEN') | ||
| TARGET_SHA = os.getenv('PR_SHA') | ||
| print('debug', TARGET_SHA) | ||
| api_url = f'https://api.github.com/repos/AMD-Lightning-Internal/llvm-project/statuses/{TARGET_SHA}' | ||
| headers = { | ||
| 'Authorization': f'token {GITHUB_TOKEN}', | ||
| 'Content-Type': 'application/json' | ||
| } | ||
| payload = { | ||
| 'state': 'pending', | ||
| 'context': 'buildbot', | ||
| 'description': 'Build is in queue' | ||
| } | ||
| response = requests.post(api_url, json=payload, headers=headers) | ||
| if response.status_code == 201: | ||
| print('Status set to pending successfully.') | ||
| else: | ||
| print(f'Failed to set status: {response.status_code} {response.text}') | ||
| \"" | ||
|
|
||
| - name: Poll Buildbot build status | ||
| run: | | ||
| python3 -c " | ||
| import os | ||
| import time | ||
| import requests | ||
| GITHUB_TOKEN = os.getenv('GITHUB_TOKEN') | ||
| BUILD_URL = 'http://${{ secrets.BUILDBOT_HOST }}:${{ secrets.BUILDBOT_MASTER_PORT }}/api/v2/builds' | ||
| TARGET_SHA = os.getenv('PR_SHA') | ||
| print('debug', TARGET_SHA) | ||
| MAX_RETRIES = 10 | ||
| RETRY_INTERVAL = 30 # seconds | ||
|
|
||
| def get_build_properties(build_id): | ||
| build_properties_url = f'http://${{ secrets.BUILDBOT_HOST }}:${{ secrets.BUILDBOT_MASTER_PORT }}/api/v2/builds/{build_id}/properties' | ||
| response = requests.get(build_properties_url, headers={'Accept': 'application/json', 'Authorization': f'token {GITHUB_TOKEN}'}) | ||
| return response.json() | ||
|
|
||
| for i in range(MAX_RETRIES): | ||
| response = requests.get(BUILD_URL, headers={'Accept': 'application/json'}) | ||
| response_json = response.json() | ||
| print(f'Attempt {i + 1}: Buildbot response:', response_json) | ||
|
|
||
| # Check if any build has the target SHA | ||
| builds = response_json.get('builds', []) | ||
| print (builds) | ||
| build_with_sha = None | ||
| for build in builds: | ||
| build_id = build['buildid'] | ||
| properties = get_build_properties(build_id) | ||
| #print(properties) | ||
| #prop = properties.get('revision', []) | ||
|
|
||
| if 'properties' in properties: | ||
| print (properties['properties']) | ||
| if 'revision' in properties['properties'][0]: | ||
| print(properties['properties'][0]) | ||
| if 'revision' in properties['properties'][0] and properties['properties'][0]['revision'] [0] == TARGET_SHA: | ||
| build_with_sha = build | ||
| break | ||
|
|
||
| if build_with_sha: | ||
| print('Build started successfully for SHA:', TARGET_SHA) | ||
| break | ||
| else: | ||
| print('Build for SHA not started yet, retrying in', RETRY_INTERVAL, 'seconds') | ||
| time.sleep(RETRY_INTERVAL) | ||
| else: | ||
| print('Build did not start for SHA:', TARGET_SHA, 'after maximum retries') | ||
| exit(1) | ||
| " | ||
|
|
||
| - name: Stop and remove container | ||
| if: always() | ||
| run: | | ||
| docker stop "${{env.CONTAINER_NAME}}" | ||
| docker rm "${{env.CONTAINER_NAME}}" |
Check warning
Code scanning / CodeQL
Workflow does not contain permissions
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 21 days ago
The correct fix is to add a permissions: block, defining least-privilege permissions needed to run the workflow. Since the job updates commit statuses via the GitHub API, it requires statuses: write. The default workflow steps (like checkout) may need contents: read, but since the workflow does not modify repository code or artifacts, no other write permissions are necessary.
Add the permissions: block at the workflow root (above jobs:) to cover all jobs in this workflow, or (alternatively but equivalently) at the individual job level. The recommended patch is to add at the root:
- Insert
right after the
permissions: contents: read statuses: write
name:andon:blocks, beforejobs:.
No additional imports or definitions are needed since only the workflow YAML is being updated.
-
Copy modified lines R9-R12
| @@ -6,6 +6,10 @@ | ||
| types: [opened, reopened, synchronize, ready_for_review] | ||
|
|
||
|
|
||
| permissions: | ||
| contents: read | ||
| statuses: write | ||
|
|
||
| jobs: | ||
| trigger-build: | ||
| if: github.event.pull_request.draft == false |
| runs-on: | ||
| group: compiler-generic-runners | ||
|
|
||
| steps: | ||
| - name: Set environment variable for container image | ||
| run: | | ||
| echo "CONTAINER_IMAGE=${{ secrets.JENKINS_TRIGGER_DOCKER_IMAGE }}" >> $GITHUB_ENV | ||
| echo "CONTAINER_NAME=my_container_${{ github.run_id }}" >> $GITHUB_ENV | ||
|
|
||
| - name: Pull container image | ||
| run: docker pull "${{env.CONTAINER_IMAGE}}" | ||
|
|
||
| - name: Run container | ||
| run: | | ||
| docker run -d --name "${{env.CONTAINER_NAME}}" $CONTAINER_IMAGE sleep infinity | ||
| docker exec "${{env.CONTAINER_NAME}}" /bin/bash -c "echo 'Running commands inside the container'" | ||
|
|
||
| - name: Trigger compute-rocm-dkms-afar job | ||
| run: | | ||
| docker exec "${{env.CONTAINER_NAME}}" /bin/bash -c "python -c \" | ||
| import requests | ||
| import time | ||
| from requests.auth import HTTPBasicAuth | ||
|
|
||
| jenkins_user = '${{ secrets.CI_JENKINS_USER }}' | ||
| jenkins_token = '${{ secrets.ROCM_JENKINS_CI_TOKEN }}' | ||
| jenkins_host = '${{ secrets.ROCM_JENKINS_HOST }}' | ||
| jenkins_job = '${{ secrets.ROCM_JENKINS_OSDB_JOB }}' | ||
|
|
||
| jenkins_url = f'{jenkins_host}/job/{jenkins_job}/buildWithParameters' | ||
|
|
||
| response = requests.post(jenkins_url, auth=HTTPBasicAuth(jenkins_user, jenkins_token)) | ||
|
|
||
| if response.status_code == 201: | ||
| print('Jenkins job triggered successfully!') | ||
| queue_url = response.headers.get('Location') | ||
| if queue_url: | ||
| print(f'Queue URL: {queue_url}') | ||
| print(f'Getting build URL(max 5 attempts with 10seconds interval)...') | ||
| # Poll the queue item to get the build number, limited to 5 attempts | ||
| max_attempts = 5 | ||
| attempts = 0 | ||
| while attempts < max_attempts: | ||
| queue_response = requests.get(queue_url + 'api/json', auth=HTTPBasicAuth(jenkins_user, jenkins_token)) | ||
| queue_data = queue_response.json() | ||
| if 'executable' in queue_data: | ||
| build_number = queue_data['executable']['number'] | ||
| build_url = f'{jenkins_host}/job/{jenkins_job}/{build_number}/' | ||
| print(f'Build URL: {build_url}') | ||
| break | ||
| attempts += 1 | ||
| time.sleep(10) # Wait for 10 seconds before polling again | ||
| else: | ||
| print('Exceeded maximum attempts to get the build URL. The trigger happened, so not failing the workflow') | ||
| else: | ||
| print('Build URL not found in the response headers.') | ||
|
|
||
| elif response.status_code == 200: | ||
| print('Request was successful, but check the response content for details.') | ||
| print(response.text) | ||
| else: | ||
| print(f'Failed to trigger Jenkins job. Status code: {response.status_code}') | ||
| \"" | ||
|
|
||
| - name: Stop and remove container | ||
| if: always() | ||
| run: | | ||
| docker stop "${{env.CONTAINER_NAME}}" | ||
| docker rm "${{env.CONTAINER_NAME}}" |
Check warning
Code scanning / CodeQL
Workflow does not contain permissions
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 21 days ago
The best way to fix this problem is to add a permissions block at the top level of the workflow file (before or after on:), or at the job level, explicitly declaring only the needed permissions for GITHUB_TOKEN. In this workflow, no steps interact with repository contents, issues, or pull-requests, so the minimal block is most likely:
permissions:
contents: readThis restricts the GITHUB_TOKEN so the workflow cannot write to repository contents, or perform unnecessary actions. Make this change between the name: and on: block or immediately under on: in .github/workflows/compute-rocm-dkmd-afar-trigger.yml.
No additional imports, methods, or external dependencies are required.
-
Copy modified lines R3-R5
| @@ -1,5 +1,8 @@ | ||
| name: Trigger compute-rocm-dkms-afar job on push | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| on: | ||
| push: # This triggers the workflow on push events | ||
| branches: |
|
|
1dc7790 to
83fe820
Compare
Print out the loaded envars when
LIBOMPTARGET_DEBUG=1example output: