Draft: Github CI, PR checks, weekly retagger #574

Ariouz · 2025-12-09T10:10:49Z

As it adds a few things, detailed descriptions are located in fork's PRs.

Python unittest retagger jobs + PR checks: Ariouz#1 (Ariouz@68cff18)
Weekly retagger workflow, github exclusion tag: Ariouz#3
Auto PR on weekly retagger: Ariouz#8 merged in 3
Windows build: Ariouz#5 merged in 3

Platforms: Linux and Windows, github hosted runner for macos has too low mem to build.
This also applies all retags for linux amd64, linux aarch64, windows retagger is still running

Tests ran on PR: https://github.com/Ariouz/graalpython/actions/runs/20066056924
Weekly retagger WF: https://github.com/Ariouz/graalpython/actions/runs/20058297116: Ariouz#13

.github/scripts/extract_matrix.py

.github/workflows/ci-matrix-gen.yml

timfel · 2025-12-10T12:43:46Z

ci.jsonnet

        "python-unittest-cpython": cpygate + platform_spec(no_jobs) + platform_spec({
            "linux:amd64:jdk-latest"     : tier1,
        }),
-        "python-unittest-retagger": ut_retagger + platform_spec(no_jobs) + platform_spec({


Those cannot change, they must stay weekly here. I suppose we need to allow weekly when generating the matrix and treat them as after the tiered jobs. I guess then the tier2 -> tier1 change for python-svm-build could also go away

Fixed, i set weekly jobs as tier1 in extract-matrix.py

timfel · 2025-12-10T12:44:22Z

ci/graal/ci/common.jsonnet

Please rebase the branch on master, we shouldn't change anything under ci/graal

Fixed except for common.json, the build fails if jvmci version doesn't match the one that mx downloads

timfel · 2025-12-10T12:46:07Z

ci/python-gate.libsonnet

        windows: {
            common: {
-                PATH: "$MAVEN_HOME\\bin;$PATH",
+                PATH: "$MAVEN_HOME\\bin;$PATH;C:\\hostedtoolcache\\windows\\Python\\3.12.3\\x64\\Scripts;C:\\hostedtoolcache\\windows\\Python\\3.12.3\\x64;C:\\Program Files\\PowerShell\\7;C:\\windows\\system32;C:\\windows;C:\\windows\\System32\\Wbem;C:\\windows\\System32\\WindowsPowerShell\\v1.0\\;C:\\windows\\System32\\OpenSSH\\;C:\\Program Files\\dotnet\\;C:\\Program Files\\Microsoft SQL Server\\150\\Tools\\Binn\\;C:\\Program Files\\Git\\cmd;C:\\Program Files\\Git\\usr\\bin;C:\\hostedtoolcache\\windows\\Ruby\\3.2.3\\x64\\bin;C:\\hostedtoolcache\\windows\\Go\\1.22.2\\x64\\bin;C:\\hostedtoolcache\\windows\\Node\\20.11.1\\x64\\;C:\\npm\\prefix;C:\\Program Files\\Amazon\\AWSCLIV2\\;C:\\Program Files\\Amazon\\SessionManagerPlugin\\bin\\;C:\\Program Files\\LLVM\\bin;C:\\Program Files\\CMake\\bin;C:\\Program Files\\OpenSSL\\bin;C:\\msys64\\usr\\bin;C:\\hostedtoolcache\\windows\\php\\8.2.15\\x64;C:\\Program Files\\Azure DevOps CLI\\bin;C:\\hostedtoolcache\\windows\\R\\4.3.3\\x64\\bin;C:\\Program Files (x86)\\Microsoft Visual Studio\\2022\\Enterprise\\Common7\\IDE\\Extensions\\Microsoft\\NuGet;",


That seems like a hack, why is it needed?

Looks like PATH isn't expanded and is set to PATH=$PATH as 'text'
Got issues such as tar, git, mvn not found

edit: PATH=$PATH comes from the common env a few lines before

timfel · 2025-12-10T12:46:38Z

ci/python-gate.libsonnet

        local artifact_name = name + os + arch,
        setup+: [
-            ["unpack-artifact", artifact_name],
+            [".github/scripts/unpack-artifact", artifact_name],


these paths shouldn't change, .github/scripts should just be on the PATH in our github jobs

graalpython/com.oracle.graal.python.test/src/runner.py

timfel · 2025-12-10T12:48:13Z

graalpython/com.oracle.graal.python.test/src/runner.py

The changes in this file look ok to me, but I'll defer to @msimacek for a proper review of this

timfel · 2025-12-10T12:50:00Z

graalpython/com.oracle.graal.python.test/src/tests/unittest_tags/test_atexit.txt

Looks like our current retagger merge script sorts the platforms differently. Can you work with @msimacek to make sure we don't get into a merge war while we still run the retagger for some platforms (darwin) internally only?

I think i know why the sorted output is different, i'll fix it

Signed-off-by: Ariouz <virgil.calvez@gmail.com>

This fixes: python-unittest-standalone by downloading and exporting GRADLE_JAVA_HOME NOTE: This only works on linux at this time, other platforms will follow WIP: Download Links depending on platform Todo: Build link using self.runs_on

…102056

… on Python

…jobs

Default PATH seems to not be evaluated even in defined in jsonnet file, manually set

…g them

Ariouz · 2025-12-10T15:41:56Z

note: Had to force push to rebase from master

List of fixed requested changes: Ariouz#14 Ariouz#15

…x rebase

oracle-contributor-agreement bot added the OCA Verified All contributors have signed the Oracle Contributor Agreement. label Dec 9, 2025

Ariouz force-pushed the vcalvez/github_ci branch 2 times, most recently from 49f0149 to b082c57 Compare December 10, 2025 07:52

timfel requested changes Dec 10, 2025

View reviewed changes

Ariouz and others added 26 commits December 10, 2025 16:05

Forked from oracle/graalpython - Initial Commit with ci

540b03b

Fix set-export / unpack-artifact permissions

c53861f

Disabled OOM/Buff Overflow tests

445be61

Signed-off-by: Ariouz <virgil.calvez@gmail.com>

Add phase shift warmup benchmark.

68c5dd4

Use succinct option format.

6781f3a

Add documentation.

aad9f30

Process jobs "Downloads" and extract 'key' to env

ff9f041

This fixes: python-unittest-standalone by downloading and exporting GRADLE_JAVA_HOME NOTE: This only works on linux at this time, other platforms will follow WIP: Download Links depending on platform Todo: Build link using self.runs_on

Fix PATH on ci

ff8d4c2

Retag bytecode DSL tests

4165828

Retag more Bytecode DSL tests

1d4d9bd

Retag more Bytecode DSL tests

bda8238

Fix test_custom_iterator_return

008a521

Retag more Bytecode DSL tests

3c75c08

Untag test failing in CI although it passes locally

2d81110

Ignore some flaky ZoneInfoTest tests

dd6ed4d

Remove transiently timing out test_threading:test_print_exception_gh_…

ed34a7f

…102056

Update TCK provider to communicate that subscripting works with types…

7156473

… on Python

Fix PATH error on Linux/MacOS + Update imports

614748b

Disable Linux aarch64 svm build

a494af4

Construct download links using string format + Disable linux aarch64 …

b742111

…jobs

Split python-unittest-retagger to 8 batches like python-tagged-unittests

9f55132

Build: disable -g and enable -Ob while GITHUB_CI env is set

ed53829

Windows set-export / unpack-artifact scripts

0706bfe

Add env: ${{ matrix.env }} to tier2 and 3 jobs

927f2e1

Default PATH seems to not be evaluated even in defined in jsonnet file, manually set

Add build arg -Ob on libpythonvm library build

59639ed

Add -J-XX:MaxRAMPercentage=90.0 on graalpy native / libpythonvm builds

fce92cc

Ariouz and others added 20 commits December 10, 2025 16:21

Enable Windows in ci-unittest workflow

0aadcf2

Update PR unittest trigger

9d17178

Rm debugs, fix github_ci_build_args not used

0df91d5

Apply retags for linux-x86_64

5e1461d

Apply retags for win32-AMD64

8ce1883

Update imports

4fbdefc

Revert 'update imports', mx labsjdk version isn't up to date yet.

f3f43f1

Fix psutil package removed by mx update imports

9ecb978

test_repr_deep broken after mx update ?

be4379c

Fix choco / maven in PATH

72adc6c

Remove comments

9de5ec0

Fix tagged test platform sort order

5a28761

Add retagger workflow input

b07efff

Revert retagger gate to weekly, allow weekly in matrix

6381b6e

Fix .github/scripts/ path in github path

eca2884

Run weekly jobs only in tier1 otherwise it would run multiple times

bb9345a

Build standalone artifacts before unittest/retags to avoid re-buildin…

b5d9b6f

…g them

Handle case when no tag has changed

62b58b4

tmp: Add debug to retagger

0c708df

Excluce 'bench' from pr unittest

22ee071

Ariouz force-pushed the vcalvez/github_ci branch from b082c57 to 22ee071 Compare December 10, 2025 15:28

Revert mx update in ci/graal/common.jsonnet

d591ea3

Ariouz force-pushed the vcalvez/github_ci branch from a7b09ad to d591ea3 Compare December 11, 2025 07:47

Ariouz added 6 commits December 11, 2025 08:55

Update msbuild action / graal versions

cd9a093

Fix empty weekly retagger branch

4b93051

Rebase ci/graal on master

6104bd6

Update microsoft/setup-msbuild version, fix empty retagger branch, fi…

7031498

…x rebase

Override linux aarch64 python version

469adc5

Override linux aarch64 python version

1391889

Draft: Github CI, PR checks, weekly retagger #574

Are you sure you want to change the base?

Draft: Github CI, PR checks, weekly retagger #574

Conversation

Ariouz commented Dec 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Ariouz Dec 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Ariouz Dec 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Ariouz Dec 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Ariouz commented Dec 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Ariouz commented Dec 9, 2025 •

edited

Loading

Ariouz Dec 10, 2025 •

edited

Loading

Ariouz Dec 10, 2025 •

edited

Loading

Ariouz Dec 10, 2025 •

edited

Loading

Ariouz commented Dec 10, 2025 •

edited

Loading