Add pull-gateway-api-inference-extension-test-unit-python-main for Python Unit Tests

[shotarok]

Resolve kubernetes-sigs/gateway-api-inference-extension#987

Add CI testing for the Python unit tests in tools/dynamic-lora-sidecar to run automatically on each PR. The dynamic-lora-sidecar has unit tests (test_sidecar.py). However, they're not run in CI. I confirmed the newly added job worked with Kind locally. I attached the outputs of the commands.

./pj-on-kind.sh pull-gateway-api-inference-extension-test-unit-python-main

laborant@docker-01:config$ ./pj-on-kind.sh pull-gateway-api-inference-extension-test-unit-python-main
job=pull-gateway-api-inference-extension-test-unit-python-main
CONFIG_PATH=/home/laborant/test-infra/config/prow/config.yaml
JOB_CONFIG_PATH=/home/laborant/test-infra/config/jobs
OUT_DIR=/mnt/disks/prowjob-out   (Only used when creating a new kind cluster.)
KIND_CONFIG=   (Only used when creating a new kind cluster.)
NODE_DIR=/mnt/disks/kind-node   (Only used when creating a new kind cluster.)
latest: Pulling from k8s-prow/mkpj
Digest: sha256:6e71c2d9f675bce23136443cd8093fad59ed3344325d305963e89eadb7d3b6d5
Status: Image is up to date for gcr.io/k8s-prow/mkpj:latest
gcr.io/k8s-prow/mkpj:latest
time="2025-06-20T08:16:43Z" level=warning msg="empty -github-token-path, will use anonymous github client"
time="2025-06-20T08:16:43Z" level=info msg="Throttle(0, 0, [])" client=github
PR Number: 1021
time="2025-06-20T08:16:47Z" level=info msg="GetPullRequest(kubernetes-sigs, gateway-api-inference-extension, 1021)" client=github
time="2025-06-20T08:16:47Z" level=debug msg="Using GitHub REST API Version: 2022-11-28" client=github
time="2025-06-20T08:16:48Z" level=debug msg="Used token for request" cache-mode= client=github method=GET path=/repos/kubernetes-sigs/gateway-api-inference-extension/pulls/1021 throttled=true
time="2025-06-20T08:16:48Z" level=debug msg="GetPullRequest(kubernetes-sigs, gateway-api-inference-extension, 1021) finished" client=github duration=386.994772ms
latest: Pulling from k8s-prow/mkpod
Digest: sha256:1a95d701f6cafae2c004088e47f15efc0b8e5b8d8fb6a23a2e2ccd94471eda46
Status: Image is up to date for gcr.io/k8s-prow/mkpod:latest
gcr.io/k8s-prow/mkpod:latest
time="2025-06-20T08:16:49Z" level=info msg="Generated build-id for job." build-id=1935975109177118720
time="2025-06-20T08:16:49Z" level=info msg="Pod-utils configured for local mode. Instead of uploading to GCS, files will be copied to an output dir on the node." out-dir=/mnt/disks/prowjob-out/pull-gateway-api-inference-extension-test-unit-python-main/1935975109177118720
Applying pod to the mkpod cluster. Configure kubectl for the mkpod cluster with:
>  export KUBECONFIG='/home/laborant/.kube/kind-config-mkpod'
NAME                                   READY   STATUS     RESTARTS   AGE
f6a3c329-6d30-49bb-943c-2564a7882e62   0/2     Init:0/3   0          0s
f6a3c329-6d30-49bb-943c-2564a7882e62   0/2     Init:0/3   0          1s
f6a3c329-6d30-49bb-943c-2564a7882e62   0/2     Init:1/3   0          4s
f6a3c329-6d30-49bb-943c-2564a7882e62   0/2     Init:2/3   0          5s
f6a3c329-6d30-49bb-943c-2564a7882e62   0/2     PodInitializing   0          6s
f6a3c329-6d30-49bb-943c-2564a7882e62   2/2     Running           0          7s
f6a3c329-6d30-49bb-943c-2564a7882e62   0/2     Completed         0          17s
f6a3c329-6d30-49bb-943c-2564a7882e62   0/2     Completed         0          19s

k logs f6a3c329-6d30-49bb-943c-2564a7882e62

laborant@docker-01:~$ k logs f6a3c329-6d30-49bb-943c-2564a7882e62
Defaulted container "test" out of: test, sidecar, clonerefs (init), initupload (init), place-entrypoint (init)
python3.10 -m venv venv
venv/bin/pip install --upgrade pip
Requirement already satisfied: pip in ./venv/lib/python3.10/site-packages (23.0.1)
Collecting pip
  Downloading pip-25.1.1-py3-none-any.whl (1.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.8/1.8 MB 27.7 MB/s eta 0:00:00
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 23.0.1
    Uninstalling pip-23.0.1:
      Successfully uninstalled pip-23.0.1
Successfully installed pip-25.1.1
venv/bin/pip install -r requirements.txt
Collecting aiohttp==3.12.12 (from -r requirements.txt (line 1))
  Downloading aiohttp-3.12.12-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (7.6 kB)
Collecting jsonschema==4.24.0 (from -r requirements.txt (line 2))
  Downloading jsonschema-4.24.0-py3-none-any.whl.metadata (7.8 kB)
Collecting prometheus_client==0.22.1 (from -r requirements.txt (line 3))
  Downloading prometheus_client-0.22.1-py3-none-any.whl.metadata (1.9 kB)
Collecting PyYAML==6.0.2 (from -r requirements.txt (line 4))
  Downloading PyYAML-6.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (2.1 kB)
Collecting requests==2.32.4 (from -r requirements.txt (line 5))
  Downloading requests-2.32.4-py3-none-any.whl.metadata (4.9 kB)
Collecting watchfiles==1.0.5 (from -r requirements.txt (line 6))
  Downloading watchfiles-1.0.5-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.9 kB)
Collecting watchdog==6.0.0 (from -r requirements.txt (line 7))
  Downloading watchdog-6.0.0-py3-none-manylinux2014_x86_64.whl.metadata (44 kB)
Collecting aiohappyeyeballs>=2.5.0 (from aiohttp==3.12.12->-r requirements.txt (line 1))
  Downloading aiohappyeyeballs-2.6.1-py3-none-any.whl.metadata (5.9 kB)
Collecting aiosignal>=1.1.2 (from aiohttp==3.12.12->-r requirements.txt (line 1))
  Downloading aiosignal-1.3.2-py2.py3-none-any.whl.metadata (3.8 kB)
Collecting async-timeout<6.0,>=4.0 (from aiohttp==3.12.12->-r requirements.txt (line 1))
  Downloading async_timeout-5.0.1-py3-none-any.whl.metadata (5.1 kB)
Collecting attrs>=17.3.0 (from aiohttp==3.12.12->-r requirements.txt (line 1))
  Downloading attrs-25.3.0-py3-none-any.whl.metadata (10 kB)
Collecting frozenlist>=1.1.1 (from aiohttp==3.12.12->-r requirements.txt (line 1))
  Downloading frozenlist-1.7.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (18 kB)
Collecting multidict<7.0,>=4.5 (from aiohttp==3.12.12->-r requirements.txt (line 1))
  Downloading multidict-6.5.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (5.3 kB)
Collecting propcache>=0.2.0 (from aiohttp==3.12.12->-r requirements.txt (line 1))
  Downloading propcache-0.3.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (12 kB)
Collecting yarl<2.0,>=1.17.0 (from aiohttp==3.12.12->-r requirements.txt (line 1))
  Downloading yarl-1.20.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (73 kB)
Collecting jsonschema-specifications>=2023.03.6 (from jsonschema==4.24.0->-r requirements.txt (line 2))
  Downloading jsonschema_specifications-2025.4.1-py3-none-any.whl.metadata (2.9 kB)
Collecting referencing>=0.28.4 (from jsonschema==4.24.0->-r requirements.txt (line 2))
  Downloading referencing-0.36.2-py3-none-any.whl.metadata (2.8 kB)
Collecting rpds-py>=0.7.1 (from jsonschema==4.24.0->-r requirements.txt (line 2))
  Downloading rpds_py-0.25.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.1 kB)
Collecting charset_normalizer<4,>=2 (from requests==2.32.4->-r requirements.txt (line 5))
  Downloading charset_normalizer-3.4.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (35 kB)
Collecting idna<4,>=2.5 (from requests==2.32.4->-r requirements.txt (line 5))
  Downloading idna-3.10-py3-none-any.whl.metadata (10 kB)
Collecting urllib3<3,>=1.21.1 (from requests==2.32.4->-r requirements.txt (line 5))
  Downloading urllib3-2.5.0-py3-none-any.whl.metadata (6.5 kB)
Collecting certifi>=2017.4.17 (from requests==2.32.4->-r requirements.txt (line 5))
  Downloading certifi-2025.6.15-py3-none-any.whl.metadata (2.4 kB)
Collecting anyio>=3.0.0 (from watchfiles==1.0.5->-r requirements.txt (line 6))
  Downloading anyio-4.9.0-py3-none-any.whl.metadata (4.7 kB)
Collecting typing-extensions>=4.1.0 (from multidict<7.0,>=4.5->aiohttp==3.12.12->-r requirements.txt (line 1))
  Downloading typing_extensions-4.14.0-py3-none-any.whl.metadata (3.0 kB)
Collecting exceptiongroup>=1.0.2 (from anyio>=3.0.0->watchfiles==1.0.5->-r requirements.txt (line 6))
  Downloading exceptiongroup-1.3.0-py3-none-any.whl.metadata (6.7 kB)
Collecting sniffio>=1.1 (from anyio>=3.0.0->watchfiles==1.0.5->-r requirements.txt (line 6))
  Downloading sniffio-1.3.1-py3-none-any.whl.metadata (3.9 kB)
Downloading aiohttp-3.12.12-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.6/1.6 MB 39.3 MB/s eta 0:00:00
Downloading jsonschema-4.24.0-py3-none-any.whl (88 kB)
Downloading prometheus_client-0.22.1-py3-none-any.whl (58 kB)
Downloading PyYAML-6.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (751 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 751.2/751.2 kB 44.2 MB/s eta 0:00:00
Downloading requests-2.32.4-py3-none-any.whl (64 kB)
Downloading watchfiles-1.0.5-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (454 kB)
Downloading watchdog-6.0.0-py3-none-manylinux2014_x86_64.whl (79 kB)
Downloading async_timeout-5.0.1-py3-none-any.whl (6.2 kB)
Downloading charset_normalizer-3.4.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (149 kB)
Downloading idna-3.10-py3-none-any.whl (70 kB)
Downloading multidict-6.5.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (226 kB)
Downloading urllib3-2.5.0-py3-none-any.whl (129 kB)
Downloading yarl-1.20.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (326 kB)
Downloading aiohappyeyeballs-2.6.1-py3-none-any.whl (15 kB)
Downloading aiosignal-1.3.2-py2.py3-none-any.whl (7.6 kB)
Downloading anyio-4.9.0-py3-none-any.whl (100 kB)
Downloading attrs-25.3.0-py3-none-any.whl (63 kB)
Downloading certifi-2025.6.15-py3-none-any.whl (157 kB)
Downloading exceptiongroup-1.3.0-py3-none-any.whl (16 kB)
Downloading frozenlist-1.7.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (222 kB)
Downloading jsonschema_specifications-2025.4.1-py3-none-any.whl (18 kB)
Downloading propcache-0.3.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (198 kB)
Downloading referencing-0.36.2-py3-none-any.whl (26 kB)
Downloading rpds_py-0.25.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (386 kB)
Downloading sniffio-1.3.1-py3-none-any.whl (10 kB)
Downloading typing_extensions-4.14.0-py3-none-any.whl (43 kB)
Installing collected packages: watchdog, urllib3, typing-extensions, sniffio, rpds-py, PyYAML, propcache, prometheus_client, idna, frozenlist, charset_normalizer, certifi, attrs, async-timeout, aiohappyeyeballs, requests, referencing, multidict, exceptiongroup, aiosignal, yarl, jsonschema-specifications, anyio, watchfiles, jsonschema, aiohttp

Successfully installed PyYAML-6.0.2 aiohappyeyeballs-2.6.1 aiohttp-3.12.12 aiosignal-1.3.2 anyio-4.9.0 async-timeout-5.0.1 attrs-25.3.0 certifi-2025.6.15 charset_normalizer-3.4.2 exceptiongroup-1.3.0 frozenlist-1.7.0 idna-3.10 jsonschema-4.24.0 jsonschema-specifications-2025.4.1 multidict-6.5.0 prometheus_client-0.22.1 propcache-0.3.2 referencing-0.36.2 requests-2.32.4 rpds-py-0.25.1 sniffio-1.3.1 typing-extensions-4.14.0 urllib3-2.5.0 watchdog-6.0.0 watchfiles-1.0.5 yarl-1.20.1
venv/bin/python -m unittest discover -v -s sidecar
test_health_check_settings (test_sidecar.LoraReconcilerTest)
Test that health check settings are properly initialized from command line args ... 2025-06-20 08:17:05 - INFO - sidecar.py:118 -  Settings initialized: health check timeout=180s, interval=10s, reconcile trigger=30s
2025-06-20 08:17:05 - INFO - sidecar.py:118 -  Settings initialized: health check timeout=240s, interval=15s, reconcile trigger=45s
ok
test_load_adapter (test_sidecar.LoraReconcilerTest) ... 2025-06-20 08:17:05 - INFO - sidecar.py:118 -  Settings initialized: health check timeout=180s, interval=10s, reconcile trigger=30s
2025-06-20 08:17:05 - INFO - sidecar.py:255 -  loaded model sql-lora-v1
2025-06-20 08:17:05 - INFO - sidecar.py:242 -  already_exists already present on model server localhost:8000
ok
test_metrics_endpoint (test_sidecar.LoraReconcilerTest)
Test that Prometheus metrics can be collected ... 2025-06-20 08:17:05 - INFO - sidecar.py:118 -  Settings initialized: health check timeout=180s, interval=10s, reconcile trigger=30s
ok
test_reconcile (test_sidecar.LoraReconcilerTest) ... 2025-06-20 08:17:05 - INFO - sidecar.py:118 -  Settings initialized: health check timeout=180s, interval=10s, reconcile trigger=30s
2025-06-20 08:17:05 - INFO - sidecar.py:118 -  Settings initialized: health check timeout=180s, interval=10s, reconcile trigger=30s
2025-06-20 08:17:05 - INFO - sidecar.py:281 -  reconciling model server localhost:8000 with config stored at /config/configmap.yaml
2025-06-20 08:17:05 - WARNING - sidecar.py:292 -  skipped adapters found in both `ensureExist` and `ensureNotExist` sql-lora-v3
2025-06-20 08:17:05 - INFO - sidecar.py:297 -  adapter to load already_exists, sql-lora-v1
2025-06-20 08:17:05 - INFO - sidecar.py:304 -  adapters to unload to_remove, sql-lora-v2
ok
test_unload_adapter (test_sidecar.LoraReconcilerTest) ... 2025-06-20 08:17:05 - INFO - sidecar.py:118 -  Settings initialized: health check timeout=180s, interval=10s, reconcile trigger=30s
2025-06-20 08:17:06 - INFO - sidecar.py:273 -  unloaded model to_remove
2025-06-20 08:17:06 - INFO - sidecar.py:264 -  sql-lora-v2 already doesn't exist on model server localhost:8000
ok
test_update_adapter_status_metrics (test_sidecar.LoraReconcilerTest)
Test that update_adapter_status_metrics method works correctly ... 2025-06-20 08:17:06 - INFO - sidecar.py:118 -  Settings initialized: health check timeout=180s, interval=10s, reconcile trigger=30s
2025-06-20 08:17:06 - INFO - sidecar.py:118 -  Settings initialized: health check timeout=180s, interval=10s, reconcile trigger=30s
ok

----------------------------------------------------------------------
Ran 6 tests in 0.112s

OK
laborant@docker-01:~$

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: shotarok
Once this PR has been reviewed and has the lgtm label, please assign kfswain for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• config/jobs/kubernetes-sigs/gateway-api-inference-extension/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[k8s-ci-robot]

Welcome @shotarok!

It looks like this is your first PR to kubernetes/test-infra 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes/test-infra has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

[k8s-ci-robot]

Hi @shotarok. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.