Skip to content

Add GitHub Action to run testsuite on PRs#734

Merged
averevki merged 1 commit intoKuadrant:mainfrom
averevki:kind-gh-action
Aug 27, 2025
Merged

Add GitHub Action to run testsuite on PRs#734
averevki merged 1 commit intoKuadrant:mainfrom
averevki:kind-gh-action

Conversation

@averevki
Copy link
Copy Markdown
Contributor

@averevki averevki commented Aug 26, 2025

Summary

This PR adds a GitHub Actions workflow that enables running testsuite commands on pull requests through issue comments. The workflow provides a convenient way to test PRs without requiring manual setup of test environments.

Changes Made

  • Added .github/workflows/test-pr.yml: Complete GitHub Actions workflow for PR testing
    • Triggers on issue comments starting with /make <target>
    • Validates allowed make targets against predefined list
    • Sets up complete KIND cluster with required infrastructure
    • Installs and configures all dependencies (MetalLB, Gateway API, cert-manager, Sail Operator, Istio, Kuadrant)
    • Creates necessary namespaces and secrets for testing
    • Runs the specified make target and reports results back to the PR

Verification Steps

Eye review is sufficient for this PR. The workflow is self-contained and includes comprehensive environment setup that mirrors the existing testsuite requirements.

Future works

  • Create helm chart for kind testsuite env deployment
  • Fix metrics tests on kind by configuring prometheus

Closes #735 Closes Kuadrant/architecture#43

@averevki
Copy link
Copy Markdown
Contributor Author

Test run completed (make authorino-standalone) and can be found here

Short Test Summary
=========================== short test summary info ============================
FAILED testsuite/tests/singlecluster/authorino/identity/plain/test_jwt_user_story.py::test_jwt_user_story - assert 500 == 200
 +  where 500 = <testsuite.httpx.Result object at 0x7fbb8338f890>.status_code
FAILED testsuite/tests/singlecluster/authorino/operator/tracing/test_tracing.py::test_tracing - assert 0 == 1
 +  where 0 = len([])
FAILED testsuite/tests/singlecluster/authorino/operator/tracing/test_tracing_tags.py::test_tracing_tags - assert 0 == 1
 +  where 0 = len([])
Full Output
poetry lock
Creating virtualenv kuadrant-testsuite-6EPd9kcO-py3.12 in /home/runner/.cache/pypoetry/virtualenvs
Updating dependencies
Resolving dependencies...

Writing lock file
Installing dependencies from lock file

Package operations: 46 installs, 0 updates, 0 removals

  - Installing pycparser (2.22)
  - Installing certifi (2025.8.3)
  - Installing cffi (1.17.1)
  - Installing charset-normalizer (3.4.3)
  - Installing h11 (0.16.0)
  - Installing idna (3.10)
  - Installing iniconfig (2.1.0)
  - Installing packaging (25.0)
  - Installing pluggy (1.6.0)
  - Installing pygments (2.19.2)
  - Installing sniffio (1.3.1)
  - Installing typing-extensions (4.15.0)
  - Installing urllib3 (2.5.0)
  - Installing anyio (4.10.0)
  - Installing cryptography (45.0.6)
  - Installing hpack (4.1.0)
  - Installing httpcore (1.0.9)
  - Installing hyperframe (6.1.0)
  - Installing markupsafe (3.0.2)
  - Installing pytest (8.4.1)
  - Installing pyproject-hooks (1.2.0)
  - Installing requests (2.32.5)
  - Installing aiofiles (24.1.0)
  - Installing build (1.3.0)
  - Installing async-property (0.2.2)
  - Installing deprecation (2.1.0)
  - Installing execnet (2.1.1)
  - Installing h2 (4.3.0)
  - Installing httpx (0.28.1)
  - Installing jinja2 (3.1.6)
  - Installing jwcrypto (1.5.6)
  - Installing pytest-metadata (3.1.1)
  - Installing pyyaml (6.0.2)
  - Installing requests-toolbelt (1.0.0)
  - Installing six (1.17.0)
  - Installing apyproxy (0.2.2)
  - Installing backoff (2.2.1)
  - Installing dnspython (2.7.0)
  - Installing dynaconf (3.2.11)
  - Installing lxml (6.0.1)
  - Installing pyjwt (2.10.1)
  - Installing pytest-html (4.1.1)
  - Installing openshift-client (2.0.5)
  - Installing pytest-xdist (3.8.0)
  - Installing python-keycloak (5.8.1)
  - Installing weakget (1.0)
poetry run python -m pytest --tb=short -o cache_dir=./.pytest_cache.authorino-standalone -n4 -m 'authorino and not kuadrant_only' --dist loadfile --enforce --standalone -vv testsuite/tests/singlecluster/authorino
============================= test session starts ==============================
platform linux -- Python 3.12.3, pytest-8.4.1, pluggy-1.6.0 -- /home/runner/.cache/pypoetry/virtualenvs/kuadrant-testsuite-6EPd9kcO-py3.12/bin/python
cachedir: .pytest_cache.authorino-standalone
metadata: {'Python': '3.12.3', 'Platform': 'Linux-6.11.0-1018-azure-x86_64-with-glibc2.39', 'Packages': {'pytest': '8.4.1', 'pluggy': '1.6.0'}, 'Plugins': {'xdist': '3.8.0', 'metadata': '3.1.1', 'anyio': '4.10.0', 'html': '4.1.1'}, 'CI': 'true', 'JAVA_HOME': '/usr/lib/jvm/temurin-17-jdk-amd64', 'Kuadrant': []}

rootdir: /home/runner/work/testsuite/testsuite
configfile: pyproject.toml
plugins: xdist-3.8.0, metadata-3.1.1, anyio-4.10.0, html-4.1.1
created: 4/4 workers
4 workers [206 items]

scheduling tests via LoadFileScheduling

testsuite/tests/singlecluster/authorino/identity/keycloak/test_auth_credentials.py::test_custom_selector[authorizationHeader] 
testsuite/tests/singlecluster/authorino/metrics/test_metrics.py::test_metrics[controller_runtime_reconcile_total] 
testsuite/tests/singlecluster/authorino/dinosaur/test_dinosaur.py::test_deny_email 
testsuite/tests/singlecluster/authorino/identity/api_key/test_auth_credentials.py::test_custom_selector[authorizationHeader] 
[gw1] [  0%] SKIPPED testsuite/tests/singlecluster/authorino/metrics/test_metrics.py::test_metrics[controller_runtime_reconcile_total] 
testsuite/tests/singlecluster/authorino/metrics/test_metrics.py::test_metrics[controller_runtime_reconcile_errors_total] 
[gw1] [  0%] SKIPPED testsuite/tests/singlecluster/authorino/metrics/test_metrics.py::test_metrics[controller_runtime_reconcile_errors_total] 
testsuite/tests/singlecluster/authorino/metrics/test_metrics.py::test_metrics[controller_runtime_max_concurrent_reconciles] 
[gw1] [  1%] SKIPPED testsuite/tests/singlecluster/authorino/metrics/test_metrics.py::test_metrics[controller_runtime_max_concurrent_reconciles] 
testsuite/tests/singlecluster/authorino/metrics/test_metrics.py::test_metrics[workqueue_adds_total] 
[gw1] [  1%] SKIPPED testsuite/tests/singlecluster/authorino/metrics/test_metrics.py::test_metrics[workqueue_adds_total] 
testsuite/tests/singlecluster/authorino/metrics/test_metrics.py::test_metrics[workqueue_depth] 
[gw1] [  2%] SKIPPED testsuite/tests/singlecluster/authorino/metrics/test_metrics.py::test_metrics[workqueue_depth] 
testsuite/tests/singlecluster/authorino/metrics/test_metrics.py::test_metrics[workqueue_longest_running_processor_seconds] 
[gw1] [  2%] SKIPPED testsuite/tests/singlecluster/authorino/metrics/test_metrics.py::test_metrics[workqueue_longest_running_processor_seconds] 
testsuite/tests/singlecluster/authorino/metrics/test_metrics.py::test_metrics[workqueue_retries_total] 
[gw1] [  3%] SKIPPED testsuite/tests/singlecluster/authorino/metrics/test_metrics.py::test_metrics[workqueue_retries_total] 
testsuite/tests/singlecluster/authorino/metrics/test_metrics.py::test_metrics[workqueue_unfinished_work_seconds] 
[gw1] [  3%] SKIPPED testsuite/tests/singlecluster/authorino/metrics/test_metrics.py::test_metrics[workqueue_unfinished_work_seconds] 
testsuite/tests/singlecluster/authorino/metrics/test_metrics.py::test_metrics[rest_client_requests_total] 
[gw1] [  4%] SKIPPED testsuite/tests/singlecluster/authorino/metrics/test_metrics.py::test_metrics[rest_client_requests_total] 
testsuite/tests/singlecluster/authorino/metrics/test_metrics.py::test_metrics_histogram[controller_runtime_reconcile_time_seconds] 
[gw1] [  4%] SKIPPED testsuite/tests/singlecluster/authorino/metrics/test_metrics.py::test_metrics_histogram[controller_runtime_reconcile_time_seconds] 
testsuite/tests/singlecluster/authorino/metrics/test_metrics.py::test_metrics_histogram[workqueue_queue_duration_seconds] 
[gw1] [  5%] SKIPPED testsuite/tests/singlecluster/authorino/metrics/test_metrics.py::test_metrics_histogram[workqueue_queue_duration_seconds] 
testsuite/tests/singlecluster/authorino/metrics/test_metrics.py::test_metrics_histogram[workqueue_work_duration_seconds] 
[gw1] [  5%] SKIPPED testsuite/tests/singlecluster/authorino/metrics/test_metrics.py::test_metrics_histogram[workqueue_work_duration_seconds] 
testsuite/tests/singlecluster/authorino/metrics/test_metrics.py::test_server_metrics[auth_server_authconfig_total] 
[gw1] [  6%] SKIPPED testsuite/tests/singlecluster/authorino/metrics/test_metrics.py::test_server_metrics[auth_server_authconfig_total] 
testsuite/tests/singlecluster/authorino/metrics/test_metrics.py::test_server_metrics[auth_server_authconfig_response_status] 
[gw1] [  6%] SKIPPED testsuite/tests/singlecluster/authorino/metrics/test_metrics.py::test_server_metrics[auth_server_authconfig_response_status] 
testsuite/tests/singlecluster/authorino/metrics/test_metrics.py::test_server_metrics[auth_server_response_status] 
[gw1] [  7%] SKIPPED testsuite/tests/singlecluster/authorino/metrics/test_metrics.py::test_server_metrics[auth_server_response_status] 
testsuite/tests/singlecluster/authorino/metrics/test_metrics.py::test_server_metrics[grpc_server_handled_total] 
[gw1] [  7%] SKIPPED testsuite/tests/singlecluster/authorino/metrics/test_metrics.py::test_server_metrics[grpc_server_handled_total] 
testsuite/tests/singlecluster/authorino/metrics/test_metrics.py::test_server_metrics[grpc_server_msg_received_total] 
[gw1] [  8%] SKIPPED testsuite/tests/singlecluster/authorino/metrics/test_metrics.py::test_server_metrics[grpc_server_msg_received_total] 
testsuite/tests/singlecluster/authorino/metrics/test_metrics.py::test_server_metrics[grpc_server_msg_sent_total] 
[gw1] [  8%] SKIPPED testsuite/tests/singlecluster/authorino/metrics/test_metrics.py::test_server_metrics[grpc_server_msg_sent_total] 
testsuite/tests/singlecluster/authorino/metrics/test_metrics.py::test_server_metrics[grpc_server_started_total] 
[gw1] [  9%] SKIPPED testsuite/tests/singlecluster/authorino/metrics/test_metrics.py::test_server_metrics[grpc_server_started_total] 
testsuite/tests/singlecluster/authorino/metrics/test_metrics.py::test_server_metrics_histogram[auth_server_authconfig_duration_seconds] 
[gw1] [  9%] SKIPPED testsuite/tests/singlecluster/authorino/metrics/test_metrics.py::test_server_metrics_histogram[auth_server_authconfig_duration_seconds] 
testsuite/tests/singlecluster/authorino/metrics/test_metrics.py::test_server_metrics_histogram[grpc_server_handling_seconds] 
[gw1] [ 10%] SKIPPED testsuite/tests/singlecluster/authorino/metrics/test_metrics.py::test_server_metrics_histogram[grpc_server_handling_seconds] 
testsuite/tests/singlecluster/authorino/conditions/test_patternref_expressions.py::test_get_rule[/get-403] 
[gw3] [ 10%] PASSED testsuite/tests/singlecluster/authorino/identity/keycloak/test_auth_credentials.py::test_custom_selector[authorizationHeader] 
testsuite/tests/singlecluster/authorino/identity/keycloak/test_auth_credentials.py::test_custom_header[authorizationHeader] 
[gw3] [ 11%] PASSED testsuite/tests/singlecluster/authorino/identity/keycloak/test_auth_credentials.py::test_custom_header[authorizationHeader] 
testsuite/tests/singlecluster/authorino/identity/keycloak/test_auth_credentials.py::test_query[authorizationHeader] 
[gw3] [ 11%] PASSED testsuite/tests/singlecluster/authorino/identity/keycloak/test_auth_credentials.py::test_query[authorizationHeader] 
testsuite/tests/singlecluster/authorino/identity/keycloak/test_auth_credentials.py::test_cookie[authorizationHeader] 
[gw3] [ 12%] PASSED testsuite/tests/singlecluster/authorino/identity/keycloak/test_auth_credentials.py::test_cookie[authorizationHeader] 
testsuite/tests/singlecluster/authorino/identity/keycloak/test_auth_credentials.py::test_custom_selector[customHeader] 
[gw3] [ 12%] PASSED testsuite/tests/singlecluster/authorino/identity/keycloak/test_auth_credentials.py::test_custom_selector[customHeader] 
testsuite/tests/singlecluster/authorino/identity/keycloak/test_auth_credentials.py::test_custom_header[customHeader] 
[gw3] [ 13%] PASSED testsuite/tests/singlecluster/authorino/identity/keycloak/test_auth_credentials.py::test_custom_header[customHeader] 
testsuite/tests/singlecluster/authorino/identity/keycloak/test_auth_credentials.py::test_query[customHeader] 
[gw3] [ 13%] PASSED testsuite/tests/singlecluster/authorino/identity/keycloak/test_auth_credentials.py::test_query[customHeader] 
testsuite/tests/singlecluster/authorino/identity/keycloak/test_auth_credentials.py::test_cookie[customHeader] 
[gw3] [ 14%] PASSED testsuite/tests/singlecluster/authorino/identity/keycloak/test_auth_credentials.py::test_cookie[customHeader] 
testsuite/tests/singlecluster/authorino/identity/keycloak/test_auth_credentials.py::test_custom_selector[queryString] 
[gw3] [ 14%] PASSED testsuite/tests/singlecluster/authorino/identity/keycloak/test_auth_credentials.py::test_custom_selector[queryString] 
testsuite/tests/singlecluster/authorino/identity/keycloak/test_auth_credentials.py::test_custom_header[queryString] 
[gw3] [ 15%] PASSED testsuite/tests/singlecluster/authorino/identity/keycloak/test_auth_credentials.py::test_custom_header[queryString] 
testsuite/tests/singlecluster/authorino/identity/keycloak/test_auth_credentials.py::test_query[queryString] 
[gw3] [ 15%] PASSED testsuite/tests/singlecluster/authorino/identity/keycloak/test_auth_credentials.py::test_query[queryString] 
testsuite/tests/singlecluster/authorino/identity/keycloak/test_auth_credentials.py::test_cookie[queryString] 
[gw3] [ 16%] PASSED testsuite/tests/singlecluster/authorino/identity/keycloak/test_auth_credentials.py::test_cookie[queryString] 
testsuite/tests/singlecluster/authorino/identity/keycloak/test_auth_credentials.py::test_custom_selector[cookie] 
[gw1] [ 16%] PASSED testsuite/tests/singlecluster/authorino/conditions/test_patternref_expressions.py::test_get_rule[/get-403] 
testsuite/tests/singlecluster/authorino/conditions/test_patternref_expressions.py::test_get_rule[/anything/rock-403] 
[gw1] [ 16%] PASSED testsuite/tests/singlecluster/authorino/conditions/test_patternref_expressions.py::test_get_rule[/anything/rock-403] 
testsuite/tests/singlecluster/authorino/conditions/test_patternref_expressions.py::test_get_rule[/anything/apple-403] 
[gw1] [ 17%] PASSED testsuite/tests/singlecluster/authorino/conditions/test_patternref_expressions.py::test_get_rule[/anything/apple-403] 
testsuite/tests/singlecluster/authorino/conditions/test_patternref_expressions.py::test_get_rule[/anything/pear-403] 
[gw1] [ 17%] PASSED testsuite/tests/singlecluster/authorino/conditions/test_patternref_expressions.py::test_get_rule[/anything/pear-403] 
testsuite/tests/singlecluster/authorino/conditions/test_patternref_expressions.py::test_get_rule[/anything/dog-200] 
[gw1] [ 18%] PASSED testsuite/tests/singlecluster/authorino/conditions/test_patternref_expressions.py::test_get_rule[/anything/dog-200] 
testsuite/tests/singlecluster/authorino/conditions/test_patternref_expressions.py::test_get_rule[/anything/cat-200] 
[gw1] [ 18%] PASSED testsuite/tests/singlecluster/authorino/conditions/test_patternref_expressions.py::test_get_rule[/anything/cat-200] 
testsuite/tests/singlecluster/authorino/conditions/test_patternref_expressions.py::test_post_rule[/post-403] 
[gw0] [ 19%] PASSED testsuite/tests/singlecluster/authorino/dinosaur/test_dinosaur.py::test_deny_email 
[gw1] [ 19%] PASSED testsuite/tests/singlecluster/authorino/conditions/test_patternref_expressions.py::test_post_rule[/post-403] 
testsuite/tests/singlecluster/authorino/dinosaur/test_dinosaur.py::test_allow_org_id 
testsuite/tests/singlecluster/authorino/conditions/test_patternref_expressions.py::test_post_rule[/anything/rock-403] 
[gw1] [ 20%] PASSED testsuite/tests/singlecluster/authorino/conditions/test_patternref_expressions.py::test_post_rule[/anything/rock-403] 
testsuite/tests/singlecluster/authorino/conditions/test_patternref_expressions.py::test_post_rule[/anything/apple-200] 
[gw1] [ 20%] PASSED testsuite/tests/singlecluster/authorino/conditions/test_patternref_expressions.py::test_post_rule[/anything/apple-200] 
testsuite/tests/singlecluster/authorino/conditions/test_patternref_expressions.py::test_post_rule[/anything/pear-200] 
[gw1] [ 21%] PASSED testsuite/tests/singlecluster/authorino/conditions/test_patternref_expressions.py::test_post_rule[/anything/pear-200] 
testsuite/tests/singlecluster/authorino/conditions/test_patternref_expressions.py::test_post_rule[/anything/dog-403] 
[gw1] [ 21%] PASSED testsuite/tests/singlecluster/authorino/conditions/test_patternref_expressions.py::test_post_rule[/anything/dog-403] 
testsuite/tests/singlecluster/authorino/conditions/test_patternref_expressions.py::test_post_rule[/anything/cat-403] 
[gw1] [ 22%] PASSED testsuite/tests/singlecluster/authorino/conditions/test_patternref_expressions.py::test_post_rule[/anything/cat-403] 
testsuite/tests/singlecluster/authorino/conditions/test_patternref_expressions.py::test_special_header_rule 
[gw1] [ 22%] PASSED testsuite/tests/singlecluster/authorino/conditions/test_patternref_expressions.py::test_special_header_rule 
testsuite/tests/singlecluster/authorino/operator/tls/mtls/test_mtls_identity.py::test_mtls_success 
[gw0] [ 23%] PASSED testsuite/tests/singlecluster/authorino/dinosaur/test_dinosaur.py::test_allow_org_id 
testsuite/tests/singlecluster/authorino/dinosaur/test_dinosaur.py::test_deny_invalid_org_id 
[gw0] [ 23%] PASSED testsuite/tests/singlecluster/authorino/dinosaur/test_dinosaur.py::test_deny_invalid_org_id 
testsuite/tests/singlecluster/authorino/dinosaur/test_dinosaur.py::test_admin_sso_get[user_with_full_role] 
[gw0] [ 24%] PASSED testsuite/tests/singlecluster/authorino/dinosaur/test_dinosaur.py::test_admin_sso_get[user_with_full_role] 
testsuite/tests/singlecluster/authorino/dinosaur/test_dinosaur.py::test_admin_sso_get[user_with_read_role] 
[gw3] [ 24%] PASSED testsuite/tests/singlecluster/authorino/identity/keycloak/test_auth_credentials.py::test_custom_selector[cookie] 
testsuite/tests/singlecluster/authorino/identity/keycloak/test_auth_credentials.py::test_custom_header[cookie] 
[gw3] [ 25%] PASSED testsuite/tests/singlecluster/authorino/identity/keycloak/test_auth_credentials.py::test_custom_header[cookie] 
testsuite/tests/singlecluster/authorino/identity/keycloak/test_auth_credentials.py::test_query[cookie] 
[gw3] [ 25%] PASSED testsuite/tests/singlecluster/authorino/identity/keycloak/test_auth_credentials.py::test_query[cookie] 
testsuite/tests/singlecluster/authorino/identity/keycloak/test_auth_credentials.py::test_cookie[cookie] 
[gw3] [ 26%] PASSED testsuite/tests/singlecluster/authorino/identity/keycloak/test_auth_credentials.py::test_cookie[cookie] 
[gw0] [ 26%] PASSED testsuite/tests/singlecluster/authorino/dinosaur/test_dinosaur.py::test_admin_sso_get[user_with_read_role] 
testsuite/tests/singlecluster/authorino/dinosaur/test_dinosaur.py::test_admin_sso_get[user_with_write_role] 
testsuite/tests/singlecluster/authorino/response/test_base64.py::test_base64[My name is "John"] 
[gw0] [ 27%] PASSED testsuite/tests/singlecluster/authorino/dinosaur/test_dinosaur.py::test_admin_sso_get[user_with_write_role] 
testsuite/tests/singlecluster/authorino/dinosaur/test_dinosaur.py::test_admin_sso_patch[user_with_full_role] 
[gw0] [ 27%] PASSED testsuite/tests/singlecluster/authorino/dinosaur/test_dinosaur.py::test_admin_sso_patch[user_with_full_role] 
testsuite/tests/singlecluster/authorino/dinosaur/test_dinosaur.py::test_admin_sso_patch[user_with_write_role] 
[gw0] [ 28%] PASSED testsuite/tests/singlecluster/authorino/dinosaur/test_dinosaur.py::test_admin_sso_patch[user_with_write_role] 
testsuite/tests/singlecluster/authorino/dinosaur/test_dinosaur.py::test_admin_sso_patch_deny 
[gw0] [ 28%] PASSED testsuite/tests/singlecluster/authorino/dinosaur/test_dinosaur.py::test_admin_sso_patch_deny 
testsuite/tests/singlecluster/authorino/dinosaur/test_dinosaur.py::test_admin_sso_delete 
[gw2] [ 29%] PASSED testsuite/tests/singlecluster/authorino/identity/api_key/test_auth_credentials.py::test_custom_selector[authorizationHeader] 
testsuite/tests/singlecluster/authorino/identity/api_key/test_auth_credentials.py::test_custom_header[authorizationHeader] 
[gw2] [ 29%] PASSED testsuite/tests/singlecluster/authorino/identity/api_key/test_auth_credentials.py::test_custom_header[authorizationHeader] 
testsuite/tests/singlecluster/authorino/identity/api_key/test_auth_credentials.py::test_query[authorizationHeader] 
[gw2] [ 30%] PASSED testsuite/tests/singlecluster/authorino/identity/api_key/test_auth_credentials.py::test_query[authorizationHeader] 
testsuite/tests/singlecluster/authorino/identity/api_key/test_auth_credentials.py::test_cookie[authorizationHeader] 
[gw0] [ 30%] PASSED testsuite/tests/singlecluster/authorino/dinosaur/test_dinosaur.py::test_admin_sso_delete 
testsuite/tests/singlecluster/authorino/dinosaur/test_dinosaur.py::test_admin_sso_delete_deny[user_with_read_role] 
[gw2] [ 31%] PASSED testsuite/tests/singlecluster/authorino/identity/api_key/test_auth_credentials.py::test_cookie[authorizationHeader] 
testsuite/tests/singlecluster/authorino/identity/api_key/test_auth_credentials.py::test_custom_selector[customHeader] 
[gw0] [ 31%] PASSED testsuite/tests/singlecluster/authorino/dinosaur/test_dinosaur.py::test_admin_sso_delete_deny[user_with_read_role] 
testsuite/tests/singlecluster/authorino/dinosaur/test_dinosaur.py::test_admin_sso_delete_deny[user_with_write_role] 
[gw0] [ 32%] PASSED testsuite/tests/singlecluster/authorino/dinosaur/test_dinosaur.py::test_admin_sso_delete_deny[user_with_write_role] 
testsuite/tests/singlecluster/authorino/dinosaur/test_dinosaur.py::test_authz 
[gw0] [ 32%] PASSED testsuite/tests/singlecluster/authorino/dinosaur/test_dinosaur.py::test_authz 
testsuite/tests/singlecluster/authorino/dinosaur/test_dinosaur.py::test_create 
[gw3] [ 33%] PASSED testsuite/tests/singlecluster/authorino/response/test_base64.py::test_base64[My name is "John"] 
testsuite/tests/singlecluster/authorino/response/test_base64.py::test_base64[My name is "John] 
[gw3] [ 33%] PASSED testsuite/tests/singlecluster/authorino/response/test_base64.py::test_base64[My name is "John] 
testsuite/tests/singlecluster/authorino/response/test_base64.py::test_base64[My name is 'John'] 
[gw3] [ 33%] PASSED testsuite/tests/singlecluster/authorino/response/test_base64.py::test_base64[My name is 'John'] 
testsuite/tests/singlecluster/authorino/response/test_base64.py::test_base64[My name is 'John] 
[gw3] [ 34%] PASSED testsuite/tests/singlecluster/authorino/response/test_base64.py::test_base64[My name is 'John] 
testsuite/tests/singlecluster/authorino/response/test_base64.py::test_base64[{"json": true}] 
[gw3] [ 34%] PASSED testsuite/tests/singlecluster/authorino/response/test_base64.py::test_base64[{"json": true}] 
testsuite/tests/singlecluster/authorino/identity/api_key/test_match_expression.py::test_correct_auth 
[gw2] [ 35%] PASSED testsuite/tests/singlecluster/authorino/identity/api_key/test_auth_credentials.py::test_custom_selector[customHeader] 
testsuite/tests/singlecluster/authorino/identity/api_key/test_auth_credentials.py::test_custom_header[customHeader] 
[gw2] [ 35%] PASSED testsuite/tests/singlecluster/authorino/identity/api_key/test_auth_credentials.py::test_custom_header[customHeader] 
testsuite/tests/singlecluster/authorino/identity/api_key/test_auth_credentials.py::test_query[customHeader] 
[gw2] [ 36%] PASSED testsuite/tests/singlecluster/authorino/identity/api_key/test_auth_credentials.py::test_query[customHeader] 
testsuite/tests/singlecluster/authorino/identity/api_key/test_auth_credentials.py::test_cookie[customHeader] 
[gw2] [ 36%] PASSED testsuite/tests/singlecluster/authorino/identity/api_key/test_auth_credentials.py::test_cookie[customHeader] 
testsuite/tests/singlecluster/authorino/identity/api_key/test_auth_credentials.py::test_custom_selector[queryString] 
[gw0] [ 37%] PASSED testsuite/tests/singlecluster/authorino/dinosaur/test_dinosaur.py::test_create 
testsuite/tests/singlecluster/authorino/dinosaur/test_dinosaur.py::test_metrics_federate 
[gw0] [ 37%] PASSED testsuite/tests/singlecluster/authorino/dinosaur/test_dinosaur.py::test_metrics_federate 
testsuite/tests/singlecluster/authorino/dinosaur/test_dinosaur.py::test_service_accounts 
[gw0] [ 38%] PASSED testsuite/tests/singlecluster/authorino/dinosaur/test_dinosaur.py::test_service_accounts 
testsuite/tests/singlecluster/authorino/dinosaur/test_dinosaur.py::test_instance_types 
[gw0] [ 38%] PASSED testsuite/tests/singlecluster/authorino/dinosaur/test_dinosaur.py::test_instance_types 
testsuite/tests/singlecluster/authorino/dinosaur/test_dinosaur.py::test_agent_clusters 
[gw2] [ 39%] PASSED testsuite/tests/singlecluster/authorino/identity/api_key/test_auth_credentials.py::test_custom_selector[queryString] 
testsuite/tests/singlecluster/authorino/identity/api_key/test_auth_credentials.py::test_custom_header[queryString] 
[gw2] [ 39%] PASSED testsuite/tests/singlecluster/authorino/identity/api_key/test_auth_credentials.py::test_custom_header[queryString] 
testsuite/tests/singlecluster/authorino/identity/api_key/test_auth_credentials.py::test_query[queryString] 
[gw2] [ 40%] PASSED testsuite/tests/singlecluster/authorino/identity/api_key/test_auth_credentials.py::test_query[queryString] 
testsuite/tests/singlecluster/authorino/identity/api_key/test_auth_credentials.py::test_cookie[queryString] 
[gw2] [ 40%] PASSED testsuite/tests/singlecluster/authorino/identity/api_key/test_auth_credentials.py::test_cookie[queryString] 
testsuite/tests/singlecluster/authorino/identity/api_key/test_auth_credentials.py::test_custom_selector[cookie] 
[gw3] [ 41%] PASSED testsuite/tests/singlecluster/authorino/identity/api_key/test_match_expression.py::test_correct_auth 
testsuite/tests/singlecluster/authorino/identity/api_key/test_match_expression.py::test_invalid_api_key 
[gw3] [ 41%] PASSED testsuite/tests/singlecluster/authorino/identity/api_key/test_match_expression.py::test_invalid_api_key 
testsuite/tests/singlecluster/authorino/identity/api_key/test_match_expression.py::test_no_auth 
[gw3] [ 42%] PASSED testsuite/tests/singlecluster/authorino/identity/api_key/test_match_expression.py::test_no_auth 
testsuite/tests/singlecluster/authorino/identity/api_key/test_match_expression.py::test_not_existing_api_key 
[gw3] [ 42%] PASSED testsuite/tests/singlecluster/authorino/identity/api_key/test_match_expression.py::test_not_existing_api_key 
[gw0] [ 43%] PASSED testsuite/tests/singlecluster/authorino/dinosaur/test_dinosaur.py::test_agent_clusters 
testsuite/tests/singlecluster/authorino/dinosaur/test_dinosaur.py::test_resource_is_owner 
[gw2] [ 43%] PASSED testsuite/tests/singlecluster/authorino/identity/api_key/test_auth_credentials.py::test_custom_selector[cookie] 
testsuite/tests/singlecluster/authorino/identity/api_key/test_auth_credentials.py::test_custom_header[cookie] 
[gw2] [ 44%] PASSED testsuite/tests/singlecluster/authorino/identity/api_key/test_auth_credentials.py::test_custom_header[cookie] 
testsuite/tests/singlecluster/authorino/identity/api_key/test_auth_credentials.py::test_query[cookie] 
[gw2] [ 44%] PASSED testsuite/tests/singlecluster/authorino/identity/api_key/test_auth_credentials.py::test_query[cookie] 
testsuite/tests/singlecluster/authorino/identity/api_key/test_auth_credentials.py::test_cookie[cookie] 
[gw2] [ 45%] PASSED testsuite/tests/singlecluster/authorino/identity/api_key/test_auth_credentials.py::test_cookie[cookie] 
testsuite/tests/singlecluster/authorino/identity/api_key/test_match_label.py::test_correct_auth 
testsuite/tests/singlecluster/authorino/metrics/test_deep_metrics.py::test_deep_metrics[authorization] 
[gw2] [ 45%] SKIPPED testsuite/tests/singlecluster/authorino/metrics/test_deep_metrics.py::test_deep_metrics[authorization] 
testsuite/tests/singlecluster/authorino/metrics/test_deep_metrics.py::test_deep_metrics[identity] 
[gw2] [ 46%] SKIPPED testsuite/tests/singlecluster/authorino/metrics/test_deep_metrics.py::test_deep_metrics[identity] 
testsuite/tests/singlecluster/authorino/metrics/test_deep_metrics.py::test_deep_metrics[metadata] 
[gw2] [ 46%] SKIPPED testsuite/tests/singlecluster/authorino/metrics/test_deep_metrics.py::test_deep_metrics[metadata] 
testsuite/tests/singlecluster/authorino/metrics/test_deep_metrics.py::test_deep_metrics[response] 
[gw2] [ 47%] SKIPPED testsuite/tests/singlecluster/authorino/metrics/test_deep_metrics.py::test_deep_metrics[response] 
testsuite/tests/singlecluster/authorino/identity/api_key/test_reconciliation.py::test_create_new_api_key 
[gw0] [ 47%] PASSED testsuite/tests/singlecluster/authorino/dinosaur/test_dinosaur.py::test_resource_is_owner 
testsuite/tests/singlecluster/authorino/dinosaur/test_dinosaur.py::test_resource_is_not_owner_client_denied 
[gw3] [ 48%] PASSED testsuite/tests/singlecluster/authorino/identity/api_key/test_match_label.py::test_correct_auth 
testsuite/tests/singlecluster/authorino/identity/api_key/test_match_label.py::test_no_auth 
[gw3] [ 48%] PASSED testsuite/tests/singlecluster/authorino/identity/api_key/test_match_label.py::test_no_auth 
testsuite/tests/singlecluster/authorino/identity/api_key/test_match_label.py::test_invalid_api_key 
[gw3] [ 49%] PASSED testsuite/tests/singlecluster/authorino/identity/api_key/test_match_label.py::test_invalid_api_key 
testsuite/tests/singlecluster/authorino/identity/api_key/test_match_label.py::test_invalid_api_key_secret 
[gw3] [ 49%] PASSED testsuite/tests/singlecluster/authorino/identity/api_key/test_match_label.py::test_invalid_api_key_secret 
[gw2] [ 50%] PASSED testsuite/tests/singlecluster/authorino/identity/api_key/test_reconciliation.py::test_create_new_api_key 
testsuite/tests/singlecluster/authorino/identity/api_key/test_reconciliation.py::test_delete_api_key 
[gw0] [ 50%] PASSED testsuite/tests/singlecluster/authorino/dinosaur/test_dinosaur.py::test_resource_is_not_owner_client_denied 
testsuite/tests/singlecluster/authorino/dinosaur/test_dinosaur.py::test_resource_is_not_owner_resource_denied 
testsuite/tests/singlecluster/authorino/metadata/test_token_introspect.py::test_no_token 
[gw2] [ 50%] PASSED testsuite/tests/singlecluster/authorino/identity/api_key/test_reconciliation.py::test_delete_api_key 
testsuite/tests/singlecluster/authorino/identity/api_key/test_reconciliation.py::test_update_api_key 
[gw3] [ 51%] PASSED testsuite/tests/singlecluster/authorino/metadata/test_token_introspect.py::test_no_token 
testsuite/tests/singlecluster/authorino/metadata/test_token_introspect.py::test_access_token 
[gw3] [ 51%] PASSED testsuite/tests/singlecluster/authorino/metadata/test_token_introspect.py::test_access_token 
testsuite/tests/singlecluster/authorino/metadata/test_token_introspect.py::test_revoked_token 
[gw2] [ 52%] PASSED testsuite/tests/singlecluster/authorino/identity/api_key/test_reconciliation.py::test_update_api_key 
[gw3] [ 52%] PASSED testsuite/tests/singlecluster/authorino/metadata/test_token_introspect.py::test_revoked_token 
[gw0] [ 53%] PASSED testsuite/tests/singlecluster/authorino/dinosaur/test_dinosaur.py::test_resource_is_not_owner_resource_denied 
testsuite/tests/singlecluster/authorino/operator/tls/test_tls.py::test_valid_certificate 
testsuite/tests/singlecluster/authorino/operator/tls/mtls/test_mtls_trust_chain.py::test_mtls_trust_chain_success 
testsuite/tests/singlecluster/authorino/identity/auth/test_multiple_auth_identities.py::test_correct_auth 
[gw0] [ 53%] PASSED testsuite/tests/singlecluster/authorino/identity/auth/test_multiple_auth_identities.py::test_correct_auth 
testsuite/tests/singlecluster/authorino/identity/auth/test_multiple_auth_identities.py::test_no_auth 
[gw0] [ 54%] PASSED testsuite/tests/singlecluster/authorino/identity/auth/test_multiple_auth_identities.py::test_no_auth 
testsuite/tests/singlecluster/authorino/identity/auth/test_multiple_auth_identities.py::test_wrong_auth 
[gw0] [ 54%] PASSED testsuite/tests/singlecluster/authorino/identity/auth/test_multiple_auth_identities.py::test_wrong_auth 
testsuite/tests/singlecluster/authorino/response/test_headers.py::test_headers[123456789] 
[gw0] [ 55%] PASSED testsuite/tests/singlecluster/authorino/response/test_headers.py::test_headers[123456789] 
testsuite/tests/singlecluster/authorino/response/test_headers.py::test_headers[standardCharacters] 
[gw0] [ 55%] PASSED testsuite/tests/singlecluster/authorino/response/test_headers.py::test_headers[standardCharacters] 
testsuite/tests/singlecluster/authorino/response/test_headers.py::test_headers[specialcharacters+*-.] 
[gw0] [ 56%] PASSED testsuite/tests/singlecluster/authorino/response/test_headers.py::test_headers[specialcharacters+*-.] 
testsuite/tests/singlecluster/authorino/wristband/test_wristband.py::test_wristband_token_claims 
[gw0] [ 56%] PASSED testsuite/tests/singlecluster/authorino/wristband/test_wristband.py::test_wristband_token_claims 
testsuite/tests/singlecluster/authorino/wristband/test_wristband.py::test_wristband_success 
[gw0] [ 57%] PASSED testsuite/tests/singlecluster/authorino/wristband/test_wristband.py::test_wristband_success 
testsuite/tests/singlecluster/authorino/wristband/test_wristband.py::test_wristband_fail 
[gw0] [ 57%] PASSED testsuite/tests/singlecluster/authorino/wristband/test_wristband.py::test_wristband_fail 
testsuite/tests/singlecluster/authorino/authorization/opa/external_registry/test_cache.py::test_caching 
[gw0] [ 58%] PASSED testsuite/tests/singlecluster/authorino/authorization/opa/external_registry/test_cache.py::test_caching 
testsuite/tests/singlecluster/authorino/authorization/opa/external_registry/test_cache.py::test_cache_refresh 
[gw0] [ 58%] PASSED testsuite/tests/singlecluster/authorino/authorization/opa/external_registry/test_cache.py::test_cache_refresh 
testsuite/tests/singlecluster/authorino/authorization/opa/external_registry/test_external_registry.py::test_allowed_by_opa 
[gw0] [ 59%] PASSED testsuite/tests/singlecluster/authorino/authorization/opa/external_registry/test_external_registry.py::test_allowed_by_opa 
testsuite/tests/singlecluster/authorino/authorization/opa/external_registry/test_external_registry.py::test_denied_by_opa 
[gw0] [ 59%] PASSED testsuite/tests/singlecluster/authorino/authorization/opa/external_registry/test_external_registry.py::test_denied_by_opa 
testsuite/tests/singlecluster/authorino/authorization/opa/test_authorization_services.py::test_user_managed_access 
[gw1] [ 60%] PASSED testsuite/tests/singlecluster/authorino/operator/tls/mtls/test_mtls_identity.py::test_mtls_success 
testsuite/tests/singlecluster/authorino/operator/tls/mtls/test_mtls_identity.py::test_mtls_fail[Self-signed cert] 
[gw1] [ 60%] PASSED testsuite/tests/singlecluster/authorino/operator/tls/mtls/test_mtls_identity.py::test_mtls_fail[Self-signed cert] 
testsuite/tests/singlecluster/authorino/operator/tls/mtls/test_mtls_identity.py::test_mtls_fail[Invalid certificate] 
[gw1] [ 61%] PASSED testsuite/tests/singlecluster/authorino/operator/tls/mtls/test_mtls_identity.py::test_mtls_fail[Invalid certificate] 
testsuite/tests/singlecluster/authorino/operator/tls/mtls/test_mtls_identity.py::test_mtls_fail[Without certificate] 
[gw1] [ 61%] PASSED testsuite/tests/singlecluster/authorino/operator/tls/mtls/test_mtls_identity.py::test_mtls_fail[Without certificate] 
testsuite/tests/singlecluster/authorino/operator/tls/mtls/test_mtls_identity.py::test_mtls_fail[Unknown authority] 
[gw1] [ 62%] PASSED testsuite/tests/singlecluster/authorino/operator/tls/mtls/test_mtls_identity.py::test_mtls_fail[Unknown authority] 
testsuite/tests/singlecluster/authorino/operator/tls/mtls/test_mtls_identity.py::test_mtls_unmatched_attributes 
[gw1] [ 62%] PASSED testsuite/tests/singlecluster/authorino/operator/tls/mtls/test_mtls_identity.py::test_mtls_unmatched_attributes 
testsuite/tests/singlecluster/authorino/identity/auth/test_auth_identity.py::test_auth_identity[keycloak] 
[gw3] [ 63%] PASSED testsuite/tests/singlecluster/authorino/operator/tls/test_tls.py::test_valid_certificate 
testsuite/tests/singlecluster/authorino/operator/tls/test_tls.py::test_no_certificate 
[gw0] [ 63%] PASSED testsuite/tests/singlecluster/authorino/authorization/opa/test_authorization_services.py::test_user_managed_access 
testsuite/tests/singlecluster/authorino/authorization/opa/test_authorization_services.py::test_access_non_existent_resource 
[gw3] [ 64%] PASSED testsuite/tests/singlecluster/authorino/operator/tls/test_tls.py::test_no_certificate 
testsuite/tests/singlecluster/authorino/operator/tls/test_tls.py::test_invalid_certificate 
[gw3] [ 64%] PASSED testsuite/tests/singlecluster/authorino/operator/tls/test_tls.py::test_invalid_certificate 
[gw0] [ 65%] PASSED testsuite/tests/singlecluster/authorino/authorization/opa/test_authorization_services.py::test_access_non_existent_resource 
testsuite/tests/singlecluster/authorino/authorization/opa/test_inline_rego.py::test_authorized_by_opa 
[gw1] [ 65%] PASSED testsuite/tests/singlecluster/authorino/identity/auth/test_auth_identity.py::test_auth_identity[keycloak] 
testsuite/tests/singlecluster/authorino/identity/auth/test_auth_identity.py::test_auth_identity[auth0] 
[gw0] [ 66%] PASSED testsuite/tests/singlecluster/authorino/authorization/opa/test_inline_rego.py::test_authorized_by_opa 
testsuite/tests/singlecluster/authorino/authorization/opa/test_inline_rego.py::test_rejected_by_opa 
testsuite/tests/singlecluster/authorino/identity/keycloak/test_keycloak_roles.py::test_user_with_role 
[gw0] [ 66%] PASSED testsuite/tests/singlecluster/authorino/authorization/opa/test_inline_rego.py::test_rejected_by_opa 
testsuite/tests/singlecluster/authorino/identity/plain/test_plain_identity.py::test_plain_identity[non-existent AuthJson path to retrieve identity from] 
[gw1] [ 66%] PASSED testsuite/tests/singlecluster/authorino/identity/auth/test_auth_identity.py::test_auth_identity[auth0] 
testsuite/tests/singlecluster/authorino/identity/extended_properties/test_extended_properties.py::test_basic 
[gw0] [ 67%] PASSED testsuite/tests/singlecluster/authorino/identity/plain/test_plain_identity.py::test_plain_identity[non-existent AuthJson path to retrieve identity from] 
testsuite/tests/singlecluster/authorino/identity/plain/test_plain_identity.py::test_plain_identity[existing AuthJson path to retrieve identity from] 
[gw3] [ 67%] PASSED testsuite/tests/singlecluster/authorino/identity/keycloak/test_keycloak_roles.py::test_user_with_role 
testsuite/tests/singlecluster/authorino/identity/keycloak/test_keycloak_roles.py::test_user_without_role 
[gw3] [ 68%] PASSED testsuite/tests/singlecluster/authorino/identity/keycloak/test_keycloak_roles.py::test_user_without_role 
testsuite/tests/singlecluster/authorino/metadata/test_http.py::test_authorized 
[gw1] [ 68%] PASSED testsuite/tests/singlecluster/authorino/identity/extended_properties/test_extended_properties.py::test_basic 
testsuite/tests/singlecluster/authorino/identity/extended_properties/test_extended_properties.py::test_chain 
[gw1] [ 69%] PASSED testsuite/tests/singlecluster/authorino/identity/extended_properties/test_extended_properties.py::test_chain 
[gw0] [ 69%] PASSED testsuite/tests/singlecluster/authorino/identity/plain/test_plain_identity.py::test_plain_identity[existing AuthJson path to retrieve identity from] 
testsuite/tests/singlecluster/authorino/metadata/test_user_info.py::test_correct_auth 
testsuite/tests/singlecluster/authorino/metadata/test_uma.py::test_uma_resource_authorized 
[gw3] [ 70%] PASSED testsuite/tests/singlecluster/authorino/metadata/test_http.py::test_authorized 
testsuite/tests/singlecluster/authorino/metadata/test_http.py::test_decline_authorization 
[gw3] [ 70%] PASSED testsuite/tests/singlecluster/authorino/metadata/test_http.py::test_decline_authorization 
[gw1] [ 71%] PASSED testsuite/tests/singlecluster/authorino/metadata/test_user_info.py::test_correct_auth 
testsuite/tests/singlecluster/authorino/metadata/test_user_info.py::test_incorrect_auth 
[gw1] [ 71%] PASSED testsuite/tests/singlecluster/authorino/metadata/test_user_info.py::test_incorrect_auth 
testsuite/tests/singlecluster/authorino/multiple_hosts/test_multiple_hosts.py::test_original_host 
testsuite/tests/singlecluster/authorino/operator/clusterwide/test_all_namespace_api_key.py::test_correct_auth 
[gw2] [ 72%] PASSED testsuite/tests/singlecluster/authorino/operator/tls/mtls/test_mtls_trust_chain.py::test_mtls_trust_chain_success 
testsuite/tests/singlecluster/authorino/operator/tls/mtls/test_mtls_trust_chain.py::test_mtls_trust_chain_fail 
[gw2] [ 72%] PASSED testsuite/tests/singlecluster/authorino/operator/tls/mtls/test_mtls_trust_chain.py::test_mtls_trust_chain_fail 
testsuite/tests/singlecluster/authorino/operator/tls/mtls/test_mtls_trust_chain.py::test_mtls_trust_chain_rejected_cert 
[gw2] [ 73%] PASSED testsuite/tests/singlecluster/authorino/operator/tls/mtls/test_mtls_trust_chain.py::test_mtls_trust_chain_rejected_cert 
[gw3] [ 73%] PASSED testsuite/tests/singlecluster/authorino/multiple_hosts/test_multiple_hosts.py::test_original_host 
testsuite/tests/singlecluster/authorino/multiple_hosts/test_multiple_hosts.py::test_second_host 
[gw3] [ 74%] PASSED testsuite/tests/singlecluster/authorino/multiple_hosts/test_multiple_hosts.py::test_second_host 
[gw0] [ 74%] PASSED testsuite/tests/singlecluster/authorino/metadata/test_uma.py::test_uma_resource_authorized 
testsuite/tests/singlecluster/authorino/metadata/test_uma.py::test_uma_resource_forbidden 
testsuite/tests/singlecluster/authorino/operator/clusterwide/test_wildcard_collision.py::test_wildcard_first_authorization 
[gw0] [ 75%] PASSED testsuite/tests/singlecluster/authorino/metadata/test_uma.py::test_uma_resource_forbidden 
testsuite/tests/singlecluster/authorino/operator/clusterwide/test_clusterwide.py::test_auth[First namespace] 
testsuite/tests/singlecluster/authorino/operator/tls/mtls/test_mtls_attributes.py::test_mtls_multiple_attributes_success 
[gw1] [ 75%] PASSED testsuite/tests/singlecluster/authorino/operator/clusterwide/test_all_namespace_api_key.py::test_correct_auth 
testsuite/tests/singlecluster/authorino/operator/clusterwide/test_all_namespace_api_key.py::test_invalid_api_key 
[gw1] [ 76%] PASSED testsuite/tests/singlecluster/authorino/operator/clusterwide/test_all_namespace_api_key.py::test_invalid_api_key 
testsuite/tests/singlecluster/authorino/operator/raw_http/test_raw_http.py::test_authorized_via_http 
[gw3] [ 76%] PASSED testsuite/tests/singlecluster/authorino/operator/clusterwide/test_wildcard_collision.py::test_wildcard_first_authorization 
testsuite/tests/singlecluster/authorino/operator/clusterwide/test_wildcard_collision.py::test_wildcard_second_authorization 
[gw3] [ 77%] PASSED testsuite/tests/singlecluster/authorino/operator/clusterwide/test_wildcard_collision.py::test_wildcard_second_authorization 
testsuite/tests/singlecluster/authorino/operator/tls/test_webhook.py::test_authorized_via_http 
[gw0] [ 77%] PASSED testsuite/tests/singlecluster/authorino/operator/clusterwide/test_clusterwide.py::test_auth[First namespace] 
testsuite/tests/singlecluster/authorino/operator/clusterwide/test_clusterwide.py::test_auth[Second namespace] 
[gw0] [ 78%] PASSED testsuite/tests/singlecluster/authorino/operator/clusterwide/test_clusterwide.py::test_auth[Second namespace] 
testsuite/tests/singlecluster/authorino/response/test_auth_json.py::test_auth_json_path[path_and_value0] 
[gw0] [ 78%] PASSED testsuite/tests/singlecluster/authorino/response/test_auth_json.py::test_auth_json_path[path_and_value0] 
testsuite/tests/singlecluster/authorino/response/test_auth_json.py::test_auth_json_path[path_and_value1] 
[gw0] [ 79%] PASSED testsuite/tests/singlecluster/authorino/response/test_auth_json.py::test_auth_json_path[path_and_value1] 
testsuite/tests/singlecluster/authorino/caching/metadata/test_ttl.py::test_cached_ttl 
[gw0] [ 79%] PASSED testsuite/tests/singlecluster/authorino/caching/metadata/test_ttl.py::test_cached_ttl 
testsuite/tests/singlecluster/authorino/conditions/section_conditions/test_authorization_condition.py::test_skip_authorization 
[gw0] [ 80%] PASSED testsuite/tests/singlecluster/authorino/conditions/section_conditions/test_authorization_condition.py::test_skip_authorization 
testsuite/tests/singlecluster/authorino/conditions/section_conditions/test_identity_condition.py::test_skip_identity 
[gw0] [ 80%] PASSED testsuite/tests/singlecluster/authorino/conditions/section_conditions/test_identity_condition.py::test_skip_identity 
testsuite/tests/singlecluster/authorino/conditions/section_conditions/test_metadata_condition.py::test_skip_metadata 
[gw1] [ 81%] SKIPPED testsuite/tests/singlecluster/authorino/operator/raw_http/test_raw_http.py::test_authorized_via_http 
testsuite/tests/singlecluster/authorino/operator/raw_http/test_raw_http.py::test_unauthorized_via_http 
[gw1] [ 81%] SKIPPED testsuite/tests/singlecluster/authorino/operator/raw_http/test_raw_http.py::test_unauthorized_via_http 
[gw0] [ 82%] PASSED testsuite/tests/singlecluster/authorino/conditions/section_conditions/test_metadata_condition.py::test_skip_metadata 
testsuite/tests/singlecluster/authorino/conditions/section_conditions/test_response_condition.py::test_skip_response 
testsuite/tests/singlecluster/authorino/caching/metadata/test_caching.py::test_cached 
[gw0] [ 82%] PASSED testsuite/tests/singlecluster/authorino/conditions/section_conditions/test_response_condition.py::test_skip_response 
testsuite/tests/singlecluster/authorino/conditions/test_top_level_condition.py::test_skip_auth_config 
[gw1] [ 83%] PASSED testsuite/tests/singlecluster/authorino/caching/metadata/test_caching.py::test_cached 
testsuite/tests/singlecluster/authorino/identity/anonymous/test_anonymous_context.py::test_anonymous_context 
[gw0] [ 83%] PASSED testsuite/tests/singlecluster/authorino/conditions/test_top_level_condition.py::test_skip_auth_config 
testsuite/tests/singlecluster/authorino/identity/anonymous/test_anonymous_identity.py::test_anonymous_identity 
[gw1] [ 83%] PASSED testsuite/tests/singlecluster/authorino/identity/anonymous/test_anonymous_context.py::test_anonymous_context 
testsuite/tests/singlecluster/authorino/identity/api_key/test_api_key_context.py::tests_api_key_context 
[gw1] [ 84%] PASSED testsuite/tests/singlecluster/authorino/identity/api_key/test_api_key_context.py::tests_api_key_context 
testsuite/tests/singlecluster/authorino/identity/extended_properties/test_token_normalization.py::test_token_normalization 
[gw0] [ 84%] PASSED testsuite/tests/singlecluster/authorino/identity/anonymous/test_anonymous_identity.py::test_anonymous_identity 
testsuite/tests/singlecluster/authorino/identity/extended_properties/test_overwriting.py::test_overwrite 
[gw2] [ 85%] PASSED testsuite/tests/singlecluster/authorino/operator/tls/mtls/test_mtls_attributes.py::test_mtls_multiple_attributes_success 
testsuite/tests/singlecluster/authorino/operator/tls/mtls/test_mtls_attributes.py::test_mtls_multiple_attributes_fail 
[gw2] [ 85%] PASSED testsuite/tests/singlecluster/authorino/operator/tls/mtls/test_mtls_attributes.py::test_mtls_multiple_attributes_fail 
[gw1] [ 86%] PASSED testsuite/tests/singlecluster/authorino/identity/extended_properties/test_token_normalization.py::test_token_normalization 
[gw0] [ 86%] PASSED testsuite/tests/singlecluster/authorino/identity/extended_properties/test_overwriting.py::test_overwrite 
testsuite/tests/singlecluster/authorino/identity/keycloak/test_jwt_ttl.py::test_jwt_ttl 
testsuite/tests/singlecluster/authorino/identity/keycloak/test_keycloak_context.py::test_keycloak_context 
testsuite/tests/singlecluster/authorino/response/test_deny_with.py::test_unauthenticated 
[gw1] [ 87%] PASSED testsuite/tests/singlecluster/authorino/identity/keycloak/test_keycloak_context.py::test_keycloak_context 
testsuite/tests/singlecluster/authorino/identity/plain/test_jwt_plain_identity.py::test_jwt_plain_identity 
[gw2] [ 87%] PASSED testsuite/tests/singlecluster/authorino/response/test_deny_with.py::test_unauthenticated 
testsuite/tests/singlecluster/authorino/response/test_deny_with.py::test_unauthorized 
[gw2] [ 88%] PASSED testsuite/tests/singlecluster/authorino/response/test_deny_with.py::test_unauthorized 
testsuite/tests/singlecluster/authorino/identity/token_review/test_audiences.py::test_custom_audience 
[gw2] [ 88%] PASSED testsuite/tests/singlecluster/authorino/identity/token_review/test_audiences.py::test_custom_audience 
testsuite/tests/singlecluster/authorino/metadata/test_multi_element_json.py::test_metadata_contents 
[gw2] [ 89%] PASSED testsuite/tests/singlecluster/authorino/metadata/test_multi_element_json.py::test_metadata_contents 
testsuite/tests/singlecluster/authorino/multiple_hosts/test_remove_host.py::test_removing_host 
[gw2] [ 89%] PASSED testsuite/tests/singlecluster/authorino/multiple_hosts/test_remove_host.py::test_removing_host 
testsuite/tests/singlecluster/authorino/operator/sharding/test_preexisting_auth.py::test_preexisting_auth 
[gw3] [ 90%] PASSED testsuite/tests/singlecluster/authorino/operator/tls/test_webhook.py::test_authorized_via_http 
testsuite/tests/singlecluster/authorino/operator/tls/test_webhook.py::test_unauthorized_via_http 
[gw3] [ 90%] PASSED testsuite/tests/singlecluster/authorino/operator/tls/test_webhook.py::test_unauthorized_via_http 
testsuite/tests/singlecluster/authorino/caching/metadata/test_not_cached.py::test_no_caching 
[gw3] [ 91%] PASSED testsuite/tests/singlecluster/authorino/caching/metadata/test_not_cached.py::test_no_caching 
testsuite/tests/singlecluster/authorino/operator/test_wildcard.py::test_wildcard 
[gw1] [ 91%] PASSED testsuite/tests/singlecluster/authorino/identity/plain/test_jwt_plain_identity.py::test_jwt_plain_identity 
testsuite/tests/singlecluster/authorino/identity/subject_access_review/test_resource_attributes.py::test_subject_access_review_resource_attributes 
[gw1] [ 92%] PASSED testsuite/tests/singlecluster/authorino/identity/subject_access_review/test_resource_attributes.py::test_subject_access_review_resource_attributes 
testsuite/tests/singlecluster/authorino/identity/token_review/test_host.py::test_host_audience 
[gw1] [ 92%] PASSED testsuite/tests/singlecluster/authorino/identity/token_review/test_host.py::test_host_audience 
testsuite/tests/singlecluster/authorino/priority/test_dependency.py::test_dependency 
[gw1] [ 93%] PASSED testsuite/tests/singlecluster/authorino/priority/test_dependency.py::test_dependency 
testsuite/tests/singlecluster/authorino/priority/test_sequence_anonymous.py::test_priority_anonymous 
[gw1] [ 93%] PASSED testsuite/tests/singlecluster/authorino/priority/test_sequence_anonymous.py::test_priority_anonymous 
testsuite/tests/singlecluster/authorino/priority/test_sequence_api_key.py::test_priority_api_key 
[gw1] [ 94%] PASSED testsuite/tests/singlecluster/authorino/priority/test_sequence_api_key.py::test_priority_api_key 
testsuite/tests/singlecluster/authorino/response/test_multiple_responses.py::test_multiple_responses 
[gw1] [ 94%] PASSED testsuite/tests/singlecluster/authorino/response/test_multiple_responses.py::test_multiple_responses 
testsuite/tests/singlecluster/authorino/response/test_redirect.py::test_redirect 
[gw1] [ 95%] PASSED testsuite/tests/singlecluster/authorino/response/test_redirect.py::test_redirect 
testsuite/tests/singlecluster/authorino/response/test_simple_response.py::test_simple_response_with 
[gw1] [ 95%] PASSED testsuite/tests/singlecluster/authorino/response/test_simple_response.py::test_simple_response_with 
[gw0] [ 96%] PASSED testsuite/tests/singlecluster/authorino/identity/keycloak/test_jwt_ttl.py::test_jwt_ttl 
testsuite/tests/singlecluster/authorino/identity/plain/test_jwt_user_story.py::test_jwt_user_story 
[gw3] [ 96%] PASSED testsuite/tests/singlecluster/authorino/operator/test_wildcard.py::test_wildcard 
testsuite/tests/singlecluster/authorino/operator/tracing/test_tracing.py::test_tracing 
[gw2] [ 97%] PASSED testsuite/tests/singlecluster/authorino/operator/sharding/test_preexisting_auth.py::test_preexisting_auth 
testsuite/tests/singlecluster/authorino/operator/sharding/test_sharding.py::test_sharding 
[gw0] [ 97%] FAILED testsuite/tests/singlecluster/authorino/identity/plain/test_jwt_user_story.py::test_jwt_user_story 
testsuite/tests/singlecluster/authorino/identity/subject_access_review/test_non_resource_attributes.py::test_subject_access_review_non_resource_attributes 
[gw0] [ 98%] PASSED testsuite/tests/singlecluster/authorino/identity/subject_access_review/test_non_resource_attributes.py::test_subject_access_review_non_resource_attributes 
[gw3] [ 98%] FAILED testsuite/tests/singlecluster/authorino/operator/tracing/test_tracing.py::test_tracing 
testsuite/tests/singlecluster/authorino/operator/tracing/test_tracing_tags.py::test_tracing_tags 
[gw2] [ 99%] PASSED testsuite/tests/singlecluster/authorino/operator/sharding/test_sharding.py::test_sharding 
testsuite/tests/singlecluster/authorino/operator/test_deploy_authorino.py::test_authorino_ready 
[gw2] [ 99%] PASSED testsuite/tests/singlecluster/authorino/operator/test_deploy_authorino.py::test_authorino_ready 
[gw3] [100%] FAILED testsuite/tests/singlecluster/authorino/operator/tracing/test_tracing_tags.py::test_tracing_tags 

=================================== FAILURES ===================================
_____________________________ test_jwt_user_story ______________________________
[gw0] linux -- Python 3.12.3 /home/runner/.cache/pypoetry/virtualenvs/kuadrant-testsuite-6EPd9kcO-py3.12/bin/python
testsuite/tests/singlecluster/authorino/identity/plain/test_jwt_user_story.py:116: in test_jwt_user_story
    assert response.status_code == 200
E   assert 500 == 200
E    +  where 500 = <testsuite.httpx.Result object at 0x7fbb8338f890>.status_code
------------------------------ Captured log setup ------------------------------
15:29:07 +0000 INFO:httpx:HTTP Request: PUT http://172.18.255.204:1080/mockserver/expectation "HTTP/1.1 201 Created"
------------------------------ Captured log call -------------------------------
15:29:08 +0000 INFO:httpx:HTTP Request: GET http://172.18.255.206:8080/get "HTTP/1.1 500 Internal Server Error"
---------------------------- Captured log teardown -----------------------------
15:29:09 +0000 INFO:httpx:HTTP Request: PUT http://172.18.255.204:1080/mockserver/clear "HTTP/1.1 200 OK"
_________________________________ test_tracing _________________________________
[gw3] linux -- Python 3.12.3 /home/runner/.cache/pypoetry/virtualenvs/kuadrant-testsuite-6EPd9kcO-py3.12/bin/python
testsuite/tests/singlecluster/authorino/operator/tracing/test_tracing.py:19: in test_tracing
    assert len(trace) == 1
E   assert 0 == 1
E    +  where 0 = len([])
------------------------------ Captured log call -------------------------------
15:28:54 +0000 INFO:httpx:HTTP Request: GET http://172.18.255.211:8080/get "HTTP/1.1 200 OK"
15:28:54 +0000 INFO:httpx:HTTP Request: GET http://172.18.255.201/api/traces?service=authorino&tags=%7B%22authorino.request_id%22%3A%2299f84cf8-cfc1-45ee-a61e-47a3d8b32fad%22%7D "HTTP/1.1 200 OK"
15:28:54 +0000 INFO:backoff:Backing off search(...) for 1.0s ([])
15:28:55 +0000 INFO:httpx:HTTP Request: GET http://172.18.255.201/api/traces?service=authorino&tags=%7B%22authorino.request_id%22%3A%2299f84cf8-cfc1-45ee-a61e-47a3d8b32fad%22%7D "HTTP/1.1 200 OK"
15:28:55 +0000 INFO:backoff:Backing off search(...) for 1.0s ([])
15:28:56 +0000 INFO:httpx:HTTP Request: GET http://172.18.255.201/api/traces?service=authorino&tags=%7B%22authorino.request_id%22%3A%2299f84cf8-cfc1-45ee-a61e-47a3d8b32fad%22%7D "HTTP/1.1 200 OK"
15:28:56 +0000 INFO:backoff:Backing off search(...) for 2.0s ([])
15:28:58 +0000 INFO:httpx:HTTP Request: GET http://172.18.255.201/api/traces?service=authorino&tags=%7B%22authorino.request_id%22%3A%2299f84cf8-cfc1-45ee-a61e-47a3d8b32fad%22%7D "HTTP/1.1 200 OK"
15:28:58 +0000 INFO:backoff:Backing off search(...) for 3.0s ([])
15:29:01 +0000 INFO:httpx:HTTP Request: GET http://172.18.255.201/api/traces?service=authorino&tags=%7B%22authorino.request_id%22%3A%2299f84cf8-cfc1-45ee-a61e-47a3d8b32fad%22%7D "HTTP/1.1 200 OK"
15:29:01 +0000 INFO:backoff:Backing off search(...) for 5.0s ([])
15:29:06 +0000 INFO:httpx:HTTP Request: GET http://172.18.255.201/api/traces?service=authorino&tags=%7B%22authorino.request_id%22%3A%2299f84cf8-cfc1-45ee-a61e-47a3d8b32fad%22%7D "HTTP/1.1 200 OK"
15:29:06 +0000 INFO:backoff:Backing off search(...) for 8.0s ([])
15:29:14 +0000 INFO:httpx:HTTP Request: GET http://172.18.255.201/api/traces?service=authorino&tags=%7B%22authorino.request_id%22%3A%2299f84cf8-cfc1-45ee-a61e-47a3d8b32fad%22%7D "HTTP/1.1 200 OK"
15:29:14 +0000 ERROR:backoff:Giving up search(...) after 7 tries ([])
______________________________ test_tracing_tags _______________________________
[gw3] linux -- Python 3.12.3 /home/runner/.cache/pypoetry/virtualenvs/kuadrant-testsuite-6EPd9kcO-py3.12/bin/python
testsuite/tests/singlecluster/authorino/operator/tracing/test_tracing_tags.py:30: in test_tracing_tags
    assert len(trace) == 1
E   assert 0 == 1
E    +  where 0 = len([])
------------------------------ Captured log call -------------------------------
15:29:35 +0000 INFO:httpx:HTTP Request: GET http://172.18.255.208:8080/get "HTTP/1.1 200 OK"
15:29:35 +0000 INFO:httpx:HTTP Request: GET http://172.18.255.201/api/traces?test-key=test-value&service=authorino&tags=%7B%22authorino.request_id%22%3A%22b2236ca9-b365-4063-97eb-192944ef2646%22%7D "HTTP/1.1 200 OK"
15:29:35 +0000 INFO:backoff:Backing off search(...) for 1.0s ([])
15:29:36 +0000 INFO:httpx:HTTP Request: GET http://172.18.255.201/api/traces?test-key=test-value&service=authorino&tags=%7B%22authorino.request_id%22%3A%22b2236ca9-b365-4063-97eb-192944ef2646%22%7D "HTTP/1.1 200 OK"
15:29:36 +0000 INFO:backoff:Backing off search(...) for 1.0s ([])
15:29:37 +0000 INFO:httpx:HTTP Request: GET http://172.18.255.201/api/traces?test-key=test-value&service=authorino&tags=%7B%22authorino.request_id%22%3A%22b2236ca9-b365-4063-97eb-192944ef2646%22%7D "HTTP/1.1 200 OK"
15:29:37 +0000 INFO:backoff:Backing off search(...) for 2.0s ([])
15:29:39 +0000 INFO:httpx:HTTP Request: GET http://172.18.255.201/api/traces?test-key=test-value&service=authorino&tags=%7B%22authorino.request_id%22%3A%22b2236ca9-b365-4063-97eb-192944ef2646%22%7D "HTTP/1.1 200 OK"
15:29:39 +0000 INFO:backoff:Backing off search(...) for 3.0s ([])
15:29:42 +0000 INFO:httpx:HTTP Request: GET http://172.18.255.201/api/traces?test-key=test-value&service=authorino&tags=%7B%22authorino.request_id%22%3A%22b2236ca9-b365-4063-97eb-192944ef2646%22%7D "HTTP/1.1 200 OK"
15:29:42 +0000 INFO:backoff:Backing off search(...) for 5.0s ([])
15:29:47 +0000 INFO:httpx:HTTP Request: GET http://172.18.255.201/api/traces?test-key=test-value&service=authorino&tags=%7B%22authorino.request_id%22%3A%22b2236ca9-b365-4063-97eb-192944ef2646%22%7D "HTTP/1.1 200 OK"
15:29:47 +0000 INFO:backoff:Backing off search(...) for 8.0s ([])
15:29:55 +0000 INFO:httpx:HTTP Request: GET http://172.18.255.201/api/traces?test-key=test-value&service=authorino&tags=%7B%22authorino.request_id%22%3A%22b2236ca9-b365-4063-97eb-192944ef2646%22%7D "HTTP/1.1 200 OK"
15:29:55 +0000 ERROR:backoff:Giving up search(...) after 7 tries ([])
=============================== warnings summary ===============================
testsuite/tests/singlecluster/authorino/operator/tls/mtls/test_mtls_identity.py: 5 warnings
testsuite/tests/singlecluster/authorino/operator/tls/test_tls.py: 2 warnings
testsuite/tests/singlecluster/authorino/operator/tls/mtls/test_mtls_trust_chain.py: 3 warnings
testsuite/tests/singlecluster/authorino/operator/tls/mtls/test_mtls_attributes.py: 2 warnings
  /home/runner/.cache/pypoetry/virtualenvs/kuadrant-testsuite-6EPd9kcO-py3.12/lib/python3.12/site-packages/httpx/_config.py:63: DeprecationWarning: `cert=...` is deprecated. Use `verify=<ssl_context>` instead,with `.load_cert_chain()` to configure the certificate chain.
    warnings.warn(message, DeprecationWarning)

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
FAILED testsuite/tests/singlecluster/authorino/identity/plain/test_jwt_user_story.py::test_jwt_user_story - assert 500 == 200
 +  where 500 = <testsuite.httpx.Result object at 0x7fbb8338f890>.status_code
FAILED testsuite/tests/singlecluster/authorino/operator/tracing/test_tracing.py::test_tracing - assert 0 == 1
 +  where 0 = len([])
FAILED testsuite/tests/singlecluster/authorino/operator/tracing/test_tracing_tags.py::test_tracing_tags - assert 0 == 1
 +  where 0 = len([])
====== 3 failed, 176 passed, 27 skipped, 12 warnings in 281.67s (0:04:41) ======
make: *** [Makefile:50: authorino-standalone] Error 1

@averevki
Copy link
Copy Markdown
Contributor Author

^example of final output after sending comment with /make authorino-standalone

@averevki averevki self-assigned this Aug 26, 2025
@averevki averevki added the enhancement Improvement to existing test label Aug 26, 2025
trepel
trepel previously approved these changes Aug 27, 2025
Copy link
Copy Markdown
Contributor

@trepel trepel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not very familiar with GitHub Actions but this looks great. I apparently works and I did not spot anything that would block merging.
LGTM

Do you plan to introduce this for kuadrant-operator repository as well?

I could not find timeout being set in any of our GitHub actions so we apparently are not using this but it might be something to consider:

  jobs:
    kuadrant-test:
      timeout-minutes: 120  # Add reasonable timeout

Did you intentionally use hard-coded versions? For instance golang version is hard-coded to v1.24 etc

Comment thread .github/workflows/test-pr.yml
Comment thread .github/workflows/test-pr.yml
Signed-off-by: averevki <sandyverevkin@gmail.com>
@averevki
Copy link
Copy Markdown
Contributor Author

Do you plan to introduce this for kuadrant-operator repository as well?

@trepel Yes, this is one of the steps for this "epic" #671

I could not find timeout being set in any of our GitHub actions so we apparently are not using this but it might be something to consider:

Good one, pushed the change. 120 seems like a reasonable timeout

Did you intentionally use hard-coded versions? For instance golang version is hard-coded to v1.24 etc

I can't remember everything but most of the kuadrant-operator dependencies versions here are taken from the official kuadrant helm. kind has dependency on go version if I remember correctly. There are no officially supported k8s dependencies versions we're testing with afaik, so everything assembled to just work with testsuite for now

P.S. This action could also be used as regular testing env for latest kuadrant helm, know I'm thinking about it

Copy link
Copy Markdown
Contributor

@trepel trepel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@averevki Great! Thanks for reply. LGTM

@averevki averevki merged commit 009cedc into Kuadrant:main Aug 27, 2025
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement Improvement to existing test

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Run testsuite on PRs Allow running testsuite image on PRs in components

3 participants