Add gh_report.py: stats and logs for integration tests by denik · Pull Request #3115 · databricks/cli

denik · 2025-06-25T09:30:18Z

Changes

New script: gh_parse.py that parses artifacts output by integration tests and prints a report: Add gh_report.py: stats and logs for integration tests #3115 (comment) Can also print output for failed tests (--output) and filter by test name or env name.
New script: gh_report.py that downloads integration tests artifacts for current PR/branch or specified commit and calls gh_parse.py to print a report.

Why

Faster debugging compared to clicking via Github actions UI.

Tests

Manually tested.

Getting stats:

~/work/cli-main % ./tools/gh_report.py --commit 1a96017c3816cc6a2c24b7749f95c142753d7030

Getting output:

~/work/cli-main % ./tools/gh_report.py --commit 1a96017c3816cc6a2c24b7749f95c142753d7030 --filter TestAbortBind --output

eng-dev-ecosystem-bot · 2025-06-26T15:27:54Z

Run: 15912439123

	Env	✅‌pass	❌‌FAIL	💥‌PANIC	🔄‌flaky	🙈‌skip	🤯‌MISS
❌‌	aws linux	286	10		2	365
❌‌	aws windows	289	10			364
❌‌	aws-ucws linux	382	18			265
❌‌	aws-ucws windows	383	18			264
❌‌	azure linux	288	10			364
❌‌	azure windows	289	10			363
❌‌	azure-ucws linux	382	20			262
❌‌	azure-ucws windows	374	21	1		261	5
❌‌	gcp linux	236	77			350
❌‌	gcp windows	237	77			349

Test Name	aws linux	aws windows	aws-ucws linux	aws-ucws windows	azure linux	azure windows	azure-ucws linux	azure-ucws windows	gcp linux	gcp windows
TestAccept	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL
TestAccept/bundle/deploy/dashboard/detect-change	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	❌‌FAIL	✅‌pass	❌‌FAIL	❌‌FAIL
TestAccept/bundle/deploy/dashboard/nested-folders	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	❌‌FAIL	❌‌FAIL
TestAccept/bundle/deploy/dashboard/simple	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	❌‌FAIL	❌‌FAIL
TestAccept/bundle/deploy/dashboard/simple_outside_bundle_root	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	❌‌FAIL	❌‌FAIL
TestAccept/bundle/deploy/dashboard/simple_syncroot	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	❌‌FAIL	❌‌FAIL
TestAccept/bundle/deploy/files/no-snapshot-sync	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	❌‌FAIL	❌‌FAIL
TestAccept/bundle/deploy/files/no-snapshot-sync/DATABRICKS_CLI_DEPLOYMENT=direct-exp	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	❌‌FAIL	❌‌FAIL
TestAccept/bundle/deploy/files/no-snapshot-sync/DATABRICKS_CLI_DEPLOYMENT=terraform	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	❌‌FAIL	❌‌FAIL
TestAccept/bundle/deploy/jobs/check-metadata	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	❌‌FAIL	❌‌FAIL
TestAccept/bundle/deploy/jobs/double-underscore-keys	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	❌‌FAIL	❌‌FAIL
TestAccept/bundle/deploy/jobs/fail-on-active-runs	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	❌‌FAIL	❌‌FAIL
TestAccept/bundle/deploy/jobs/fail-on-active-runs/DATABRICKS_CLI_DEPLOYMENT=direct-exp	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	❌‌FAIL	❌‌FAIL
TestAccept/bundle/deploy/jobs/fail-on-active-runs/DATABRICKS_CLI_DEPLOYMENT=terraform	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	❌‌FAIL	❌‌FAIL
TestAccept/bundle/deploy/mlops-stacks	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	❌‌FAIL	❌‌FAIL
TestAccept/bundle/deploy/pipeline/auto-approve	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	❌‌FAIL	❌‌FAIL
TestAccept/bundle/deploy/secret-scope	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	❌‌FAIL	❌‌FAIL
TestAccept/bundle/deployment/bind/cluster	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	❌‌FAIL	❌‌FAIL
TestAccept/bundle/deployment/bind/dashboard	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	❌‌FAIL	❌‌FAIL
TestAccept/bundle/deployment/bind/dashboard/recreation	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	❌‌FAIL	❌‌FAIL
TestAccept/bundle/deployment/bind/experiment	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	❌‌FAIL	❌‌FAIL
TestAccept/bundle/deployment/bind/job/job-abort-bind	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	❌‌FAIL	❌‌FAIL
TestAccept/bundle/deployment/bind/job/job-spark-python-task	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	❌‌FAIL	❌‌FAIL
TestAccept/bundle/deployment/bind/model-serving-endpoint	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	❌‌FAIL	❌‌FAIL
TestAccept/bundle/destroy/jobs-and-pipeline	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	❌‌FAIL	❌‌FAIL
TestAccept/bundle/local_state_staleness	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	❌‌FAIL	❌‌FAIL
TestAccept/bundle/local_state_staleness/DATABRICKS_CLI_DEPLOYMENT=direct-exp	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	❌‌FAIL	❌‌FAIL
TestAccept/bundle/local_state_staleness/DATABRICKS_CLI_DEPLOYMENT=terraform	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	❌‌FAIL	❌‌FAIL
TestAccept/bundle/resources/clusters/deploy/simple	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	❌‌FAIL	❌‌FAIL
TestAccept/bundle/resources/pipelines	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	❌‌FAIL	❌‌FAIL
TestAccept/bundle/resources/pipelines/DATABRICKS_CLI_DEPLOYMENT=direct-exp	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	❌‌FAIL	❌‌FAIL
TestAccept/bundle/resources/pipelines/DATABRICKS_CLI_DEPLOYMENT=terraform	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	❌‌FAIL	❌‌FAIL
TestAccept/bundle/templates/default-python/combinations	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL
TestAccept/bundle/templates/default-python/combinations/D=no/DATABRICKS_CLI_DEPLOYMENT=direct-exp/N=no/P=no/S=no	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	❌‌FAIL	❌‌FAIL
TestAccept/bundle/templates/default-python/combinations/D=no/DATABRICKS_CLI_DEPLOYMENT=direct-exp/N=no/P=no/S=yes	🙈‌skip	🙈‌skip	✅‌pass	✅‌pass	🙈‌skip	🙈‌skip	✅‌pass	✅‌pass	❌‌FAIL	❌‌FAIL
TestAccept/bundle/templates/default-python/combinations/D=no/DATABRICKS_CLI_DEPLOYMENT=direct-exp/N=no/P=yes/S=no	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL
TestAccept/bundle/templates/default-python/combinations/D=no/DATABRICKS_CLI_DEPLOYMENT=direct-exp/N=no/P=yes/S=yes	🙈‌skip	🙈‌skip	❌‌FAIL	❌‌FAIL	🙈‌skip	🙈‌skip	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL
TestAccept/bundle/templates/default-python/combinations/D=no/DATABRICKS_CLI_DEPLOYMENT=direct-exp/N=yes/P=no/S=no	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	❌‌FAIL	❌‌FAIL
TestAccept/bundle/templates/default-python/combinations/D=no/DATABRICKS_CLI_DEPLOYMENT=direct-exp/N=yes/P=no/S=yes	🙈‌skip	🙈‌skip	✅‌pass	✅‌pass	🙈‌skip	🙈‌skip	✅‌pass	✅‌pass	❌‌FAIL	❌‌FAIL
TestAccept/bundle/templates/default-python/combinations/D=no/DATABRICKS_CLI_DEPLOYMENT=direct-exp/N=yes/P=yes/S=no	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL
TestAccept/bundle/templates/default-python/combinations/D=no/DATABRICKS_CLI_DEPLOYMENT=direct-exp/N=yes/P=yes/S=yes	🙈‌skip	🙈‌skip	❌‌FAIL	❌‌FAIL	🙈‌skip	🙈‌skip	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL
TestAccept/bundle/templates/default-python/combinations/D=no/DATABRICKS_CLI_DEPLOYMENT=terraform/N=no/P=no/S=no	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	❌‌FAIL	❌‌FAIL
TestAccept/bundle/templates/default-python/combinations/D=no/DATABRICKS_CLI_DEPLOYMENT=terraform/N=no/P=no/S=yes	🙈‌skip	🙈‌skip	✅‌pass	✅‌pass	🙈‌skip	🙈‌skip	✅‌pass	✅‌pass	❌‌FAIL	❌‌FAIL
TestAccept/bundle/templates/default-python/combinations/D=no/DATABRICKS_CLI_DEPLOYMENT=terraform/N=no/P=yes/S=no	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL
TestAccept/bundle/templates/default-python/combinations/D=no/DATABRICKS_CLI_DEPLOYMENT=terraform/N=no/P=yes/S=yes	🙈‌skip	🙈‌skip	❌‌FAIL	❌‌FAIL	🙈‌skip	🙈‌skip	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL
TestAccept/bundle/templates/default-python/combinations/D=no/DATABRICKS_CLI_DEPLOYMENT=terraform/N=yes/P=no/S=no	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	❌‌FAIL	❌‌FAIL
TestAccept/bundle/templates/default-python/combinations/D=no/DATABRICKS_CLI_DEPLOYMENT=terraform/N=yes/P=no/S=yes	🙈‌skip	🙈‌skip	✅‌pass	✅‌pass	🙈‌skip	🙈‌skip	✅‌pass	✅‌pass	❌‌FAIL	❌‌FAIL
TestAccept/bundle/templates/default-python/combinations/D=no/DATABRICKS_CLI_DEPLOYMENT=terraform/N=yes/P=yes/S=no	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL
TestAccept/bundle/templates/default-python/combinations/D=no/DATABRICKS_CLI_DEPLOYMENT=terraform/N=yes/P=yes/S=yes	🙈‌skip	🙈‌skip	❌‌FAIL	❌‌FAIL	🙈‌skip	🙈‌skip	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL
TestAccept/bundle/templates/default-python/combinations/D=yes/DATABRICKS_CLI_DEPLOYMENT=direct-exp/N=no/P=no/S=no	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	❌‌FAIL	❌‌FAIL
TestAccept/bundle/templates/default-python/combinations/D=yes/DATABRICKS_CLI_DEPLOYMENT=direct-exp/N=no/P=no/S=yes	🙈‌skip	🙈‌skip	✅‌pass	✅‌pass	🙈‌skip	🙈‌skip	✅‌pass	✅‌pass	❌‌FAIL	❌‌FAIL
TestAccept/bundle/templates/default-python/combinations/D=yes/DATABRICKS_CLI_DEPLOYMENT=direct-exp/N=no/P=yes/S=no	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL
TestAccept/bundle/templates/default-python/combinations/D=yes/DATABRICKS_CLI_DEPLOYMENT=direct-exp/N=no/P=yes/S=yes	🙈‌skip	🙈‌skip	❌‌FAIL	❌‌FAIL	🙈‌skip	🙈‌skip	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL
TestAccept/bundle/templates/default-python/combinations/D=yes/DATABRICKS_CLI_DEPLOYMENT=direct-exp/N=yes/P=no/S=no	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	❌‌FAIL	❌‌FAIL
TestAccept/bundle/templates/default-python/combinations/D=yes/DATABRICKS_CLI_DEPLOYMENT=direct-exp/N=yes/P=no/S=yes	🙈‌skip	🙈‌skip	✅‌pass	✅‌pass	🙈‌skip	🙈‌skip	✅‌pass	✅‌pass	❌‌FAIL	❌‌FAIL
TestAccept/bundle/templates/default-python/combinations/D=yes/DATABRICKS_CLI_DEPLOYMENT=direct-exp/N=yes/P=yes/S=no	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL
TestAccept/bundle/templates/default-python/combinations/D=yes/DATABRICKS_CLI_DEPLOYMENT=direct-exp/N=yes/P=yes/S=yes	🙈‌skip	🙈‌skip	❌‌FAIL	❌‌FAIL	🙈‌skip	🙈‌skip	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL
TestAccept/bundle/templates/default-python/combinations/D=yes/DATABRICKS_CLI_DEPLOYMENT=terraform/N=no/P=no/S=no	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	❌‌FAIL	❌‌FAIL
TestAccept/bundle/templates/default-python/combinations/D=yes/DATABRICKS_CLI_DEPLOYMENT=terraform/N=no/P=no/S=yes	🙈‌skip	🙈‌skip	✅‌pass	✅‌pass	🙈‌skip	🙈‌skip	✅‌pass	✅‌pass	❌‌FAIL	❌‌FAIL
TestAccept/bundle/templates/default-python/combinations/D=yes/DATABRICKS_CLI_DEPLOYMENT=terraform/N=no/P=yes/S=no	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL
TestAccept/bundle/templates/default-python/combinations/D=yes/DATABRICKS_CLI_DEPLOYMENT=terraform/N=no/P=yes/S=yes	🙈‌skip	🙈‌skip	❌‌FAIL	❌‌FAIL	🙈‌skip	🙈‌skip	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL
TestAccept/bundle/templates/default-python/combinations/D=yes/DATABRICKS_CLI_DEPLOYMENT=terraform/N=yes/P=no/S=no	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	❌‌FAIL	❌‌FAIL
TestAccept/bundle/templates/default-python/combinations/D=yes/DATABRICKS_CLI_DEPLOYMENT=terraform/N=yes/P=no/S=yes	🙈‌skip	🙈‌skip	✅‌pass	✅‌pass	🙈‌skip	🙈‌skip	✅‌pass	✅‌pass	❌‌FAIL	❌‌FAIL
TestAccept/bundle/templates/default-python/combinations/D=yes/DATABRICKS_CLI_DEPLOYMENT=terraform/N=yes/P=yes/S=no	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL
TestAccept/bundle/templates/default-python/combinations/D=yes/DATABRICKS_CLI_DEPLOYMENT=terraform/N=yes/P=yes/S=yes	🙈‌skip	🙈‌skip	❌‌FAIL	❌‌FAIL	🙈‌skip	🙈‌skip	❌‌FAIL	❌‌FAIL	❌‌FAIL	❌‌FAIL
TestAccept/bundle/templates/default-python/integration_classic	🔄‌flaky	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	❌‌FAIL	❌‌FAIL
TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_CLI_DEPLOYMENT=terraform/UV_PYTHON=3.10	🔄‌flaky	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	❌‌FAIL	❌‌FAIL
TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_CLI_DEPLOYMENT=terraform/UV_PYTHON=3.11	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	❌‌FAIL	❌‌FAIL
TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_CLI_DEPLOYMENT=terraform/UV_PYTHON=3.12	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	❌‌FAIL	❌‌FAIL
TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_CLI_DEPLOYMENT=terraform/UV_PYTHON=3.13	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	❌‌FAIL	❌‌FAIL
TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_CLI_DEPLOYMENT=terraform/UV_PYTHON=3.9	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	❌‌FAIL	❌‌FAIL
TestAccept/terraform	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	❌‌FAIL	✅‌pass	✅‌pass
TestAccept/terraform/DATABRICKS_CLI_DEPLOYMENT=terraform	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	❌‌FAIL	✅‌pass	✅‌pass
TestDashboardAssumptions_WorkspaceImport	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	❌‌FAIL	✅‌pass	✅‌pass	✅‌pass
TestFetchRepositoryInfoAPI_FromRepo	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	❌‌FAIL	✅‌pass	✅‌pass
TestGenerateAndBind	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	❌‌FAIL	❌‌FAIL
TestGenerateFromExistingJobAndDeploy	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	❌‌FAIL	❌‌FAIL
TestGenerateFromExistingPipelineAndDeploy	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	❌‌FAIL	❌‌FAIL
TestSparkJarTaskDeployAndRunOnWorkspace	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	❌‌FAIL	❌‌FAIL
TestSparkJarTaskDeployAndRunOnWorkspace/Databricks_Runtime_14.3_LTS	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	❌‌FAIL	❌‌FAIL
TestSparkJarTaskDeployAndRunOnWorkspace/Databricks_Runtime_15.4_LTS	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	❌‌FAIL	❌‌FAIL
TestSyncEnsureRemotePathIsUsableIfRepoDoesntExist	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	🤯‌MISS	✅‌pass	✅‌pass
TestSyncEnsureRemotePathIsUsableIfRepoExists	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	🤯‌MISS	✅‌pass	✅‌pass
TestSyncEnsureRemotePathIsUsableInWorkspace	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	🤯‌MISS	✅‌pass	✅‌pass
TestSyncIncrementalSyncFileToPythonNotebook	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	🤯‌MISS	✅‌pass	✅‌pass
TestSyncIncrementalSyncPythonNotebookDelete	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	🤯‌MISS	✅‌pass	✅‌pass
TestSyncIncrementalSyncPythonNotebookToFile	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	✅‌pass	💥‌PANIC	✅‌pass	✅‌pass

…tests

Classify those cases either as MISS or PANIC, based on the output: ~/work/cli-main % ./tools/gh_report.py --run 15874943679 Already exists: .gh-logs/15874943679. If that directory contains partial results, delete it to re-download: rm -fr .gh-logs/15874943679 + ./tools/gh_parse.py .gh-logs/15874943679/*/* env 🙈‌skip ✅‌pass 💥‌PANIC ❌‌FAIL 🔄‌flaky ✅‌ aws-ucws/lin 265 400 ✅‌ aws-ucws/win 264 401 💥‌ az-ucws/lin 262 392 1 ✅‌ az-ucws/win 261 403 ✅‌ az/lin 364 298 ✅‌ az/win 363 299 ❌‌ gcp/lin 366 292 3 2 ❌‌ gcp/win 365 295 3 test gcp/lin gcp/win az-ucws/lin TestAccept ❌‌FAIL ❌‌FAIL TestAccept/bundle/deploy/dashboard/detect-change ❌‌FAIL ❌‌FAIL TestAccept/bundle/deployment/bind/dashboard ❌‌FAIL ❌‌FAIL TestReposCreateWithProvider 🔄‌flaky ✅‌pass TestReposDeleteByPath 🔄‌flaky ✅‌pass TestSyncNestedFolderSync ✅‌pass ✅‌pass 💥‌PANIC

denik requested review from andrewnester, anton-107, pietern and shreyas-goenka as code owners June 25, 2025 09:30

denik temporarily deployed to test-trigger-is June 25, 2025 09:30 — with GitHub Actions Inactive

denik force-pushed the denik/gh_report branch from 86a2e08 to 4f0ac16 Compare June 25, 2025 09:35

denik temporarily deployed to test-trigger-is June 25, 2025 09:35 — with GitHub Actions Inactive

denik temporarily deployed to test-trigger-is June 25, 2025 12:01 — with GitHub Actions Inactive

denik force-pushed the denik/gh_report branch from ab10b41 to d3cecbd Compare June 25, 2025 13:20

denik temporarily deployed to test-trigger-is June 25, 2025 13:20 — with GitHub Actions Inactive

denik enabled auto-merge June 25, 2025 13:24

denik temporarily deployed to test-trigger-is June 26, 2025 08:16 — with GitHub Actions Inactive

denik temporarily deployed to test-trigger-is June 26, 2025 09:07 — with GitHub Actions Inactive

denik temporarily deployed to test-trigger-is June 26, 2025 09:16 — with GitHub Actions Inactive

denik temporarily deployed to test-trigger-is June 26, 2025 09:18 — with GitHub Actions Inactive

denik force-pushed the denik/gh_report branch from 264b6f8 to 7374260 Compare June 26, 2025 10:22

denik temporarily deployed to test-trigger-is June 26, 2025 10:22 — with GitHub Actions Inactive

denik force-pushed the denik/gh_report branch from 7374260 to 055653d Compare June 26, 2025 10:35

denik temporarily deployed to test-trigger-is June 26, 2025 10:35 — with GitHub Actions Inactive

denik temporarily deployed to test-trigger-is June 26, 2025 10:40 — with GitHub Actions Inactive

denik temporarily deployed to test-trigger-is June 26, 2025 14:42 — with GitHub Actions Inactive

denik temporarily deployed to test-trigger-is June 26, 2025 15:49 — with GitHub Actions Inactive

denik temporarily deployed to test-trigger-is June 26, 2025 16:02 — with GitHub Actions Inactive

denik temporarily deployed to test-trigger-is June 26, 2025 16:03 — with GitHub Actions Inactive

denik temporarily deployed to test-trigger-is June 26, 2025 16:11 — with GitHub Actions Inactive

denik changed the title ~~Add tools/{gh_parse.py,gh_report.py}: stats and logs for integration tests~~ Add gh_report.py: stats and logs for integration tests Jun 26, 2025

denik temporarily deployed to test-trigger-is June 26, 2025 19:43 — with GitHub Actions Inactive

denik temporarily deployed to test-trigger-is June 26, 2025 20:00 — with GitHub Actions Inactive

denik temporarily deployed to test-trigger-is June 26, 2025 20:25 — with GitHub Actions Inactive

denik added 17 commits June 26, 2025 22:32

Add tools/{gh_parse.py,gh_report.py}: stats and logs for integration …

41cafc3

…tests

update gitignore

af6254d

make sure to find gh_parse.py no matter how we run

55246d5

fix bug when calculating "interesting envs"

14cb570

Also read GITHUB_REPOSITORY

b5f06e5

missing tests support

01c4e5f

try stdout/stderr reconfigure https://stackoverflow.com/a/73486671

ff3d4c9

add --markdown

b1865a3

table formatting

f3c9a4d

propagate --markdown

9e99ae4

table formatting

2d41df8

sort columns

8214468

move comment to the right place

cdb6720

clean up reconfigure(), did not help

f7b4ab2

log info about parsed files

f9e71ec

fix f-string

c0ddd52

denik force-pushed the denik/gh_report branch from 3e21f23 to c0ddd52 Compare June 26, 2025 20:32

denik temporarily deployed to test-trigger-is June 26, 2025 20:32 — with GitHub Actions Inactive

simplify, do not run through shell

c3bbde3

denik temporarily deployed to test-trigger-is June 26, 2025 20:37 — with GitHub Actions Inactive

call via python executable

1eae7a3

denik temporarily deployed to test-trigger-is June 26, 2025 20:42 — with GitHub Actions Inactive

group ucws and non-ucws together

d834569

denik temporarily deployed to test-trigger-is June 26, 2025 21:01 — with GitHub Actions Inactive

andrewnester approved these changes Jun 27, 2025

View reviewed changes

denik disabled auto-merge June 27, 2025 09:44

denik merged commit 4f398fd into main Jun 27, 2025
12 of 13 checks passed

denik deleted the denik/gh_report branch June 27, 2025 09:44

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add gh_report.py: stats and logs for integration tests#3115

Add gh_report.py: stats and logs for integration tests#3115
denik merged 20 commits intomainfrom
denik/gh_report

denik commented Jun 25, 2025 •

edited

Loading

Uh oh!

eng-dev-ecosystem-bot commented Jun 26, 2025 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

denik commented Jun 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Why

Tests

Getting stats:

Getting output:

Uh oh!

eng-dev-ecosystem-bot commented Jun 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

denik commented Jun 25, 2025 •

edited

Loading

eng-dev-ecosystem-bot commented Jun 26, 2025 •

edited

Loading