Skip to content

Fix incorrect capture of pip output in verbose mode#332

Merged
asnare merged 6 commits into
mainfrom
fix/pip-output-capture
May 1, 2026
Merged

Fix incorrect capture of pip output in verbose mode#332
asnare merged 6 commits into
mainfrom
fix/pip-output-capture

Conversation

@asnare
Copy link
Copy Markdown
Contributor

@asnare asnare commented Mar 30, 2026

This PR fixes a minor bug encountered while trying to diagnose failures with the builtin wheel-building mechanism. When the verbose-flag was enabled, the pip command was not invoked properly (and would fail immediately) because subprocess.STDOUT is only valid for the stderr argument: for stdout it must instead be None for the output to be passed through.

Verbose-mode was broken due to the incorrect value for stdout being passed.
@asnare asnare self-assigned this Mar 30, 2026
@asnare asnare requested a review from a team as a code owner March 30, 2026 16:25
@asnare asnare added the bug Something isn't working label Mar 30, 2026
@asnare asnare added this to UCX Mar 30, 2026
@asnare asnare moved this to Ready for Review in UCX Mar 30, 2026
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Apr 10, 2026

❌ 37/40 passed, 3 failed, 2 skipped, 1m57s total

❌ test_upload_dbfs: subprocess.CalledProcessError: Command '['/home/runner/work/blueprint/blueprint/.venv/bin/python', '-m', 'pip', 'wheel', '--wheel-dir', '/tmp/tmpkwedvs53', '/tmp/tmpkwedvs53/working-copy', '--no-deps']' returned non-zero exit status 1. (10.376s)
subprocess.CalledProcessError: Command '['/home/runner/work/blueprint/blueprint/.venv/bin/python', '-m', 'pip', 'wheel', '--wheel-dir', '/tmp/tmpkwedvs53', '/tmp/tmpkwedvs53/working-copy', '--no-deps']' returned non-zero exit status 1.
12:31 DEBUG [databricks.sdk] Loaded from environment
12:31 DEBUG [databricks.sdk] GET /.well-known/databricks-config
< 200 OK
< {
<   "account_id": "304d4d89-9def-4c52-8448-e8528a11d2aa",
<   "cloud_provider": "CLOUD_ENV",
<   "host_type": "workspace",
<   "oidc_endpoint": "https://DATABRICKS_HOST/oidc",
<   "token_federation_TEST_SCHEMA_oidc_audiences": [
<     "https://DATABRICKS_HOST/oidc/v1/token"
<   ],
<   "workspace_id": "841570839244994"
< }
12:31 DEBUG [databricks.sdk] Resolved account_id from host metadata: 304d4d89-9def-4c52-8448-e8528a11d2aa
12:31 DEBUG [databricks.sdk] Resolved workspace_id from host metadata: 841570839244994
12:31 DEBUG [databricks.sdk] Resolved discovery_url from host metadata: https://DATABRICKS_HOST/oidc/.well-known/oauth-authorization-server
12:31 DEBUG [databricks.sdk] Ignoring pat auth, because metadata-service is preferred
12:31 DEBUG [databricks.sdk] Ignoring basic auth, because metadata-service is preferred
12:31 DEBUG [databricks.sdk] Attempting to configure auth: metadata-service
12:31 INFO [databricks.sdk] Using Databricks Metadata Service authentication
[gw6] linux -- Python 3.10.20 /home/runner/work/blueprint/blueprint/.venv/bin/python
12:31 DEBUG [databricks.sdk] Loaded from environment
12:31 DEBUG [databricks.sdk] GET /.well-known/databricks-config
< 200 OK
< {
<   "account_id": "304d4d89-9def-4c52-8448-e8528a11d2aa",
<   "cloud_provider": "CLOUD_ENV",
<   "host_type": "workspace",
<   "oidc_endpoint": "https://DATABRICKS_HOST/oidc",
<   "token_federation_TEST_SCHEMA_oidc_audiences": [
<     "https://DATABRICKS_HOST/oidc/v1/token"
<   ],
<   "workspace_id": "841570839244994"
< }
12:31 DEBUG [databricks.sdk] Resolved account_id from host metadata: 304d4d89-9def-4c52-8448-e8528a11d2aa
12:31 DEBUG [databricks.sdk] Resolved workspace_id from host metadata: 841570839244994
12:31 DEBUG [databricks.sdk] Resolved discovery_url from host metadata: https://DATABRICKS_HOST/oidc/.well-known/oauth-authorization-server
12:31 DEBUG [databricks.sdk] Ignoring pat auth, because metadata-service is preferred
12:31 DEBUG [databricks.sdk] Ignoring basic auth, because metadata-service is preferred
12:31 DEBUG [databricks.sdk] Attempting to configure auth: metadata-service
12:31 INFO [databricks.sdk] Using Databricks Metadata Service authentication
12:31 DEBUG [databricks.labs.blueprint.wheels] Building wheel for /tmp/tmpkwedvs53/working-copy in /tmp/tmpkwedvs53
12:31 DEBUG [databricks.sdk] Loaded from environment
12:31 DEBUG [databricks.sdk] GET /.well-known/databricks-config
< 200 OK
< {
<   "account_id": "304d4d89-9def-4c52-8448-e8528a11d2aa",
<   "cloud_provider": "CLOUD_ENV",
<   "host_type": "workspace",
<   "oidc_endpoint": "https://DATABRICKS_HOST/oidc",
<   "token_federation_TEST_SCHEMA_oidc_audiences": [
<     "https://DATABRICKS_HOST/oidc/v1/token"
<   ],
<   "workspace_id": "841570839244994"
< }
12:31 DEBUG [databricks.sdk] Resolved account_id from host metadata: 304d4d89-9def-4c52-8448-e8528a11d2aa
12:31 DEBUG [databricks.sdk] Resolved workspace_id from host metadata: 841570839244994
12:31 DEBUG [databricks.sdk] Resolved discovery_url from host metadata: https://DATABRICKS_HOST/oidc/.well-known/oauth-authorization-server
12:31 DEBUG [databricks.sdk] Ignoring pat auth, because metadata-service is preferred
12:31 DEBUG [databricks.sdk] Ignoring basic auth, because metadata-service is preferred
12:31 DEBUG [databricks.sdk] Attempting to configure auth: metadata-service
12:31 INFO [databricks.sdk] Using Databricks Metadata Service authentication
12:31 DEBUG [databricks.labs.blueprint.wheels] Building wheel for /tmp/tmpkwedvs53/working-copy in /tmp/tmpkwedvs53
[gw6] linux -- Python 3.10.20 /home/runner/work/blueprint/blueprint/.venv/bin/python
❌ test_upload_wsfs: subprocess.CalledProcessError: Command '['/home/runner/work/blueprint/blueprint/.venv/bin/python', '-m', 'pip', 'wheel', '--wheel-dir', '/tmp/tmpwk7r9d0n', '/tmp/tmpwk7r9d0n/working-copy', '--no-deps']' returned non-zero exit status 1. (10.413s)
subprocess.CalledProcessError: Command '['/home/runner/work/blueprint/blueprint/.venv/bin/python', '-m', 'pip', 'wheel', '--wheel-dir', '/tmp/tmpwk7r9d0n', '/tmp/tmpwk7r9d0n/working-copy', '--no-deps']' returned non-zero exit status 1.
12:31 DEBUG [databricks.sdk] Loaded from environment
12:31 DEBUG [databricks.sdk] GET /.well-known/databricks-config
< 200 OK
< {
<   "account_id": "304d4d89-9def-4c52-8448-e8528a11d2aa",
<   "cloud_provider": "CLOUD_ENV",
<   "host_type": "workspace",
<   "oidc_endpoint": "https://DATABRICKS_HOST/oidc",
<   "token_federation_TEST_SCHEMA_oidc_audiences": [
<     "https://DATABRICKS_HOST/oidc/v1/token"
<   ],
<   "workspace_id": "841570839244994"
< }
12:31 DEBUG [databricks.sdk] Resolved account_id from host metadata: 304d4d89-9def-4c52-8448-e8528a11d2aa
12:31 DEBUG [databricks.sdk] Resolved workspace_id from host metadata: 841570839244994
12:31 DEBUG [databricks.sdk] Resolved discovery_url from host metadata: https://DATABRICKS_HOST/oidc/.well-known/oauth-authorization-server
12:31 DEBUG [databricks.sdk] Ignoring pat auth, because metadata-service is preferred
12:31 DEBUG [databricks.sdk] Ignoring basic auth, because metadata-service is preferred
12:31 DEBUG [databricks.sdk] Attempting to configure auth: metadata-service
12:31 INFO [databricks.sdk] Using Databricks Metadata Service authentication
[gw7] linux -- Python 3.10.20 /home/runner/work/blueprint/blueprint/.venv/bin/python
12:31 DEBUG [databricks.sdk] Loaded from environment
12:31 DEBUG [databricks.sdk] GET /.well-known/databricks-config
< 200 OK
< {
<   "account_id": "304d4d89-9def-4c52-8448-e8528a11d2aa",
<   "cloud_provider": "CLOUD_ENV",
<   "host_type": "workspace",
<   "oidc_endpoint": "https://DATABRICKS_HOST/oidc",
<   "token_federation_TEST_SCHEMA_oidc_audiences": [
<     "https://DATABRICKS_HOST/oidc/v1/token"
<   ],
<   "workspace_id": "841570839244994"
< }
12:31 DEBUG [databricks.sdk] Resolved account_id from host metadata: 304d4d89-9def-4c52-8448-e8528a11d2aa
12:31 DEBUG [databricks.sdk] Resolved workspace_id from host metadata: 841570839244994
12:31 DEBUG [databricks.sdk] Resolved discovery_url from host metadata: https://DATABRICKS_HOST/oidc/.well-known/oauth-authorization-server
12:31 DEBUG [databricks.sdk] Ignoring pat auth, because metadata-service is preferred
12:31 DEBUG [databricks.sdk] Ignoring basic auth, because metadata-service is preferred
12:31 DEBUG [databricks.sdk] Attempting to configure auth: metadata-service
12:31 INFO [databricks.sdk] Using Databricks Metadata Service authentication
12:31 DEBUG [databricks.labs.blueprint.wheels] Building wheel for /tmp/tmpwk7r9d0n/working-copy in /tmp/tmpwk7r9d0n
12:31 DEBUG [databricks.sdk] Loaded from environment
12:31 DEBUG [databricks.sdk] GET /.well-known/databricks-config
< 200 OK
< {
<   "account_id": "304d4d89-9def-4c52-8448-e8528a11d2aa",
<   "cloud_provider": "CLOUD_ENV",
<   "host_type": "workspace",
<   "oidc_endpoint": "https://DATABRICKS_HOST/oidc",
<   "token_federation_TEST_SCHEMA_oidc_audiences": [
<     "https://DATABRICKS_HOST/oidc/v1/token"
<   ],
<   "workspace_id": "841570839244994"
< }
12:31 DEBUG [databricks.sdk] Resolved account_id from host metadata: 304d4d89-9def-4c52-8448-e8528a11d2aa
12:31 DEBUG [databricks.sdk] Resolved workspace_id from host metadata: 841570839244994
12:31 DEBUG [databricks.sdk] Resolved discovery_url from host metadata: https://DATABRICKS_HOST/oidc/.well-known/oauth-authorization-server
12:31 DEBUG [databricks.sdk] Ignoring pat auth, because metadata-service is preferred
12:31 DEBUG [databricks.sdk] Ignoring basic auth, because metadata-service is preferred
12:31 DEBUG [databricks.sdk] Attempting to configure auth: metadata-service
12:31 INFO [databricks.sdk] Using Databricks Metadata Service authentication
12:31 DEBUG [databricks.labs.blueprint.wheels] Building wheel for /tmp/tmpwk7r9d0n/working-copy in /tmp/tmpwk7r9d0n
[gw7] linux -- Python 3.10.20 /home/runner/work/blueprint/blueprint/.venv/bin/python
❌ test_upload_upstreams: subprocess.CalledProcessError: Command '['/home/runner/work/blueprint/blueprint/.venv/bin/python', '-m', 'pip', 'wheel', '--wheel-dir', '/tmp/tmpwk39ww0q', '/tmp/tmpwk39ww0q/working-copy', '--no-deps']' returned non-zero exit status 1. (8.768s)
subprocess.CalledProcessError: Command '['/home/runner/work/blueprint/blueprint/.venv/bin/python', '-m', 'pip', 'wheel', '--wheel-dir', '/tmp/tmpwk39ww0q', '/tmp/tmpwk39ww0q/working-copy', '--no-deps']' returned non-zero exit status 1.
12:32 DEBUG [databricks.sdk] Loaded from environment
12:32 DEBUG [databricks.sdk] GET /.well-known/databricks-config
< 200 OK
< {
<   "account_id": "304d4d89-9def-4c52-8448-e8528a11d2aa",
<   "cloud_provider": "CLOUD_ENV",
<   "host_type": "workspace",
<   "oidc_endpoint": "https://DATABRICKS_HOST/oidc",
<   "token_federation_TEST_SCHEMA_oidc_audiences": [
<     "https://DATABRICKS_HOST/oidc/v1/token"
<   ],
<   "workspace_id": "841570839244994"
< }
12:32 DEBUG [databricks.sdk] Resolved account_id from host metadata: 304d4d89-9def-4c52-8448-e8528a11d2aa
12:32 DEBUG [databricks.sdk] Resolved workspace_id from host metadata: 841570839244994
12:32 DEBUG [databricks.sdk] Resolved discovery_url from host metadata: https://DATABRICKS_HOST/oidc/.well-known/oauth-authorization-server
12:32 DEBUG [databricks.sdk] Ignoring pat auth, because metadata-service is preferred
12:32 DEBUG [databricks.sdk] Ignoring basic auth, because metadata-service is preferred
12:32 DEBUG [databricks.sdk] Attempting to configure auth: metadata-service
12:32 INFO [databricks.sdk] Using Databricks Metadata Service authentication
[gw7] linux -- Python 3.10.20 /home/runner/work/blueprint/blueprint/.venv/bin/python
12:32 DEBUG [databricks.sdk] Loaded from environment
12:32 DEBUG [databricks.sdk] GET /.well-known/databricks-config
< 200 OK
< {
<   "account_id": "304d4d89-9def-4c52-8448-e8528a11d2aa",
<   "cloud_provider": "CLOUD_ENV",
<   "host_type": "workspace",
<   "oidc_endpoint": "https://DATABRICKS_HOST/oidc",
<   "token_federation_TEST_SCHEMA_oidc_audiences": [
<     "https://DATABRICKS_HOST/oidc/v1/token"
<   ],
<   "workspace_id": "841570839244994"
< }
12:32 DEBUG [databricks.sdk] Resolved account_id from host metadata: 304d4d89-9def-4c52-8448-e8528a11d2aa
12:32 DEBUG [databricks.sdk] Resolved workspace_id from host metadata: 841570839244994
12:32 DEBUG [databricks.sdk] Resolved discovery_url from host metadata: https://DATABRICKS_HOST/oidc/.well-known/oauth-authorization-server
12:32 DEBUG [databricks.sdk] Ignoring pat auth, because metadata-service is preferred
12:32 DEBUG [databricks.sdk] Ignoring basic auth, because metadata-service is preferred
12:32 DEBUG [databricks.sdk] Attempting to configure auth: metadata-service
12:32 INFO [databricks.sdk] Using Databricks Metadata Service authentication
12:32 DEBUG [databricks.labs.blueprint.wheels] Building wheel for /tmp/tmpwk39ww0q/working-copy in /tmp/tmpwk39ww0q
12:32 DEBUG [databricks.sdk] Loaded from environment
12:32 DEBUG [databricks.sdk] GET /.well-known/databricks-config
< 200 OK
< {
<   "account_id": "304d4d89-9def-4c52-8448-e8528a11d2aa",
<   "cloud_provider": "CLOUD_ENV",
<   "host_type": "workspace",
<   "oidc_endpoint": "https://DATABRICKS_HOST/oidc",
<   "token_federation_TEST_SCHEMA_oidc_audiences": [
<     "https://DATABRICKS_HOST/oidc/v1/token"
<   ],
<   "workspace_id": "841570839244994"
< }
12:32 DEBUG [databricks.sdk] Resolved account_id from host metadata: 304d4d89-9def-4c52-8448-e8528a11d2aa
12:32 DEBUG [databricks.sdk] Resolved workspace_id from host metadata: 841570839244994
12:32 DEBUG [databricks.sdk] Resolved discovery_url from host metadata: https://DATABRICKS_HOST/oidc/.well-known/oauth-authorization-server
12:32 DEBUG [databricks.sdk] Ignoring pat auth, because metadata-service is preferred
12:32 DEBUG [databricks.sdk] Ignoring basic auth, because metadata-service is preferred
12:32 DEBUG [databricks.sdk] Attempting to configure auth: metadata-service
12:32 INFO [databricks.sdk] Using Databricks Metadata Service authentication
12:32 DEBUG [databricks.labs.blueprint.wheels] Building wheel for /tmp/tmpwk39ww0q/working-copy in /tmp/tmpwk39ww0q
[gw7] linux -- Python 3.10.20 /home/runner/work/blueprint/blueprint/.venv/bin/python

Running from acceptance #429

Copy link
Copy Markdown
Collaborator

@sundarshankar89 sundarshankar89 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@asnare asnare merged commit 287fcd2 into main May 1, 2026
6 of 12 checks passed
@asnare asnare deleted the fix/pip-output-capture branch May 1, 2026 12:52
@github-project-automation github-project-automation Bot moved this from Ready for Review to Done in UCX May 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants