Skip to content

v1.27.0

Latest

Choose a tag to compare

@oss-cli oss-cli released this 19 Nov 09:47
Immutable release. Only release title and notes can be modified.
547a4b7

Notable Changes

  • The Databricks CLI is now available on PyPI as databricks: run it with uvx databricks, or install it with uv tool install databricks, pipx install databricks or pip install databricks. The package contains only the prebuilt CLI binary. Uninstall the legacy databricks-cli package first, since both install a databricks command. (#7704, #7777)

CLI

  • Add databricks sandbox fs commands (cp, ls, cat, mkdir, rm) to transfer files between your local filesystem and sandboxes using <sandbox-id>:/path. (#7372)
  • databricks fs commands retry reads (stat, list, open) and deletes on DBFS and Volumes when the server returns a transient error or does not respond within the HTTP timeout. (#7556)
  • Retry UC Volumes file operations (fs cp, fs ls, fs rm, bundle uploads to Volumes) when the Files API returns HTTP 408, 429, 502, 503 or 504 (for example "AWS S3 is throttling requests"), or on a transient network error; this regressed in v1.10.0. (#7559)
  • Stop warning when the host metadata request (/.well-known/databricks-config) times out; the CLI already falls back to your config, and the message is still shown with --debug. (#7752)
  • Add databricks aitools auto-install disable and enable to turn off (or back on) the automatic installation of Databricks AI tools on CLI upgrades. A global-scope databricks aitools uninstall also turns it off. (#7701)
  • Stop printing the Databricks AI tooling install hint on auth, configure, current-user me, version, help, and shell-completion commands. (#7452)
  • When Codex, Cursor, GitHub Copilot CLI, OpenCode, Gemini CLI, Antigravity, Pi, Goose or Kiro run the CLI without Databricks skills installed for that agent, the CLI now prints a one-line hint to install them with databricks aitools install --agents <agent> --scope global, previously shown only to Claude Code. (#7586)
  • databricks aitools install now rejects an unknown --agents name before it shows the scope prompt, instead of after. (#7469)
  • databricks aitools install no longer asks "Proceed?" after the agent picker; the picker selection confirms the plan. The picker title now shows the install scope, its hint says Enter installs, --skills-only runs label plugin agents "skills" and pre-check every agent whose config dir exists, and the plan summary after it is reduced to the scope line. (#7478)
  • databricks aitools install --output json reports more specific error_category values: SKILLS_FETCH_FAILED, RELEASE_LOOKUP_FAILED, RATE_LIMITED, MANIFEST_INVALID, STATE_INVALID and PLUGIN_CLI_TIMEOUT. A coding agent CLI that does not answer in time is now reported as timed out after <duration>, followed by any stderr it wrote before it was stopped. (#7475)
  • databricks aitools install no longer prompts when a coding agent runs it, even in a pseudo-terminal; it uses the non-interactive defaults instead and prints a note saying so. (#7477)
  • databricks aitools install now prints the directory it installed skills into (for example Installed 1 skill in ~/.databricks/aitools/skills.), so agents can open the installed SKILL.md files without searching for them. (#7668)
  • databricks aitools install now ends a failed install with a step to recover, in the text output and in a new recovery field of --output json. (#7480)
  • databricks aitools install now installs plugins for the other coding agents even when installing skill files fails. (#7479)
  • databricks aitools install now installs skill files for a detected coding agent whose CLI is not on PATH (for example a Claude Code desktop or IDE install), unless it already has the plugin, says how to switch to the plugin later, and warns when the current project also has the plugin. databricks aitools update no longer tries to install the plugin for such an agent, and no longer writes skill files next to a plugin that such an agent already has. (#7482)
  • Fix databricks aitools install waiting a long time on an unreachable or slow workspace while it resolves authentication for telemetry; that wait is now limited to a few seconds. (#7468)
  • databricks aitools install, update and list now refuse a skills manifest that names a skill or file outside the skills directory, before writing any skill files. A skill whose name or file paths are not portable is skipped with a warning, and update keeps its installed copy. (#7722)
  • databricks aitools install and databricks aitools update refresh only the marketplace the Databricks plugin is listed in, so another unreachable marketplace no longer blocks them. When that refresh fails, both commands warn, install or update the plugin from the local copy of the marketplace, and record the plugin version as unknown. (#7473)
  • databricks aitools install and databricks aitools update now wait up to 20 seconds instead of 5 for a coding agent's CLI to answer the plugin capability check, so a slow first start is less likely to report the agent as unavailable. (#7472)
  • databricks aitools install no longer fails a plugin-only install when the lookup of the latest skills release on GitHub fails or is rate-limited. The plugin is installed and its version is recorded as unknown. (#7474)
  • databricks aitools commands now send usage telemetry, and the CLI records when it recommends installing Databricks AI tools. When the configured workspace is slow or unreachable, aitools install, update and uninstall can take up to about 6 seconds longer, and list, version and runs that stop early up to about 3 seconds. Set DATABRICKS_CLI_DISABLE_TELEMETRY=1 to opt out. (#7467, #7469, #7470)
  • databricks aitools install and databricks aitools update now report a coding agent CLI shim (.cmd or .bat) on Windows that prints its plugin help and then exits with an error as shim-exited-with-error, with the command to run to see the error, instead of reporting the CLI as not on PATH. (#7476)
  • apps init no longer suggests installing skills when every detected coding agent already has the Databricks plugin. (#7467)
  • databricks auth login now gives up on the optional workspace lookup after about 5 seconds instead of retrying for about 5 minutes, and explains the fallback to the manual workspace ID prompt. (#7522)
  • Throttle Claude Code's native plugin install hint separately from the text hint: it prints at most once per 10 minutes per session, so the install dialog still comes up when a subagent, a hook, or a command outside a Claude Code subprocess runs the session's first databricks command. (#7692)
  • Print Claude Code's native plugin install hint alongside the text hint, so Claude Code offers a one-click install of the Databricks plugin. Claude Code now counts as having the tooling only when it loads it: skills only in ~/.databricks/aitools/skills don't count, and a project- or local-scope plugin counts only when the CLI runs inside that project. auth token, auth docker token and CLI subprocesses no longer print install hints. (#7585)
  • Interactive prompts and pagers now work in Git for Windows' bash and other MSYS2 shells when they run under ConPTY (the default since Git for Windows 2.47) or directly in a Windows console. (#7699)
  • Add the databricks feedback command to send feedback about the CLI to Databricks. (#7724)
  • Fix shell completion in databricks fs commands for local paths typed with a leading ./ or repeated separators, which previously offered no suggestions. (#7679)
  • Fix a panic in databricks labs commands when a non-string SDK setting such as DATABRICKS_DEBUG_HEADERS or DATABRICKS_RATE_LIMIT is set in the environment. (#7673)
  • Fix a panic in databricks labs install for projects whose labs.yml has no install section. (#7673)
  • NO_COLOR no longer disables prompts, spinners or the list pager; they run without color. (#7700)
  • Add the Beta databricks pipelines test command, which runs the tests of a pipeline, and the testing property of pipelines in bundles. (#7429)
  • Add databricks sandbox forward, which serves an app running in a sandbox at a local URL. (#7395)
  • Bound the whole telemetry upload at exit, including config resolution and authentication, by its timeout, so a stalled auth or telemetry endpoint no longer keeps a command from exiting. (#7467)
  • databricks version --check now suggests databricks aitools install on stderr when a coding agent is detected without the Databricks plugin or skills. (#7471)
  • databricks aitools install, update and uninstall now exit with code 1 when they do nothing they were asked to. install fails when it sets up nothing: no coding agent is detected, every agent is skipped or fails to install, or no skill in the release can be installed (also with --path). update fails when every update or check it attempts fails, and uninstall when every removal it attempts fails. Scripts that ran these commands as a best-effort step and relied on exit code 0 should handle the new exit code. (#7481)
  • Add file:<path> selectors to pipelines run and pipeline bundle run. --refresh and --full-refresh now use space-separated lists, such as --refresh "orders payments", instead of comma-separated lists. Single names containing spaces now require backticks, e.g. --refresh 'table:`order details`'. (#7184)

AI Runtime

  • air get -o json now reports PENDING while the run waits for compute, matching air list, and includes the run's status_message. (#7455)
  • Wrap long failure reasons and log lines in the air list info and logs panes, keep the MLflow link (labeled with its retry) when a run's last retry failed before starting, and point air logs at earlier retries when the last one has no logs. (#7456)
  • databricks air logs --node now reports an out-of-range node index when it can determine the run's node count, instead of looking like a successful run with no logs. (#7516)
  • Add experiment, duration, accelerator, and dashboard metadata to air list --output json. For retried runs, started_at now reports the latest attempt, matching air get. (#7568)
  • air list shortens MLflow run names and unnamed-run IDs by keeping their end behind a leading …, and its columns no longer overflow on wide (for example CJK) text; air get shows an unnamed run's full MLflow run ID. (#7582)
  • Shrink the Experiment and User columns of the interactive air list table to fit the terminal width, keeping the end of a shortened experiment name. (#7583)
  • Show the experiment, accelerators and configuration in the air list info pane for runs where it showed N/A. (#7587)
  • Strip terminal control characters from the MLflow run, experiment and user names shown by air list and air get, so a run name cannot clear or retitle the terminal of whoever lists it. (#7601)
  • air logs, air get, and air run --watch no longer make an extra credential-check request before fetching the run, and air get reuses the configured workspace ID instead of looking it up. (#7613)
  • air list no longer waits on extra current-user requests before listing runs, which makes the default list about 150ms faster. (#7615)
  • air list --all-status requests larger index pages, so it makes far fewer round trips once the server supports page_size. (#7616)
  • Fit the interactive air list table to narrow terminals: when the Experiment and User columns are at their floors, shorten Started to MM-DD HH:MM and size the MLflow column to its labels, then hide Accelerators, User, MLflow and Progress in that order, naming the hidden columns in the hint line. (#7629)
  • air get -o json now includes mlflow_run_id and mlflow_experiment_id, so a run can be handed to MLflow-based steps without parsing mlflow_url. They identify the MLflow run mlflow_url links to, including one from an earlier retry that mlflow_retry names. (#7636)
  • air get and the air list info pane now show the environment of runs on the Databricks AI environment (databricks_ai_v*) instead of N/A. (#7637)
  • air run now rejects a code_source.include_paths path that selects no files instead of uploading a snapshot without it; glob patterns and pathspec magic are left to git. (#7643)
  • air convert-to-dabs now suggests the --output-dir that converts a working-tree workload whose code_source.root_path is outside the bundle directory, instead of suggesting sync.paths. (#7644)
  • air run now lists a code_source.root_path outside a git repository relative to root_path, so running from a subdirectory of root_path no longer uploads an empty or partial snapshot. (#7646)
  • air list --all-status now fills --limit past purged or inaccessible indexed runs. (#7743)
  • air run now warns about missing PyTorch when a pinned snapshot's include_paths selects a Python file. (#7744)
  • air cancel JOB_RUN_ID now rejects run IDs that are not AI Runtime runs instead of cancelling unrelated Jobs runs. In -o json output, the failed[].error text for a run that does not exist or that you cannot view now starts with failed to get run <id>: instead of the cancel API error. (#7569)
  • air get -o json now reports mlflow_retry when the MLflow link comes from an earlier retry because the run's last retry never reached MLflow. (#7620)
  • databricks air run --help now shows how to install the databricks-ai-runtime coding agent skill. (#7652)
  • air run --watch -o json now reports each lifecycle STATUS transition once and never reports a non-terminal state after the run has ended, including when logs are served from MLflow artifacts. (#7621)
  • air convert-to-dabs now explains the --output-dir workaround when code_source.root_path resolves to the bundle root, and notes when the generated job carries the default max_retries. (#7579)
  • air run now refuses to upload a code_source snapshot that selects no files (for example when root_path is git-ignored) and explains the cause, and --dry-run runs the snapshot file selection and reports the file count. (#7570)
  • Repeat the Jobs run link after live log streaming ends. (#7571)
  • air get -o json includes run details including flat accelerator_type and num_accelerators. Use --include-config to include stored config, which may contain plaintext env_variables and parameters; downloads over 1 MiB are omitted, and config-dependent max_retries is otherwise null. (#7624)
  • air get and the air list info pane no longer show Max Retries 0 when the Jobs API leaves the task's retry limit out. For a run submitted with air run, the limit now comes from the run's uploaded config, shown as 3 (CLI default) when the config doesn't set max_retries; it shows N/A when the limit can't be known (job runs, multi-container runs, legacy runs with no explicit value, or an unreadable config). (#7623)
  • Reorganize databricks air help: subcommands are grouped by task, air run has examples, and YAML fields are listed under headings, with pool fields marked optional and command, compute.num_accelerators, and compute.accelerator_type correctly marked required. (#7512)
  • air run -h config now lists the environment variables AI Runtime sets on every node (such as MLFLOW_RUN_ID and CODE_SOURCE_PATH), and air logs --help notes that --download-to cannot be combined with --tail or --minutes. (#7655)
  • Support full workspace accelerator types in air list filters while preserving family matching, and reject invalid filter values. (#7515)
  • databricks air list --all-status now warns when the AI Runtime run index is unavailable and it falls back to scanning Jobs runs. That fallback scan is capped at 250 runs, and a stuck index request fails after 15 seconds instead of retrying for minutes. --all-status --filter user=<your email> now uses the run index instead of scanning the workspace's full run history. (#7619)
  • Return structured JSON errors for invalid air list --filter values when -o json is used. (#7612)
  • air list --all-status no longer fails when a listed run has been purged. (#7523)
  • Behavior change: databricks air logs now prints the logs available now and exits, also for an active run, matching kubectl logs, docker logs and databricks apps logs. Pass --follow (-f) to stream new lines until the run completes; --tail N --follow prints the last N lines first. databricks air run --watch still submits and streams. (#7658)
  • databricks air logs -o json and databricks air run --watch -o json now emit a WAITING event, for example {"type":"WAITING","message":"Waiting for accelerator compute capacity..."}, when the reported waiting status changes. (#7695)
  • databricks air logs on a run that ended in the last two minutes now waits for the log index to settle before printing, instead of sometimes dropping lines that were indexed late. (#7663)
  • air logs and air run --watch now print the waiting status (for example Waiting for accelerator compute capacity...) as a line on stderr when stderr is not a terminal, instead of printing nothing until the run produces logs. (#7657)
  • Route AIR log links to the MLflow Logs page instead of the hidden artifact path. (#7491)
  • Invalid compute.accelerator_type errors and air run -h config.compute no longer list the CLI's built-in accelerator types, which could differ from the workspace's. (#7520)
  • databricks air --help now notes that GPU pools are not required to run workloads. (#7581)
  • air run now prints the job run URL with the ?o=<workspace-id> suffix, matching air get and air list; dashboard_url in air run -o json and the --watch events gains it too. When air list cannot resolve the workspace ID, its links now omit ?o= instead of ending in ?o=0. (#7573)
  • databricks air run now reports an unknown workload YAML key by its field path, suggests the closest valid field, and points to -h config instead of naming an internal Go type. (#7574)
  • air run (including --dry-run) now warns before submitting when the code_source imports torch but PyTorch is not declared: the environment is the Standard environment (version 5 or later) with no torch dependency, so the run may fail after it waits for a GPU. (#7656)

Bundles

  • Add isolation_mode to the catalogs resource, so a catalog can be deployed as OPEN or ISOLATED. (#7463)
  • bundle validate no longer warns when resource permissions and top-level or target permissions name the same principal. (#7291)
  • Retry bundle file uploads, reads and deletes when the server does not respond within the HTTP timeout ("request timed out after ... of inactivity"). (#7521)
  • Show how to recover when bundle deploy fails because an app's service principal does not exist or was deleted. (#7533)
  • Retry transient server errors when creating Lakebase (postgres_*) resources in the direct deployment engine. (#7557)
  • Fix bundle plan reporting the spark_conf and custom_tags the backend derives from is_single_node: true as drift on clusters. (#7577)
  • Stop planning a spurious update for Lakeflow Connect pipelines whose ingestion_definition.data_staging_options is filled in by the backend. (#7603)
  • Retry the root path check in bundle destroy on transient errors, so a dropped connection no longer fails the destroy. (#7597)
  • Use the Lakebase branch when its create request times out but the branch was created, instead of failing the deploy, in the direct deployment engine. (#7602)
  • Retry transient server errors when deleting resources in the direct deployment engine. (#7708)
  • Use the Lakebase project when its create request times out but the project was created, instead of failing the deploy with "already exists", in the direct deployment engine. (#7747)
  • Use the feature, model serving endpoint or dashboard that a create request created despite timing out, instead of failing the deploy, in the direct deployment engine. (#7757)
  • Use the Lakebase role when its create request fails with a transient error but the role was created, instead of failing the deploy, in the direct deployment engine. (#7767)
  • Use the MCP service, model service or model provider service that a create request created despite failing with a timeout, a 499 or a reset stream, instead of failing the deploy, in the direct deployment engine. Create recovery for features, serving endpoints and dashboards now also covers 499 and reset streams. (#7782)
  • Retry app updates and deletes in the direct deployment engine when they fail with 400 BAD_REQUEST because the app compute is UPDATING, after waiting for the compute to settle. (#7606)
  • Enable drift detection for source_code_path and git_source on apps. (#6399)
  • bundle destroy and bundle deploy no longer fail with "internal error, missing in state" when the state holds an empty-ID entry left by an interrupted recreate in an older CLI. (#7676)
  • Fix a feature whose full_name contains uppercase letters being recreated on every deploy. (#7398)
  • Fix bundle deployment bind (and bundle generate --bind) failing with no such node "resources.internal_immutable_snapshots.immutable" when workspace.sync_mode is immutable, as reported in #7306. (#7329)
  • Retry creating the deployment lock (without --force-lock) when the request fails with a transient server error, times out ("request timed out after ... of inactivity") or loses its connection ("unexpected EOF"). (#7594)
  • Fix a materialization set member whose feature_name contains uppercase letters failing to deploy, or recreating the set on every deploy. (#7398)
  • Delete the old resource when its name follows a renamed schema. (#7718)
  • Fix pipelines dry-run starting a real run when given a job KEY, and a crash in its shell completion. (#7672)
  • Rename schemas in place; schemas and volumes follow a renamed catalog or schema. (#7343)
  • Fix a panic when validating or deploying jobs with local libraries and serverless job clusters. (#7725)
  • Fix tgz artifacts with git set producing an empty tarball or failing when the bundle is in a subdirectory of the git repository. (#7674)
  • Apply presets.trigger_pause_status and the development mode pause default to triggers[] entries on jobs. (#7450)

Dependency Updates