Skip to content

Latest commit

 

History

History
98 lines (67 loc) · 45.7 KB

File metadata and controls

98 lines (67 loc) · 45.7 KB

Databricks Workspace Conf API

Introduction

Databricks has an api that allows you to configure your workspace features. So that you don't have to login into your workspace admin control panel and manually edit them.

Sadly these properties appear to not be documented anywhere offically.

Below is my rough mapping out of these features, along with how to use this API called Workspace-conf. Please do feel free to add to the list.

Using the API

The API behaves the same as the other REST APIS. You should authenticate the same.

The endpoint for workspace-conf is as follows https://.azuredatabricks.net/api/2.0/workspace-conf

You must use PATCH not POST or PUT!

You must include the values below in your body with a property true or false value which must be a string e.g. {"enforceWorkspaceViewAcls":"true"}

Databricks Workspace features you can configure via this API

image

Properties

Feature Description Values Comment Daily Databricks Tweet
Workspace Visibiliy Control Enabling Workspace Access Control will allow users to control who can view, edit, and run notebooks in their workspace.When workspace access control is enabled, items in each user's home directory in /Users will become private.

Existing top-level items will remain shared with all users, but new top-level items will be private by default.

When workspace access control is disabled, all items in the workspace will be accessible to all users.
enableAclsConfig https://docs.microsoft.com/en-gb/azure/databricks/administration-guide/access-control/workspace-acl
Cluster Visibility Control Enabling cluster access control will allow users to control who can attach to, restart, and manage (resize/delete) clusters that they create. It will also allow administrators to control which users have permissions to create clusters. Enabling pool access control will allow users to control who can attach to and manage the pools that they create. Enabling jobs access control allows owners of a job to control who can view job results or manage runs of a job (run now/cancel)

When cluster access control is enabled, admins will still have attach, restart and manage permissions on existing clusters, as well as the ability to create clusters.

When cluster access control is disabled, all users will have permission to create clusters, as well as attach to, restart, and manage existing clusters.

When pool access control is enabled, admins will still have attach and manage permissions on existing pools, as well as the ability to create pools.

When pool access control is disabled, all users will have permission to create pools, as well as attach to and manage existing pools.
enforceClusterViewAcls https://docs.microsoft.com/azure/databricks/administration-guide/access-control/cluster-acl
enableTokensConfig Enable or disable personal access tokens for this workspace

Maximum token lifetime of new tokens in days, as an integer. If zero, new tokens are permitted to have no lifetime limit. Negative numbers are unsupported. WARNING: This limit only applies to new tokens, so there may be tokens with lifetimes longer than this value, including unlimited lifetime. Such tokens may have been created before the current maximum token lifetime was set.
enableTokensConfig , maxTokenLifetimeDays https://docs.databricks.com/dev-tools/api/latest/token-management.html#operation/set-configuration 
Job Visibility Control: Prevent users from seeing jobs that they do not have access to. enableJobViewAcls https://docs.microsoft.com/azure/databricks/administration-guide/access-control/jobs-acl#jobs-visibility
Databricks Runtime for Genomics: The Databricks Runtime for Genomics is a variant of Databricks Runtime optimized for working with genomic data. Enabling the Databricks Runtime for Genomics allows users in your workspace to select it when configuring a cluster.

After enabling or disabling the Databricks Runtime for Genomics, you must refresh the page for the change to take effect.
enableHlsRuntime https://docs.microsoft.com/azure/databricks/runtime/genomicsruntime
Container Services: Databricks Container Services allows users in your workspace to specify a Docker image when creating clusters.

Note: after enabling or disabling Container Services, you must refresh the page for the change to take effect
enableDcs https://docs.microsoft.com/azure/databricks/clusters/custom-containers
Repos: Enable or disable Repos. You should see a new Repos icon in your workspace's left navigation when this feature is enabled. enableProjectTypeInWorkspace https://docs.microsoft.com/azure/databricks/repos
Files in Repos: DBR 11.0+ Enable or disable the Files in Repos feature.

When Files in Repos is set to 'DBR 8.4+', arbitrary files will be included in Repo operations and can be accessed from clusters running DBR 8.4 and above.

When Files in Repos is set to 'DBR 11.0+', arbitrary files will be included in Repo operations and can be accessed from clusters running DBR 11.0 and above.

When Files in Repos is disabled, arbitrary files will not be included in Repo operations and cannot be accessed from clusters
enableWorkspaceFilesystem : "true"}, {enableWorkspaceFilesystem: "dbr11.0+"}- cann accept true,false or this?
Enable Repos Git URL Allow List: Enable or disable restricting commit and push operations in Repos to a configurable allow list. The allow list will be empty by default. You should configure the allow list in the panel below. enableProjectsAllowList
Repos Git URL Allow LIST Configure a comma separated list of url prefixes that will be used to restrict the Git repositories users can commit and push to in Repos. Users will only be able to commit and push to Git repositories that start with one of the URL prefixes defined here.

For example, you can set this configuration to "https://foo,https://bar" and users will only be able to commit and push to Git repositories with a url that starts with "https://foo" or "https://bar". If you have not set a value or if you save an empty input, you should see "Empty list". This is equivalent to restricting all repositories. Allow all repositories by disabling the feature in the panel above.

Note that the list you save will overwrite the existing set of saved URL prefixes.
projectsAllowList comma seperated urls https://foo.bar,https://fun.vom
Third-party iFraming prevention Sending the “X-Frame-Options: sameorigin” response header prevents third-party domains from iframing Databricks. enable-X-Frame-Options https://developer.mozilla.org/docs/Web/HTTP/Headers/X-Frame-Options
MIME type sniffing prevention: Sending the “X-Content-Type-Options: nosniff” response header instructs browsers not to perform MIME type sniffing. enable-X-Content-Type-Options https://developer.mozilla.org/docs/Web/HTTP/Headers/X-Content-Type-Options
XSS attack page rendering prevention: Sending the “X-XSS-Protection: 1; mode=block” response header instructs browsers to prevent page rendering if an attack is detected. enable-X-XSS-Protection https://developer.mozilla.org/docs/Web/HTTP/Headers/X-XSS-Protection
Download button for notebook results Enable or disable the download button for notebook results. enableResultsDownloading
Upload data using the UI: Enable or disable uploading data to Databricks File System (DBFS) directly from the homepage, the Data tab, and the File menu in a notebook. enableUploadDataUis https://docs.microsoft.com/azure/databricks/administration-guide/workspace/dbfs-ui-upload
Notebook Exporting Enable or disable exporting notebooks and cells within notebooks. enableExportNotebook
Notebook Git Versioning: Enable or disable git versioning for notebooks. enableNotebookGitVersioning
Notebook Table Clipboard Features: Enable or disable the ability of users to copy tabular data to the clipboard via the Notebooks UI. enableNotebookTableClipboard
Web Terminal: Enable or disable web terminal for clusters. enableWebTerminal https://docs.microsoft.com/azure/databricks/administration-guide/clusters/web-terminal
DBFS File Browser: Enable or disable DBFS File Browser enableDbfsFileBrowser
Databricks Autologging Enable or disable Databricks Autologging for this workspace. When enabled, ML model training runs executed interactively on clusters with supported versions of the Databricks Runtime for Machine Learning will automatically be logged to MLflow. enableDatabricksAutologgingAdminConf https://docs.microsoft.com/azure/databricks/applications/mlflow/databricks-autologging
MLflow Run Artifact Download: Enable or disable the downloading of artifacts logged to an MLflow run. They will still be viewable in the UI. mlflowRunArtifactDownloadEnabled
MLflow Classic Model Serving Endpoint Creation: Enable or disable Classic model serving for this workspace. Disabling this option will not disable the existing model serving endpoints. mlflowModelServingEndpointCreationEnabled
MLflow Model Registry Email Notifications: Enable or disable model registry email notifications for this workspace. mlflowModelRegistryEmailNotificationsEnabled
RStudio Home Directory: This value sets the default base directory for RStudio Server users' home in the form of absolute unix path. This configuration applies to all clusters running RStudio Server under the workspace.

Any existing directory can be used as the base for home directories. If the directory does not exist, it will be created upon the first user login.

After updating this configuration, existing clusters with RStudio Server users must be restarted to use the updated home director
rStudioUserDefaultHomeBase : "/SCOTT"
Store Interactive Notebook Results in Customer Account: When enabled, all interactive notebook results are stored in the customer account. storeInteractiveNotebookResultsInCustomerAccount
Verbose Audit Logs: Enable or disable verbose audit logs. enableVerboseAuditLogs
Enforce User Isolation enforceUserIsolation
New template for Jobs Email Notifications enableJobsEmailsV2
Allow Repos to Export IPYNB outputs reposIpynbResultsExportPermissions
Increased number of jobs {jobsListBackendPaginationEnabled: "true", jobsListBackendPaginationOptOut: "false"}
FileStore Endpoint enableFileStoreEndpoint

If you paste this Markdown into a platform that supports it (like

Sample request

image

image

--header 'Authorization: Bearer <PAT TOKEN>' \
--header 'Content-Type: application/json' \
--data-raw '{"enforceWorkspaceViewAcls":"true"}'

Using Terraform

You may use Terraform as your IaC approach. The Offical databricks provider has a workspace_conf resource where you can specfify the above parameters.

resource "databricks_workspace_conf" "this" {
  custom_config = {
    "enableIpAccessLists" : true,
     "enableWebTerminal" : true,
     	"enableJobsEmailsV2": true
  }
}