Self-Testing Partner CLI

The Self-Testing Partner CLI allows partners to self-test the E2E flow with their integration before fully onboarding. The CLI will create a Partner Connect tile visible only in the partner's workspace.

The APIs called by the CLIs can be called directly as well. (See API Specifications)

Before using the CLI, please discuss with the Databricks team about whether your product is a fit for Partner Connect.

Limitations

Only one self-testing partner can be active at a time in a workspace.
The Partner Connect tile will have a placeholder description and icon indicating that the tile is for self-testing and visible only within the partner's workspace.
- These values can be specified when the partner fully onboards.
The CLI is not compatible with connection file/desktop application based partners (partners which require downloading a connection file to use with a locally-installed application).
The Spark version and instance type for interactive clusters cannot be specified for self-testing partners.

Usage

Set up the prerequisite requirements
Configure the CLI
Build and run the CLI
Set up authorization

Requirements

Maven
- Set up Maven following these instructions.
- Maven is not required if you choose to use API directly
Databricks CLI
- The Databricks Secrets CLI is used to securely store endpoint credentials for the self-testing partner.
- The Authentication setup can be reused for the Self-Testing Partner CLI configuration.
- This CLI is optional as the underlying API can be called directly. (See Authorization)

Configuration

There are two configuration options: Configuration profiles and environment variables.

Both options use personal access tokens. The Self-Testing Partner CLI does not support using a Databricks username and password for authentication.

Option 1: Configuration Profiles (recommend)

This option consists of storing the host name (workspace URL) and token in a configuration profiles file.

Following the Databricks Configuration Profiles documentation, set up a configuration profiles file using your personal access token. Profile names other than [DEFAULT] are also supported.

If the DATABRICKS_CONFIG_FILE environment variable is set, the CLI will use this path instead of the default path (~/.databrickscfg). DATABRICKS_CONFIG_FILE must be a full path.

Option 2: Environment Variables

This option consists of storing the host (workspace URL with the format https://<instance-name>.cloud.databricks.com) and personal access token in the DATABRICKS_HOST and DATABRICKS_TOKEN environment variables respectively.

Example

export DATABRICKS_HOST = https://...databricks.com
export DATABRICKS_TOKEN = dapi...

Run the CLI

The CLI uses Maven to create an executable Jar file.

cd into the self-testing-partner-cli directory.
Run mvn clean package. The location of the jar will be outputted.
Run the jar with your specified subcommand and args. (e.g. java -jar target/self-testing-partner-cli.jar create -f ./sampleConfig.json)

Config Specifications

All fields but delete_url and supported_features are required.

Example

{
  "partner": {
    "name": "Test Partner",
    "category": "DATA_PREP_AND_TRANSFORM",
    "connect_url": "https://databricks.com/read/connect",
    "sign_up_url": "https://databricks.com/sign-up",
    "delete_url": "https://databricks.com/delete"
    "terms_of_service_url": "https://databricks.com/tos",
    "privacy_policy_url": "https://databricks.com/privacy",
    "hostname": "databricks.com",
    "supported_features": ["UNITY_CATALOG"],
    "integration_type": "WAREHOUSE_READ"
  }
}

Supported Features

supported_features is an array containing any of [UNITY_CATALOG, NON_ADMIN, EXTERNAL_LOCATION] and can be empty. EXTERNAL_LOCATION only applies to partners who are writing data to Databricks. Before using the NON_ADMIN feature, contact your SA to discuss non-admin integrations.

Integration Type

integration_type must be one of [NO_INTEGRATION, CLUSTER_COMPUTE, WAREHOUSE_READ, WAREHOUSE_READ_WRITE, WAREHOUSE_WRITE, WORKSPACE_MANAGEMENT].

Use the below table to determine the integration type for the self-testing partner. If you have further requirements or questions, reach out to your SA.

Integration Type	Description
NO_INTEGRATION	No Databricks service principal needed
CLUSTER_COMPUTE	Commonly used by Notebook Partners. Databricks provides the partner an Interactive Cluster and personal access token (PAT) with permissions. The partner can use the PAT to publish a Databricks Notebook and configure the Interactive Cluster.
WAREHOUSE_READ	Used by partners that only need to read data from the Lakehouse. In Partner Connect, the user selects which data to grant access to your product. Databricks provides the partner a SQL Warehouse and PAT with permissions to query that data. This is often used by Business Intelligence and Data Quality partners
WAREHOUSE_READ_WRITE	Used by partners that need to read from and write to the Lakehouse. In Partner Connect, the user selects which catalog to grant write access and which schemas to grant read access for your product. Databricks provides the partner a SQL Warehouse and PAT with permissions to create schemas and tables in that catalog, as well as query the selected data. This is often used by Data Preparation partners.
WAREHOUSE_WRITE	Used by partners that only need to write (ingest) data into the Lakehouse. In Partner Connect, the user selects which catalog to grant write access to your product. Databricks provides the partner a SQL Warehouse and PAT with permissions to create schemas and tables in that catalog. This is often used by Ingestion partners.
WORKSPACE_MANAGEMENT	Used by partners that need to create clusters and jobs. Databricks provides the partner with a PAT which can be used to call Databricks REST APIs on clusters in the workspace. This does not give the partner access to Databricks SQL warehouses.

Usage

java -jar target/self-testing-partner-cli.jar [subcommand] [flags]

Help

Run java -jar target/self-testing-partner-cli.jar --help to view all commands and flags.

Common Flags

Name	Short Name	Description	Argument Type	Default Value For Argument
--env	-e	If present, DATABRICKS_HOST and DATABRICKS_TOKEN will be used instead of a Databricks config file.	-	-
--profile	-p	Specifies the name of the profile to be used without the surrounding square brackets (e.g. `SELF-TESTING-CLI`). This flag is for use with configuration profiles files and will be ignored if `--env` is set.	String	DEFAULT

If both flags are specified, --env takes precedence.

Examples

Use environment variables instead of a Databricks configuration profile file

java -jar target/self-testing-partner-cli.jar create -f ./sampleConfig.json -e

java -jar target/self-testing-partner-cli.jar delete --env

Use a different profile name than DEFAULT

java -jar target/self-testing-partner-cli.jar delete -p "SELF-TESTING-CLI"

java -jar target/self-testing-partner-cli.jar create -f ./sampleConfig.json --profile "SELF-TESTING-CLI"

Create

Create takes in the path to a json config file which contains the fields used to set up the self-testing partner and returns the ID of the created partner.

The created self-testing partner will be visible in Partner Connect only for the workspace configured in the Configuration step.

Example Tile

Authorization

After creating a self-testing partner, set up authorization.

The self-testing partner cannot be used without setting up authorization.

Flags

Name	Short Name	Description	Argument Type	Default Value For Argument
file	-f	The path to the config file for the self-testing partner (e.g. `~/sampleConfig.json`, `./sampleConfig.json`)	String	-

Example

java -jar target/self-testing-partner-cli.jar create -f ./sampleConfig.json

Get

The Get command returns the IDs and names of active self-testing partners in the workspace.

Flags

No additional flags

Example

java -jar target/self-testing-partner-cli.jar get

Update

The Update command takes in the path to a json config file and updates the self-testing partner. Any existing connections will be deleted.

Flags

Name	Short Name	Description	Argument Type	Default Value For Argument
file	-f	The path to the config file for the self-testing partner (e.g. `~/sampleConfig.json`, `./sampleConfig.json`)	String	-

The format of the config file is the same as for Create.

Example

java -jar target/self-testing-partner-cli.jar update -f ./config-v2.json

Delete

The Delete command will delete the active self-testing partner if there is one. Any existing connections will be deleted.

Flags

No additional flags

Example

java -jar target/self-testing-partner-cli.jar delete

Tests

Run mvn test to run unit tests.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

README.md

README.md

Self-Testing Partner CLI

Limitations

Usage

Requirements

Configuration

Option 1: Configuration Profiles (recommend)

Option 2: Environment Variables

Example

Run the CLI

Config Specifications

Example

Category

Supported Features

Integration Type

Usage

Help

Common Flags

Examples

Use environment variables instead of a Databricks configuration profile file

Use a different profile name than DEFAULT

Create

Example Tile

Authorization

Flags

Example

Get

Flags

Example

Update

Flags

Example

Delete

Flags

Example

Tests

Files

README.md

Latest commit

History

README.md

File metadata and controls

Self-Testing Partner CLI

Limitations

Usage

Requirements

Configuration

Option 1: Configuration Profiles (recommend)

Option 2: Environment Variables

Example

Run the CLI

Config Specifications

Example

Category

Supported Features

Integration Type

Usage

Help

Common Flags

Examples

Use environment variables instead of a Databricks configuration profile file

Use a different profile name than DEFAULT

Create

Example Tile

Authorization

Flags

Example

Get

Flags

Example

Update

Flags

Example

Delete

Flags

Example

Tests