Databricks Connect Java Application

This Java application demonstrates how to connect to Databricks using the Databricks Connect API. It provides multiple ways to configure and establish connections to Databricks clusters, along with examples of Spark operations.

Prerequisites

• Access to a Databricks workspace
• A Databricks cluster or serverless compute
• Databricks personal access token

Setup

1. Clone and Build the Project

git clone <your-repo-url>
cd javadbconnect
mvn clean compile

2. Configure Databricks Connection

You have several options to configure the connection:

Properties File

Update the databricks.properties file with your actual values:

databricks.host=https://your-workspace.cloud.databricks.com
databricks.token=your-personal-access-token
databricks.cluster.id=your-cluster-id

3. Get Your Databricks Connection Details

1. Workspace Hostname: Go to your Databricks workspace URL and copy the hostname (e.g., adb-1234567890123456.7.azuredatabricks.net)

2. Personal Access Token:

   • Go to User Settings → Developer → Access Tokens
   • Click "Generate New Token"
   • Copy the generated token

3. Cluster ID (for classic compute):

   • Go to your cluster in the Databricks workspace
   • Copy the cluster ID from the URL (e.g., 1234-567890-abc123)

4. Compute Sample Used for testing this code:

Running the Application

Build and Run

# Compile the project
mvn clean compile

# Run UserDataExample
java --add-opens=java.base/java.nio=ALL-UNNAMED \
     --add-opens=java.base/sun.misc=ALL-UNNAMED \
     --add-exports=java.base/sun.security.action=ALL-UNNAMED \
     -Dio.netty.tryReflectionSetAccessible=true \
     -cp target/classes:$(mvn dependency:build-classpath -q -Dmdep.outputFile=/dev/stdout) \
     com.example.databricks.UserDataExample

# Run BatchUserStreamingExample (batch processing  10M rows generation)
java --add-opens=java.base/java.nio=ALL-UNNAMED \
     --add-opens=java.base/sun.misc=ALL-UNNAMED \
     --add-exports=java.base/sun.security.action=ALL-UNNAMED \
     -Dio.netty.tryReflectionSetAccessible=true \
     -cp target/classes:$(mvn dependency:build-classpath -q -Dmdep.outputFile=/dev/stdout) \
     com.example.databricks.BatchUserStreamingExample

References

• Databricks Connect Documentation

License

This project is provided as an example and is not officially supported by Databricks.