Enhancement: Spice Java SDK

[sgrebnov]

Goal-State/What/Result

Developers can call Spice OSS through a Spice Java SDK from Java apps.

Why/Purpose

Java is a large developer base and developers in Java currently can not call Spice OSS through official SDKs.

By When

July 12, 2024 5:00pm PST

Issue/Spec written and reviewed: July 15
Done-Done: July 15

Done-Done

• First Principles Driven
• The Algorithm
• PM/Design Review
• UX Review
• Threat Model / Security Review
• Tests
• Documentation
• Samples/Quickstarts/Tutorials
• Release Notes

The Algorithm

• Every requirement questioned?
• Delete (Scope) any part you can.
• Simplify.
• Break down into smaller iterations/milestones.
• Opportunities for automation.

Spec

SDK API

• Supported/tested Java versions:
  OpenJDK 21 LTS
  OpenJDK 17 LTS
  OracleJDK 21 LTS
  OracleJDK 17 LTS
  OracleJDK 22 (latest Oracle)

• SDK API follows Java builder pattern to initialize the client and supports Spice OSS and Spice Cloud Platform scenarios.

Client initialization for Spice OSS

 SpiceClient client = SpiceClient.builder() // Flight endpoint is set to http://localhost:50051
     .build();

Client initialization for Spice Cloud Platform

 SpiceClient client = SpiceClient.builder() //  Flight endpoint is set to https://flight.spiceai.io:443
     .withApiKey(API_KEY)
     .withSpiceCloud()
     .build();

• Support specifying flight url during client initialization (also viaSPICE_FLIGHT_URL ENV variable):

 SpiceClient client = SpiceClient.builder()
      .withFlightAddress(new URI("grpc+tcp://ip-address:50051"))
     .build();

• Query API

// see above for client initialization

FlightStream stream = client.query("SELECT * FROM eth.recent_blocks LIMIT 10;");

while (stream.next()) {
    System.out.println(stream.getRoot().contentToTSVString());
}

• Exceptions - SDK uses standard IllegalArgumentException, etc and Arrow Flight exceptions. Custom exceptions must be used only in cases where the standard exceptions are not descriptive (to be defined during implementation, if any such cases).

• Implements connection retry mechanism (configurable, 3 attempts by default). Retries are performed for connection and system internal errors.

CI and Publishing

• GH Action based E2E test on PR
• GH Action based workflow to build and publish SDK on Maven Central
• Spice SDK is published to Maven Central and can be used by Maven and Gradle apps.

<dependency>
  <groupId>ai.spice</groupId>
  <artifactId>spiceai</artifactId>
  <version>version</version>
  <scope>compile</scope>
</dependency>

Documentation

Covers both Maven and Gradle

• https://docs.spiceai.org/sdks
• Readme.md in the repo
• docs.spice.ai
• Samples: https://github.com/spiceai/samples/tree/trunk/client-sdk

How/MAP

1. Implement SDK functionality and tests using Maven.
2. Implement SDK publishing to Maven Central
3. Publish to Maven Central
4. Create documentation (including Gradle examples - supports distributed via a Maven repository)

Implementation (issues)

1. Java SDK core functionality and tests #1
2. Add publishing to Maven Central #2
3. Implement connection retry mechanism #5
4. Implement AutoCloseable interface for SpiceClient #7
5. Document Spice Java SDK docs#335
6. Add Java SDK sample for Maven and Gradle users samples#91

QA

1. Run tests against Local Spice and SCP
2. Create and test Gradle and Maven apps after SDK publishing to Maven.

[sgrebnov]

Completed