Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Setup: add skeleton #2

Merged
merged 7 commits into from
Oct 3, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 54 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -1 +1,55 @@
# xdb-java-sdk

[![Coverage Status](https://codecov.io/github/xdblab/xdb-java-sdk/coverage.svg?branch=main)](https://app.codecov.io/gh/xdblab/xdb-java-sdk/branch/main)
[![Build status](https://github.com/xdblab/xdb-java-sdk/actions/workflows/ci-test.yml/badge.svg?branch=main)](https://github.com/xdblab/xdb-java-sdk/actions/workflows/ci-test.yml)

Java SDK for [xdb](https://github.com/xdblab/xdb)

## Requirements

- Java 1.8+

# Development Plan

## 1.0

- [ ] StartProcessExecution API
- [ ] Basic
- [ ] ProcessIdReusePolicy
- [ ] Process timeout
- [ ] Retention policy after closed
- [ ] Executing `wait_until`/`execute` APIs
- [] Basic
- [] Parallel execution of multiple states
- [ ] StateOption: WaitUntil/Execute API timeout and retry policy
- [ ] AsyncState failure policy for recovery
- [ ] StateDecision
- [ ] Single next State
- [ ] Multiple next states
- [ ] Force completing process
- [ ] Graceful completing process
- [ ] Force fail process
- [ ] Dead end
- [ ] Conditional complete process with checking queue emptiness
- [ ] Commands
- [ ] AnyOfCompletion and AllOfCompletion waitingType
- [ ] TimerCommand
- [ ] LocalQueue
- [ ] LocalQueueCommand
- [ ] MessageId for deduplication
- [ ] SendMessage API without RPC
- [ ] LocalAttribute persistence
- [ ] LoadingPolicy (attribute selection + locking)
- [ ] InitialUpsert
- [ ] GlobalAttribute persistence
- [ ] LoadingPolicy (attribute selection + locking)
- [ ] InitialUpsert
- [ ] Multi-tables
- [ ] RPC
- [ ] API error handling for canceled, failed, timeout, terminated
- [ ] StopProcessExecution API
- [ ] WaitForStateCompletion API
- [ ] ResetStateExecution for operation
- [ ] DescribeProcessExecution API
- [ ] WaitForProcessCompletion API
- [ ] History events for operation/debugging
46 changes: 0 additions & 46 deletions src/main/java/io/xdb/core/Client.java

This file was deleted.

14 changes: 0 additions & 14 deletions src/main/java/io/xdb/core/ClientOptions.java

This file was deleted.

98 changes: 98 additions & 0 deletions src/main/java/io/xdb/core/client/BasicClient.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
package io.xdb.core.client;

import feign.Feign;
import feign.FeignException;
import feign.Retryer;
import io.xdb.core.ServerErrorDecoder;
import io.xdb.core.exception.XDBHttpException;
import io.xdb.core.process.BasicClientProcessOptions;
import io.xdb.core.process.ProcessOptions;
import io.xdb.gen.api.ApiClient;
import io.xdb.gen.api.DefaultApi;
import io.xdb.gen.models.ProcessExecutionDescribeRequest;
import io.xdb.gen.models.ProcessExecutionDescribeResponse;
import io.xdb.gen.models.ProcessExecutionStartRequest;
import io.xdb.gen.models.ProcessExecutionStartResponse;
import io.xdb.gen.models.ProcessStartConfig;

/**
* {@link BasicClient} serves as a foundational client without a process {@link io.xdb.core.registry}.
* It represents the internal implementation of the {@link Client}.
* However, it can also be utilized directly if there is a compelling reason, allowing you to invoke APIs on the xdb server with no type validation checks, such as process type, queue names, and so forth.
*/
public class BasicClient {

private final ClientOptions clientOptions;
private final DefaultApi defaultApi;

public BasicClient(final ClientOptions clientOptions) {
this.clientOptions = clientOptions;
this.defaultApi = buildDefaultApi();
}

public String startProcess(
final String processType,
final String processId,
final String startStateId,
final Object input,
final BasicClientProcessOptions processOptions
) {
final ProcessExecutionStartRequest request = new ProcessExecutionStartRequest()
.processType(processType)
.processId(processId)
.workerUrl(clientOptions.getWorkerUrl())
.startStateId(startStateId)
.startStateInput(clientOptions.getObjectEncoder().encode(input));

if (processOptions.getProcessOptionsOptional().isPresent()) {
final ProcessOptions options = processOptions.getProcessOptionsOptional().get();
request.processStartConfig(
new ProcessStartConfig()
.idReusePolicy(options.getProcessIdReusePolicy())
.timeoutSeconds(options.getTimeoutSeconds())
);
}

if (processOptions.getStartStateConfig().isPresent()) {
request.startStateConfig(processOptions.getStartStateConfig().get());
}

final ProcessExecutionStartResponse response;
try {
response = defaultApi.apiV1XdbServiceProcessExecutionStartPost(request);
} catch (final FeignException.FeignClientException e) {
throw XDBHttpException.fromFeignException(clientOptions.getObjectEncoder(), e);
}

return response.getProcessExecutionId();
}

public ProcessExecutionDescribeResponse describeCurrentProcessExecution(final String processId) {
final ProcessExecutionDescribeRequest request = new ProcessExecutionDescribeRequest().processId(processId);

try {
return defaultApi.apiV1XdbServiceProcessExecutionDescribePost(request);
} catch (final FeignException.FeignClientException e) {
throw XDBHttpException.fromFeignException(clientOptions.getObjectEncoder(), e);
}
}

private DefaultApi buildDefaultApi() {
final ApiClient apiClient = new ApiClient().setBasePath(clientOptions.getServerUrl());
apiClient.setObjectMapper(clientOptions.getObjectEncoder().getObjectMapper());

final Feign.Builder feignBuilder = apiClient.getFeignBuilder();
final ServerApiRetryConfig apiRetryConfig = clientOptions.getServerApiRetryConfig();
feignBuilder.retryer(
new Retryer.Default(
apiRetryConfig.getInitialIntervalMills(),
apiRetryConfig.getMaximumIntervalMills(),
apiRetryConfig.getMaximumAttempts()
)
);
feignBuilder.errorDecoder(new ServerErrorDecoder());
apiClient.setFeignBuilder(feignBuilder);

return apiClient.buildClient(DefaultApi.class);
}
}
70 changes: 70 additions & 0 deletions src/main/java/io/xdb/core/client/Client.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
package io.xdb.core.client;

import io.xdb.core.process.BasicClientProcessOptions;
import io.xdb.core.process.Process;
import io.xdb.core.process.ProcessOptions;
import io.xdb.core.registry.Registry;
import io.xdb.core.state.AsyncState;
import io.xdb.core.utils.ProcessUtil;
import io.xdb.gen.models.AsyncStateConfig;
import java.util.Optional;

public class Client {

private final Registry registry;
private final ClientOptions clientOptions;

private final BasicClient basicClient;

public Client(final Registry registry, final ClientOptions clientOptions) {
this.registry = registry;
this.clientOptions = clientOptions;
this.basicClient = new BasicClient(clientOptions);
}

public String startProcess(final Process process, final String processId, final Object input) {
final String processType = ProcessUtil.getProcessType(process);
return startProcessInternal(processType, processId, input, process.getOptions());
}

/**
* Caution: if you intend to override certain process options, utilize the {@link Client#startProcess(Process, String, Object)} method
*
* @param processClass
* @param processId
* @param input
* @return
*/
public String startProcess(
final Class<? extends Process> processClass,
final String processId,
final Object input
) {
final String processType = ProcessUtil.getProcessType(processClass);
return startProcessInternal(processType, processId, input, null);
}

private String startProcessInternal(
final String processType,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be better to use Process processDefinition to make it more "strongly typing experience".

Copy link
Contributor

@longquanzheng longquanzheng Oct 2, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry 😢 I forgot this. In Java it’s even better to just use the class type like this https://github.com/indeedeng/iwf-java-sdk/blob/d2ae8439d4b7ed996c20303be57f248eb00fe4ef/src/main/java/io/iworkflow/core/Client.java#L63 override process/workflow type is really rare and they can probably use basic client instead

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think users will be confused when they define type in the options but pass the class in the startProcess method, and find the customized type is not working.

So let's keep using the definition here to avoid such confusion.

Copy link
Contributor

@longquanzheng longquanzheng Oct 2, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can provide both the string processType and the Class like iwf-java-sdk.

Or even just not allow customizing processType for now. It's really not useful so far. In our new python SDK, I just don't expose the workflowType customization: https://github.com/indeedeng/iwf-python-sdk/blob/main/iwf/workflow.py#L8 And it's not late to add it later when needed.

On the other side, using Class is much more friendly than using the workflow instance, it save the user code for dependency injection. As a result, I deleted this one in Java SDK recently: indeedeng/iwf-java-sdk#195 (still keep the string WorkflowType)

final String processId,
final Object input,
final ProcessOptions processOptions
) {
AsyncStateConfig asyncStateConfig = null;
String startingStateId = "";

final Optional<AsyncState> startingState = registry.getProcessStartingState(processType);
if (startingState.isPresent()) {
asyncStateConfig =
new AsyncStateConfig().skipWaitUntil(AsyncState.shouldSkipWaitUntil(startingState.get()));
startingStateId = ProcessUtil.getStateId(startingState.get());
}

final BasicClientProcessOptions basicClientProcessOptions = new BasicClientProcessOptions(
processOptions,
asyncStateConfig
);

return basicClient.startProcess(processType, processId, startingStateId, input, basicClientProcessOptions);
}
}
26 changes: 26 additions & 0 deletions src/main/java/io/xdb/core/client/ClientOptions.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
package io.xdb.core.client;

import io.xdb.core.encoder.JacksonJsonObjectEncoder;
import io.xdb.core.encoder.ObjectEncoder;
import lombok.Builder;
import lombok.Getter;

@Builder
@Getter
public class ClientOptions {

private final String serverUrl;
private final String workerUrl;
private final ObjectEncoder objectEncoder;
private final ServerApiRetryConfig serverApiRetryConfig;

public static ClientOptions getDefaultLocal() {
return ClientOptions
.builder()
.serverUrl("http://localhost:8801")
.workerUrl("http://localhost:8802")
.objectEncoder(new JacksonJsonObjectEncoder())
.serverApiRetryConfig(ServerApiRetryConfig.getDefault())
.build();
}
}
22 changes: 22 additions & 0 deletions src/main/java/io/xdb/core/client/ServerApiRetryConfig.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
package io.xdb.core.client;

import lombok.Builder;
import lombok.Getter;

@Builder
@Getter
public class ServerApiRetryConfig {

private long initialIntervalMills;
private long maximumIntervalMills;
private int maximumAttempts;

public static ServerApiRetryConfig getDefault() {
return ServerApiRetryConfig
.builder()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, looks like lombok is better/friendly than immutable 👍 I had to manually add those builder helpers myself when using immutable

.initialIntervalMills(100)
.maximumIntervalMills(1000)
.maximumAttempts(10)
.build();
}
}
Loading