Skip to content
This repository has been archived by the owner on Dec 13, 2023. It is now read-only.

doc updates.md #3364

Merged
merged 3 commits into from
Jan 23, 2023
Merged

doc updates.md #3364

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
ab8805f
Update workflow_sdk.md
RizaFarheen Nov 29, 2022
983e8bb
fixed 404 link and some minor updates
RizaFarheen Nov 30, 2022
48a637b
content update
RizaFarheen Nov 30, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
30 changes: 15 additions & 15 deletions java-sdk/testing_framework.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,22 +1,22 @@
# Unit Testing framework for workflows
# Unit Testing Framework for Workflows

The framework allows you to test the workflow definitions against a specific version of Conductor server.

The unit tests allows the following:
1. **Input/Output Wiring**: Ensure the tasks are wired up correctly
2. **Parameter check**: Workflow behavior with missing mandatory parameters is expected (fail if required)
3. **Task Failure behavior**: Ensure the task definitions have right no. of retries etc.
e.g. If the task is not idempotent, it does not get retried.
4. **Branch Testing**: Given a specific input, ensure the workflow executes specific branch of the fork/decision.
The unit tests allow the following:
1. **Input/Output Wiring**: Ensure the tasks are wired up correctly.
2. **Parameter check**: Workflow behavior with missing mandatory parameters is expected (fail if required).
3. **Task Failure behavior**: Ensure the task definitions have the right number of retries etc.
For example, if the task is not idempotent, it does not get retried.
4. **Branch Testing**: Given a specific input, ensure the workflow executes a specific branch of the fork/decision.

The local test server is self-contained with no additional dependencies required and stores all the data
in memory. Once the tests complete, the server is terminated and all the data is wiped out.
in memory. Once the test completes, the server is terminated and all the data is wiped out.

## Unit Testing frameworks
The testing framework is agnostic to the framework you use for testing, can be easily integrated into
JUnit, Spock and other testing framework being used.
## Unit Testing Frameworks
The unit testing framework is agnostic to the framework you use for testing and can be easily integrated into
JUnit, Spock and other testing frameworks being used.

## Setting up the local server for testing
## Setting Up Local Server for Testing​

```java
//Setup method code - should be called once per the test lifecycle
Expand All @@ -33,15 +33,15 @@ testRunner.init("com.netflix.conductor.testing.workflows");
executor = testRunner.getWorkflowExecutor();
```

Clean up method
Clean up method:
```java
//Clean up method code -- place in a clean up method e.g. @AfterClass in Junit

//Shutdown local workers and server and clean up any local resources in use.
//Shutdown local workers and servers and clean up any local resources in use.
testRunner.shutdown();
```

Loading workflows from JSON files for testing
Loading workflows from JSON files for testing:
```java
executor.loadTaskDefs("/tasks.json");
executor.loadWorkflowDefs("/simple_workflow.json");
Expand Down
37 changes: 19 additions & 18 deletions java-sdk/worker_sdk.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
# Worker SDK
Worker SDK makes it easy to write conductor workers which are strongly typed with specific inputs and outputs.
Worker SDK makes it easy to write Conductor workers which are strongly typed with specific inputs and outputs.

Annotations for the worker methods:

* `@WorkerTask` When annotated converts a method to a conductor worker
* `@InputParam` name of the input parameter to bind to from the task's input
* `@OutputParam` name of the output key of the task's output.
* `@WorkerTask` - When annotated, convert a method to a Conductor worker.
* `@InputParam` - Name of the input parameter to bind to from the task's input.
* `@OutputParam` - Name of the output key of the task's output.

Please note, inputs and outputs to a task in Conductor are JSON documents.
Please note inputs and outputs to a task in Conductor are JSON documents.


**Examples**
Expand All @@ -21,7 +21,8 @@ Create a worker named `task1` that gets Task as input and produces TaskResult as
}
```

Create a worker named `task2` that takes `name` as a String input and produces a
Create a worker named `task2` that takes the `name` as a String input and produces an output `return "Hello, " + name`

```java
@WorkerTask("task2")
public @OutputParam("greetings") String task2(@InputParam("name") String name) {
Expand All @@ -43,7 +44,7 @@ Output:
"greetings": "Hello, conductor"
}
```
A worker that takes complex java type as input and produces complex output:
A worker that takes complex java type as input and produces the complex output:
```java
@WorkerTask("get_insurance_quote")
public InsuranceQuote getInsuranceQuote(GetInsuranceQuote quoteInput) {
Expand Down Expand Up @@ -74,39 +75,39 @@ Output:
```

## Managing Task Workers
Annotated Workers are managed by [WorkflowExecutor](https://github.com/netflix/conductor/java-sdk/src/main/java/com/netflix/conductor/sdk/workflow/executor/WorkflowExecutor.java)
Annotated Workers are managed by [WorkflowExecutor](https://github.com/Netflix/conductor/blob/main/java-sdk/src/main/java/com/netflix/conductor/sdk/workflow/executor/WorkflowExecutor.java)

### Start Workers
```java
WorkflowExecutor executor = new WorkflowExecutor("http://server/api/");
//List of packages (comma separated) to scan for annotated workers.
// Please note,the worker method MUST be public and the class in which they are defined
// Please note, the worker method MUST be public and the class in which they are defined
//MUST have a no-args constructor
executor.initWorkers("com.company.package1,com.company.package2");
```

### Stop Workers
Code fragment to stop workers at the shutdown of the application
The code fragment to stop workers at shutdown of the application.
```java
executor.shutdown();
```

### Unit Testing Workers
Workers implemented with the annotations are regular Java methods can be united tested with any testing framework.
Workers implemented with the annotations are regular Java methods that can be unit tested with any testing framework.

#### Mock workers for workflow testing
Create a mock worker in a different package (e.g. test) and scan for these packages when loading up the workers for integration testing.
#### Mock Workers for Workflow Testing​
Create a mock worker in a different package (e.g., test) and scan for these packages when loading up the workers for integration testing.

See [Unit Testing Framework](testing_framework.md) for more details on testing.

## Best Practices
In a typical production environment, you will have multiple workers across different machines/VMs/pods polling for the same task.
As with all the Conductor workers, the following best practices applies:
As with all Conductor workers, the following best practices apply:

1. Workers should be stateless and should not maintain any state on the process they are running
2. Ideally workers should be idempotent
3. Worker should follow Single Responsibility Principle and do exactly one thing they are responsible for
4. Worker should not embed any workflow logic - ie scheduling another worker, sending a message etc. Conductor has features to do this making it possible to decouple your workflow logic from worker implementation.
1. Workers should be stateless and should not maintain any state on the process they are running.
2. Ideally, workers should be idempotent.
3. The worker should follow the Single Responsibility Principle and do exactly one thing they are responsible for.
4. The worker should not embed any workflow logic - i.e., scheduling another worker, sending a message, etc. The Conductor has features to do this, making it possible to decouple your workflow logic from worker implementation.



Expand Down
55 changes: 26 additions & 29 deletions java-sdk/workflow_sdk.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ Workflow SDK provides fluent API to create workflows with strongly typed interfa
### ConductorWorkflow
[ConductorWorkflow](https://github.com/Netflix/conductor/blob/main/java-sdk/src/main/java/com/netflix/conductor/sdk/workflow/def/ConductorWorkflow.java) is the SDK representation of a Conductor workflow.

#### Create a `ConductorWorkflow` instance
#### Create a `ConductorWorkflow` Instance
```java
ConductorWorkflow<GetInsuranceQuote> conductorWorkflow = new WorkflowBuilder<GetInsuranceQuote>(executor)
.name("sdk_workflow_example")
Expand All @@ -18,18 +18,18 @@ ConductorWorkflow<GetInsuranceQuote> conductorWorkflow = new WorkflowBuilder<Get
.build();
```
### Working with Simple Worker Tasks
Use [SimpleTask](https://github.com/Netflix/conductor/blob/main/java-sdk/src/main/java/com/netflix/conductor/sdk/workflow/def/tasks/SimpleTask.java) to add simple task to a workflow.
Use [SimpleTask](https://github.com/Netflix/conductor/blob/main/java-sdk/src/main/java/com/netflix/conductor/sdk/workflow/def/tasks/SimpleTask.java) to add a simple task to a workflow.

Example:
```java
...
builder.add(new SimpleTask("send_email", "send_email"))
...
```
### Wiring inputs the task
use `input` methods to configure the inputs the task.
### Wiring Inputs to Task
Use `input` methods to configure the inputs to the task.

See https://netflix.github.io/conductor/how-tos/Tasks/task-inputs/ for details on Task Inputs/Outputs
See our doc on [task inputs](https://conductor.netflix.com/how-tos/Tasks/task-inputs.html) for more details.

Example
```java
Expand All @@ -40,21 +40,19 @@ builder.add(
);
```

### Working with operators
Each of the operator -
### Working with Operators
Each operator has its own class that can be added to the workflow builder.

[ForkJoin](https://github.com/Netflix/conductor/blob/main/java-sdk/src/main/java/com/netflix/conductor/sdk/workflow/def/tasks/ForkJoin.java),
[Wait](https://github.com/Netflix/conductor/blob/main/java-sdk/src/main/java/com/netflix/conductor/sdk/workflow/def/tasks/Wait.java),
[Switch](https://github.com/Netflix/conductor/blob/main/java-sdk/src/main/java/com/netflix/conductor/sdk/workflow/def/tasks/Switch.java),
[DynamicFork](https://github.com/Netflix/conductor/blob/main/java-sdk/src/main/java/com/netflix/conductor/sdk/workflow/def/tasks/DynamicFork.java),
[DoWhile](https://github.com/Netflix/conductor/blob/main/java-sdk/src/main/java/com/netflix/conductor/sdk/workflow/def/tasks/DoWhile.java),
[Join](https://github.com/Netflix/conductor/blob/main/java-sdk/src/main/java/com/netflix/conductor/sdk/workflow/def/tasks/Join.java),
[Dynamic](https://github.com/Netflix/conductor/blob/main/java-sdk/src/main/java/com/netflix/conductor/sdk/workflow/def/tasks/Dynamic.java),
[Terminate](https://github.com/Netflix/conductor/blob/main/java-sdk/src/main/java/com/netflix/conductor/sdk/workflow/def/tasks/Terminate.java),
[SubWorkflow](https://github.com/Netflix/conductor/blob/main/java-sdk/src/main/java/com/netflix/conductor/sdk/workflow/def/tasks/SubWorkflow.java),
[SetVariable](https://github.com/Netflix/conductor/blob/main/java-sdk/src/main/java/com/netflix/conductor/sdk/workflow/def/tasks/SetVariable.java),

have their own class that can be added to the workflow builder.
* [ForkJoin](https://github.com/Netflix/conductor/blob/main/java-sdk/src/main/java/com/netflix/conductor/sdk/workflow/def/tasks/ForkJoin.java)
* [Wait](https://github.com/Netflix/conductor/blob/main/java-sdk/src/main/java/com/netflix/conductor/sdk/workflow/def/tasks/Wait.java)
* [Switch](https://github.com/Netflix/conductor/blob/main/java-sdk/src/main/java/com/netflix/conductor/sdk/workflow/def/tasks/Switch.java)
* [DynamicFork](https://github.com/Netflix/conductor/blob/main/java-sdk/src/main/java/com/netflix/conductor/sdk/workflow/def/tasks/DynamicFork.java)
* [DoWhile](https://github.com/Netflix/conductor/blob/main/java-sdk/src/main/java/com/netflix/conductor/sdk/workflow/def/tasks/DoWhile.java)
* [Join](https://github.com/Netflix/conductor/blob/main/java-sdk/src/main/java/com/netflix/conductor/sdk/workflow/def/tasks/Join.java)
* [Dynamic](https://github.com/Netflix/conductor/blob/main/java-sdk/src/main/java/com/netflix/conductor/sdk/workflow/def/tasks/Dynamic.java)
* [Terminate](https://github.com/Netflix/conductor/blob/main/java-sdk/src/main/java/com/netflix/conductor/sdk/workflow/def/tasks/Terminate.java)
* [SubWorkflow](https://github.com/Netflix/conductor/blob/main/java-sdk/src/main/java/com/netflix/conductor/sdk/workflow/def/tasks/SubWorkflow.java)
* [SetVariable](https://github.com/Netflix/conductor/blob/main/java-sdk/src/main/java/com/netflix/conductor/sdk/workflow/def/tasks/SetVariable.java)


#### Register Workflow with Conductor Server
Expand All @@ -66,12 +64,12 @@ have their own class that can be added to the workflow builder.
//3. There are missing task definitions
boolean registered = workflow.registerWorkflow();
```
#### Overwrite existing workflow definition
#### Overwrite Existing Workflow Definition​
```java
boolean registered = workflow.registerWorkflow(true);
```

#### Overwrite existing workflow definition & register any missing task definitions
#### Overwrite existing workflow definitions & registering any missing task definitions
```java
boolean registered = workflow.registerWorkflow(true, true);
```
Expand All @@ -84,9 +82,8 @@ ConductorWorkflow<GetInsuranceQuote> conductorWorkflow =
.from("sdk_workflow_example", 1);
```

#### Start a workflow execution
Start the execution of the workflow based on the definition registered on the server.
Use register method to register a workflow on the server before executing.
#### Start Workflow Execution
Start the execution of the workflow based on the definition registered on the server. Use the register method to register a workflow on the server before executing.

```java

Expand All @@ -102,18 +99,18 @@ String workflowId = workflowRun.getWorkflowId();
//Get the status of workflow execution
WorkflowStatus status = workflowRun.getStatus();
```
See [Workflow](https://github.com/Netflix/conductor/blob/main/common/src/main/java/com/netflix/conductor/common/run/Workflow.java) for more details on Workflow object.
See [Workflow](https://github.com/Netflix/conductor/blob/main/common/src/main/java/com/netflix/conductor/common/run/Workflow.java) for more details on the Workflow object.

#### Start a dynamic workflow execution
Dynamic workflows are executed by specifying the workflow definition along with the execution and does not require registering the workflow on the server before executing.
#### Start Dynamic Workflow Execution
Dynamic workflows are executed by specifying the workflow definition along with the execution and do not require registering the workflow on the server before executing.

##### Use cases for dynamic workflows
1. Each workflow run has a unique workflow definition
2. Workflows are defined based on the user data and cannot be modeled ahead of time statically

```java
//1. Use WorkflowBuilder to create ConductorWorkflow
//2. Execute using the definition created by SDK
//1. Use WorkflowBuilder to create ConductorWorkflow.
//2. Execute using the definition created by SDK.
CompletableFuture<Workflow> execution = conductorWorkflow.executeDynamic(input);

```
Expand Down