Skip to content

Commit

Permalink
docs: add getting started
Browse files Browse the repository at this point in the history
  • Loading branch information
siyul-park committed Jul 7, 2024
1 parent 54dbea2 commit 720a2d7
Show file tree
Hide file tree
Showing 20 changed files with 359 additions and 104 deletions.
17 changes: 10 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,21 +13,23 @@ Translations:

A high-performance, extremely flexible, and easily extensible multipurpose workflow engine.

It efficiently manages tasks of varying durations, from short-term to long-term, providing a simple environment for defining data processing flows. This ensures optimal performance with low latency and high throughput across various operations.
It efficiently manages tasks of varying durations, from short-term to long-term, providing a straightforward environment for defining data processing flows. This ensures optimal performance with low latency and high throughput across various operations.

The built-in extensions are crafted for efficient execution of short-term tasks, offering a wide array of functionalities. Furthermore, they facilitate seamless integration of additional features, allowing for flexible expansion as needed.
The [built-in extensions](./ext/README.md) are designed to efficiently handle short-term tasks while offering a wide range of functionalities. They also support the seamless integration of additional features, allowing for flexible expansion when needed.

Importantly, the engine does not require the use of specific nodes; instead, all nodes can be connected to or disconnected from the engine through extensions based on the service's requirements.

Develop a service that integrates user personalization, with the added benefit of easily expanding functionality as needed.

## Principles

- **Performance:** Achieve optimal throughput, minimal latency, and maximum scalability across diverse workloads.
- **Flexibility:** Define complex data processing flows declaratively to adapt seamlessly to changing requirements, enabling dynamic modifications and real-time adjustments.
- **Flexibility:** Adjust specifications in real-time to effectively meet diverse and changing needs.
- **Extensibility:** Utilize the built-in extensions to efficiently execute various tasks, seamlessly integrating or customizing additional functionalities as needed.

## Quick Start

To run the [ping example](/examples/ping.yaml), use this command:
To run the [ping example](./examples/ping.yaml), use this command:

```shell
./uniflow start --filename example/ping.yaml
Expand Down Expand Up @@ -61,7 +63,7 @@ Configure the environment using either `.uniflow.toml` or system environment var

## Benchmarks

The benchmarking tests were conducted using a VPS S SSD (4 Core, 8 GB) from [Contabo](https://contabo.com/). Performance was measured with the [Apache HTTP server benchmarking tool](https://httpd.apache.org/docs/2.4/programs/ab.html) over the loopback network adapter (127.0.0.1). The test workflow used the [ping example](/examples/ping.yaml), consisting of `listener`, `router`, and `snippet` nodes.
The benchmarking tests were conducted using a VPS S SSD (4 Core, 8 GB) from [Contabo](https://contabo.com/). Performance was measured with the [Apache HTTP server benchmarking tool](https://httpd.apache.org/docs/2.4/programs/ab.html) over the loopback network adapter (127.0.0.1). The test workflow used the [ping example](./examples/ping.yaml), consisting of `listener`, `router`, and `snippet` nodes.

```sh
ab -n 102400 -c 1024 http://127.0.0.1:8000/ping
Expand Down Expand Up @@ -103,9 +105,10 @@ Percentage of the requests served within a certain time (ms)
100% 559 (longest request)
```

## Links
## Learn More

- [**Documentation**](/docs/README.md)
- [Getting Started](./docs/getting_started.md): Learn how to install the CLI, manage workflows, and run the engine with clear instructions.
- [Architecture](./docs/architecture.md): Understand how node specifications connect to build robust workflows and how namespaces ensure efficient and isolated execution.

<!-- Go -->

Expand Down
37 changes: 14 additions & 23 deletions README_kr.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,45 +11,35 @@

## 개요

이 엔진은 높은 성능과 극도의 유연성을 갖춘 쉽게 확장 가능한 다목적 워크플로 엔진입니다.
높은 성능과 극도의 유연성을 갖춘 다목적 워크플로 엔진으로, 쉽게 확장할 수 있습니다.

다양한 기간이 걸리는 작업을 효율적으로 처리하며 데이터 처리 흐름을 선언적으로 정의하는 환경을 제공합니다. 이를 통해 다양한 작업에서 최적의 성능, 낮은 대기 시간 및 높은 처리량을 달성할 수 있습니다.
다양한 기간이 소요되는 작업을 효율적으로 처리하며, 데이터 처리 흐름을 선언적으로 정의하고 동적으로 수정할 수 있는 환경을 제공합니다.

내장된 확장 기능은 짧은 기간이 걸리는 작업을 효율적으로 실행하는 데 중점을 두고 다양한 기능을 제공합니다. 또한 필요에 따라 유연하게 기능을 확장할 수 있도록 설계되었습니다.
[내장된 확장 기능](./ext/README_kr.md) 짧은 기간이 걸리는 작업을 효율적으로 실행하며, 다양한 기능을 구현할 수 있게 도와줍니다. 하지만 엔진은 특정 노드의 사용을 강제하지는 않습니다. 모든 노드는 서비스에 맞게 자유롭게 추가하거나 제거될 수 있습니다.

사용자에게 개인화된 경험을 제공하는 서비스를 개발하고 필요할 때 쉽게 기능을 확장해보세요.
서비스에 엔진을 통합하여 사용자에게 개인화된 경험을 제공하고 기능을 풍부하게 확장해보세요.

## 원칙
## 핵심 가치

- **성능:** 다양한 작업 부하에서 최적의 처리량, 최소 대기 시간 및 최대 확장성을 달성합니다.
- **유연성:** 변화하는 요구 사항에 신속하게 적응할있도록 복잡한 데이터 처리 흐름을 선언적으로 정의하며, 동적 수정과 실시간 조정을 가능하게 합니다.
- **확장성:** 내장된 확장 기능을 활용하여 다양한 작업을 효율적으로 실행하고 필요에 따라 기능을 추가하거나 사용자 정의할 수 있습니다.
- **성능:** 다양한 환경에서 최대 처리량, 최소 대기 시간을 달성합니다.
- **유연성:** 명세를 동적으로 수정하고 실시간으로 조정할있습니다.
- **확장성:** 새로운 노드를 자유롭게 지원하여 기능을 확장할 수 있습니다.

## 빠른 시작

[ping 예제](/examples/ping.yaml)실행하려면 다음 명령을 사용하세요:
아래 명령을 사용하여 [ping 예제](./examples/ping.yaml)실행해 보세요:

```shell
./uniflow start --filename example/ping.yaml
```

`--filename` 플래그는 네임스페이스에 노드가 이미 존재하지 않는 경우 자동으로 설치합니다.

예상대로 HTTP 엔드포인트가 제공되는지 확인하세요:
이제 예상대로 HTTP 엔드포인트가 제공되는지 확인하세요:

```shell
curl localhost:8000/ping
pong#
```

실행 중인 서버에 노드를 적용하려면 `apply` 명령을 사용하세요.

추가 세부 정보는 다음 명령 도움말을 참조하세요:

```shell
./dist/uniflow start --help
```

## 구성

`.uniflow.toml` 또는 시스템 환경 변수를 사용하여 환경을 구성하세요.
Expand All @@ -61,7 +51,7 @@ pong#

## 벤치마크

벤치마크 테스트는 [Contabo](https://contabo.com/)의 VPS S SSD (4코어, 8GB)에서 수행되었습니다. 성능은 [Apache HTTP 서버 벤치마킹 도구](https://httpd.apache.org/docs/2.4/programs/ab.html)를 이용해 루프백 네트워크 어댑터(127.0.0.1)를 통해 측정되었습니다. 테스트 워크플로는 `listener`, `router`, `snippet` 노드로 구성된 [ping 예제](/examples/ping.yaml)를 사용했습니다.
벤치마크는 [Contabo](https://contabo.com/)의 VPS S SSD (4코어, 8GB)에서 수행되고 [아파치 웹서버 성능검사 도구](https://httpd.apache.org/docs/2.4/programs/ab.html)를 이용해 측정되었습니다. 워크플로우는 `listener`, `router`, `snippet` 노드로 구성된 [ping 예제](./examples/ping.yaml)를 사용했습니다.

```sh
ab -n 102400 -c 1024 http://127.0.0.1:8000/ping
Expand Down Expand Up @@ -103,9 +93,10 @@ Percentage of the requests served within a certain time (ms)
100% 559 (longest request)
```

## 관련 자료
## 자세히 알아보기

- [**문서**](/docs/README_kr.md)
- [시작하기](./docs/getting_started_kr.md): CLI를 설치하고 워크플로우를 관리하며 엔진을 실행하는 방법을 살펴보세요.
- [아키텍처](./docs/architecture_kr.md): 노드 명세가 엔진에 어떻게 로드되고, 워크플로우가 어떻게 실행되는지 살펴보세요.

<!-- Go -->

Expand Down
16 changes: 8 additions & 8 deletions cmd/README.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
# Command
# Command Line Interface (CLI)

Effectively manage the Command Line Interface (CLI) with a diverse set of commands tailored for workflow management. This serves as the default CLI executable with pre-installed first-party plugins.
Effectively manage the Command Line Interface (CLI) with a versatile set of commands designed for seamless workflow management. This CLI serves as the default executable with [built-in extensions](../ext/README.md).

## Configuration Options

Configure the system using environment variables before executing any command. Utilize `.uniflow.toml` or system environment variables.
Before executing any commands, configure the system using environment variables. You can utilize `.uniflow.toml` or system environment variables.

| Key | Example |
| --- | --- |
| `database.url` | `mem://` or `mongodb://` |
| `database.name` | - |
| Key | Example |
|-----------------|----------------------|
| `database.url` | `mem://` or `mongodb://` |
| `database.name` | - |

Explore various commands and configurations to seamlessly adapt to your workflow needs.
If using [MongoDB](https://www.mongodb.com/), ensure that [change streams](https://www.mongodb.com/docs/manual/changeStreams/) are enabled to track modifications in node specifications. Utilize a [replica set](https://www.mongodb.com/ko-kr/docs/manual/replication/#std-label-replication) for change streams.
14 changes: 14 additions & 0 deletions cmd/README_kr.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# 명령줄 인터페이스 (CLI)

다양한 작업 흐름 관리를 위해 설계된 다목적 명령줄 인터페이스 (CLI)를 효과적으로 관리하세요. 이 CLI는 [내장 확장 기능](../ext/README.md)을 포함한 기본 실행 파일로 제공됩니다.

## 구성

명령을 실행하기 전에 환경 변수를 사용하여 시스템을 구성하세요. `.uniflow.toml` 파일이나 시스템 환경 변수를 활용할 수 있습니다.

|| 예시 |
|-----------------|-----------------------|
| `database.url` | `mem://` 또는 `mongodb://` |
| `database.name` | - |

[MongoDB](https://www.mongodb.com/)를 사용할 경우 런타임 엔진이 노드 명세의 변경을 추적할 수 있도록 [변경 스트림](https://www.mongodb.com/docs/manual/changeStreams/)이 활성화되어 있어야 합니다. 변경 스트림을 이용하기 위해 [복제본 세트](https://www.mongodb.com/ko-kr/docs/manual/replication/#std-label-replication)를 활용하세요.
4 changes: 2 additions & 2 deletions cmd/pkg/cli/apply.go
Original file line number Diff line number Diff line change
Expand Up @@ -25,11 +25,11 @@ type ApplyConfig struct {
func NewApplyCommand(config ApplyConfig) *cobra.Command {
cmd := &cobra.Command{
Use: "apply",
Short: "Apply resources in namespace",
Short: "Apply node specifications to the specified namespace",
RunE: runApplyCommand(config),
}

cmd.PersistentFlags().StringP(flagNamespace, toShorthand(flagNamespace), "", "Set the resource's namespace. If not set, use the default namespace")
cmd.PersistentFlags().StringP(flagNamespace, toShorthand(flagNamespace), spec.DefaultNamespace, "Set the resource's namespace. If not set, use the default namespace")
cmd.PersistentFlags().StringP(flagFilename, toShorthand(flagFilename), "", "Set the file path to be applied")

return cmd
Expand Down
4 changes: 2 additions & 2 deletions cmd/pkg/cli/delete.go
Original file line number Diff line number Diff line change
Expand Up @@ -23,11 +23,11 @@ type DeleteConfig struct {
func NewDeleteCommand(config DeleteConfig) *cobra.Command {
cmd := &cobra.Command{
Use: "delete",
Short: "Delete resources in namespace",
Short: "Delete node specifications from the specified namespace",
RunE: runDeleteCommand(config),
}

cmd.PersistentFlags().StringP(flagNamespace, toShorthand(flagNamespace), "", "Set the resource's namespace. If not set, use the default namespace")
cmd.PersistentFlags().StringP(flagNamespace, toShorthand(flagNamespace), spec.DefaultNamespace, "Set the resource's namespace. If not set, use the default namespace")
cmd.PersistentFlags().StringP(flagFilename, toShorthand(flagFilename), "", "Set the file path to be deleted")

return cmd
Expand Down
4 changes: 2 additions & 2 deletions cmd/pkg/cli/get.go
Original file line number Diff line number Diff line change
Expand Up @@ -19,11 +19,11 @@ type GetConfig struct {
func NewGetCommand(config GetConfig) *cobra.Command {
cmd := &cobra.Command{
Use: "get",
Short: "Get and display resources in namespace",
Short: "Get node specifications from the specified namespace",
RunE: runGetCommand(config),
}

cmd.PersistentFlags().StringP(flagNamespace, toShorthand(flagNamespace), "", "Set the resource's namespace. If not set, use all namespace")
cmd.PersistentFlags().StringP(flagNamespace, toShorthand(flagNamespace), spec.DefaultNamespace, "Set the resource's namespace. If not set, use all namespace")

return cmd
}
Expand Down
4 changes: 2 additions & 2 deletions cmd/pkg/cli/start.go
Original file line number Diff line number Diff line change
Expand Up @@ -29,11 +29,11 @@ type StartConfig struct {
func NewStartCommand(config StartConfig) *cobra.Command {
cmd := &cobra.Command{
Use: "start",
Short: "Start a application",
Short: "Starts the workflow engine within the specified namespace",
RunE: runStartCommand(config),
}

cmd.PersistentFlags().StringP(flagNamespace, toShorthand(flagNamespace), "", "Set the namespace for running")
cmd.PersistentFlags().StringP(flagNamespace, toShorthand(flagNamespace), spec.DefaultNamespace, "Set the namespace for running")
cmd.PersistentFlags().StringP(flagFilename, toShorthand(flagFilename), "", "Set the boot file path for initializing nodes")

return cmd
Expand Down
9 changes: 5 additions & 4 deletions cmd/pkg/printer/table.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ import (
"github.com/jedib0t/go-pretty/v6/table"
"github.com/jedib0t/go-pretty/v6/text"
"github.com/oliveagle/jsonpath"
"github.com/siyul-park/uniflow/pkg/spec"
"github.com/siyul-park/uniflow/pkg/types"
)

Expand Down Expand Up @@ -64,13 +65,13 @@ var style = table.Style{
}

// PrintTable prints tabular data to the specified writer using the provided columns.
func PrintTable(writer io.Writer, data any, columns []TableColumnDefinition) error {
func PrintTable(writer io.Writer, specs []spec.Spec, columns []TableColumnDefinition) error {
tablePrinter, err := NewTable(columns)
if err != nil {
return err
}

table, err := tablePrinter.Print(data)
table, err := tablePrinter.Print(specs)
if err != nil {
return err
}
Expand Down Expand Up @@ -99,8 +100,8 @@ func NewTable(columns []TableColumnDefinition) (*TablePrinter, error) {
}

// Print formats and prints the provided data as a table.
func (p *TablePrinter) Print(data any) (string, error) {
value, err := types.TextEncoder.Encode(data)
func (p *TablePrinter) Print(specs []spec.Spec) (string, error) {
value, err := types.TextEncoder.Encode(specs)
if err != nil {
return "", err
}
Expand Down
33 changes: 14 additions & 19 deletions cmd/pkg/printer/table_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,46 +4,41 @@ import (
"bytes"
"testing"

"github.com/siyul-park/uniflow/pkg/spec"
"github.com/stretchr/testify/assert"
)

func TestTablePrintTable(t *testing.T) {
data := []map[string]any{
{
"foo": "foo",
"bar": "bar",
specs := []spec.Spec{
&spec.Meta{
Kind: "foo",
Name: "bar",
},
}

buf := new(bytes.Buffer)

expect := " FOO BAR \n foo bar "
expect := " ID KIND NAMESPACE NAME LINKS \n <nil> foo <nil> bar <nil> "

err := PrintTable(buf, data, []TableColumnDefinition{
{Name: "foo", Format: "$.foo"},
{Name: "bar", Format: "$.bar"},
})
err := PrintTable(buf, specs, SpecTableColumnDefinitions)
assert.NoError(t, err)
assert.Equal(t, expect, buf.String())
}

func TestTablePrinter_Print(t *testing.T) {
data := []map[string]any{
{
"foo": "foo",
"bar": "bar",
specs := []spec.Spec{
&spec.Meta{
Kind: "foo",
Name: "bar",
},
}

p, err := NewTable([]TableColumnDefinition{
{Name: "foo", Format: "$.foo"},
{Name: "bar", Format: "$.bar"},
})
p, err := NewTable(SpecTableColumnDefinitions)
assert.NoError(t, err)

expect := " FOO BAR \n foo bar "
expect := " ID KIND NAMESPACE NAME LINKS \n <nil> foo <nil> bar <nil> "

table, err := p.Print(data)
table, err := p.Print(specs)
assert.NoError(t, err)
assert.Equal(t, expect, table)
}
7 changes: 0 additions & 7 deletions docs/README.md

This file was deleted.

7 changes: 0 additions & 7 deletions docs/README_kr.md

This file was deleted.

Loading

0 comments on commit 720a2d7

Please sign in to comment.