Skip to content

Commit

Permalink
[INLONG-2017] Add more guide documents for Pulsar (apache#229)
Browse files Browse the repository at this point in the history
Co-authored-by: dockerzhang <dockerzhang@tencent.com>
  • Loading branch information
dockerzhang and dockerzhang authored Dec 17, 2021
1 parent 9b14018 commit b4c2c6b
Show file tree
Hide file tree
Showing 6 changed files with 62 additions and 26 deletions.
9 changes: 7 additions & 2 deletions docs/deployment/bare_metal.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,16 @@ sidebar_position: 4
- ZooKeeper 3.5+
- MySQL 5.7+
- Flink 1.9.x
- Apache Pulsar 2.6+ (Optional)

## Deploy InLong TubeMQ Server (Optional)
If you use Apache Pulsar, you don’t need to install this component.

## Deploy InLong TubeMQ Server
[deploy InLong TubeMQ Server](modules/tubemq/quick_start.md)

## Deploy InLong TubeMQ Manager
## Deploy InLong TubeMQ Manager (Optional)
If you use Apache Pulsar, you don’t need to install this component.

[deploy InLong TubeMQ Manager](modules/tubemq/tubemq-manager/quick_start.md)

## Deploy InLong Manager
Expand Down
32 changes: 22 additions & 10 deletions docs/modules/manager/quick_start.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,9 +13,9 @@ title: Deployment
mysql -uDB_USER -pDB_PASSWD < sql/apache_inlong_manager.sql
```

## Deploy and start manager-web
## Deploy manager

**manager-web is a background service that interacts with the front-end page.**
manager-web is a background service that interacts with the front-end page.

### Modify configuration

Expand All @@ -39,25 +39,37 @@ The dev configuration is specified above, then modify the `conf/application-dev.
spring.datasource.password=DB_PASSWD
```

2) Modify the connection information of the Tube and ZooKeeper clusters, among which `cluster.zk.root` suggests using
the default value:
2) Configure the Message Queue Service, you could choose InLong TubeMQ or Apache Pulsar:

- Configuration TubeMQ cluster information if using TubeMQ
```properties
# Manager address of Tube cluster, used to create Topic
# Manager address of TubeMQ cluster, used to create Topic
cluster.tube.manager=http://127.0.0.1:8081
# Broker used to manage Tube
# Broker used to manage TubeMQ
cluster.tube.master=127.0.0.1:8000,127.0.0.1:8010
# Tube cluster ID
# TubeMQ cluster ID
cluster.tube.clusterId=1
```

- Configuration Pulsar cluster information if using Pulsar
```properties
# Pulsar admin URL
pulsar.adminUrl=http://127.0.0.1:8080,127.0.0.2:8080,127.0.0.3:8080
# Pulsar broker address
pulsar.serviceUrl=pulsar://127.0.0.1:6650,127.0.0.1:6650,127.0.0.1:6650
# Default tenant of Pulsar
pulsar.defaultTenant=public
```

3) Configure ZooKeeper clusters information:

```properties
# ZK cluster, used to push the configuration of Sort
cluster.zk.url=127.0.0.1:2181
cluster.zk.root=inlong_hive
# Sort application name, that is, set the cluster-id parameter of Sort, the default value is "inlong_app"
# application name, that is the cluster-id parameter of InLong Sort
sort.appName=inlong_app
```

### Start the service

Enter the decompressed directory, execute `sh bin/startup.sh` to start the service, and check the
Expand Down
4 changes: 2 additions & 2 deletions docs/user_guide/user_manual.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,8 @@ Click [Create], there are two steps to fill in data access information: Group in

Access requirements require users to choose message middleware: high throughput (TUBE) or High reliability (PULSAR):

high throughput (TUBE): high-throughput message transmission component, suitable for log message transmission.
high reliability (PULSAR): high-reliability message transmission component, suitable for billing transmission.
- high throughput (Inlong TubeMQ): high-throughput message transmission component, suitable for log message transmission.
- high reliability (Apache PULSAR): high-reliability message transmission component, suitable for billing transmission.

#### Group Information

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,16 @@ sidebar_position: 4
- ZooKeeper 3.5+
- MySQL 5.7+
- Flink 1.9.x
- Apache Pulsar 2.6+ (可选)

## 部署InLong TubeMQ Server (可选)
如果使用Apache Pulsar,可不安装该组件。

## 部署InLong TubeMQ Server
[部署InLong TubeMQ Server](modules/tubemq/quick_start.md)

## 部署InLong TubeMQ Manager
## 部署InLong TubeMQ Manager (可选)
如果使用Apache Pulsar,可不安装该组件。

[部署InLong TubeMQ Manager](modules/tubemq/tubemq-manager/quick_start.md)

## 部署InLong Manager
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,9 @@ title: 安装部署
mysql -uDB_USER -pDB_PASSWD < sql/apache_inlong_manager.sql
```

## 部署、启动 manager-web
## 部署manager

**manager-web 是与前端页面交互的后台服务。**
manager-web 是与前端页面交互的后台服务。

### 修改配置

Expand All @@ -38,21 +38,35 @@ spring.profiles.active=dev
spring.datasource.password=DB_PASSWD
```

2) 修改 Tube 和 ZooKeeper 集群的连接信息,其中 `cluster.zk.root` 建议使用默认值
2) 配置消息队列服务,可以使用InLong TubeMQ 或者 Apache Pulsar

- 若使用TubeMQ,配置TubeMQ 集群信息
```properties
# Tube 集群的 Manager 地址,用来创建 Topic
# TubeMQ 集群的 Manager 地址,用来创建 Topic
cluster.tube.manager=http://127.0.0.1:8081
# 用来管理 Tube 的 Broker
# 用来管理 TubeMQ 的 Broker
cluster.tube.master=127.0.0.1:8000,127.0.0.1:8010
# Tube 集群的 ID
# TubeMQ 集群的 ID
cluster.tube.clusterId=1
```

- 若使用Pulsar,配置Pulsar 集群信息
```properties
# Pulsar admin URL
pulsar.adminUrl=http://127.0.0.1:8080,127.0.0.2:8080,127.0.0.3:8080
# Pulsar broker address
pulsar.serviceUrl=pulsar://127.0.0.1:6650,127.0.0.1:6650,127.0.0.1:6650
# Default tenant of Pulsar
pulsar.defaultTenant=public
```

3) 配置ZooKeeper 集群信息:

```properties
# ZK 集群,用来推送 Sort 的配置
cluster.zk.url=127.0.0.1:2181
cluster.zk.root=inlong_hive
# Sort 应用名称,即设置 Sort 的 cluster-id 参数,默认值为"inlong_app"
# 应用名称,即InLong Sort 的 cluster-id 参数
sort.appName=inlong_app
```

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,8 @@ sidebar_position: 1

接入要求需要用户进行选择消息中间件:高吞吐(TUBE)或者高可靠(PULSAR):

高吞吐—Tube :高吞吐消息传输组件,适用于日志类的消息传递。
高可靠—Pulsar :高可靠消息传输组件,适用于计费等场景。
- 高吞吐(InLong TubeMQ):高吞吐消息传输组件,适用于日志类的消息传递。
- 高可靠(Apache Pulsar):高可靠消息传输组件,适用于计费等场景。

#### Group 信息

Expand Down

0 comments on commit b4c2c6b

Please sign in to comment.