Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 20 additions & 13 deletions docs/Plugins/feishu.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,16 +10,23 @@ This plugin collects Feishu meeting data through [Feishu Openapi](https://open.f

## Configuration

In order to fully use this plugin, you will need to get app_id and app_secret from a Feishu administrator (for help on App info, please see [official Feishu Docs](https://open.feishu.cn/document/ukTMukTMukTM/ukDNz4SO0MjL5QzM/auth-v3/auth/tenant_access_token_internal)),
then set these two parameters via Dev Lake's `.env`.
In order to fully use this plugin, you will need to get `app_id` and `app_secret` from a Feishu administrator (for help on App info, please see [official Feishu Docs](https://open.feishu.cn/document/ukTMukTMukTM/ukDNz4SO0MjL5QzM/auth-v3/auth/tenant_access_token_internal)),

### By `.env`

The connection aspect of the configuration screen requires the following key fields to connect to the Feishu API. As Feishu is a single-source data provider at the moment, the connection name is read-only as there is only one instance to manage. As we continue our development roadmap we may enable multi-source connections for Feishu in the future.
A connection should be created before you can collection any data. Currently, this plugin supports creating connection by requesting `connections` API:

```
FEISHU_APPID=app_id
FEISHU_APPSCRECT=app_secret
curl 'http://localhost:8080/plugins/feishu/connections' \
--header 'Content-Type: application/json' \
--data-raw '
{
"name": "feishu",
"endpoint": "https://open.feishu.cn/open-apis/vc/v1/",
"proxy": "http://localhost:1080",
"rateLimitPerHour": 20000,
"appId": "<YOUR_APP_ID>",
"appSecret": "<YOUR_APP_SECRET>"
}
'
```

## Collect data from Feishu
Expand All @@ -33,8 +40,8 @@ To collect data, select `Advanced Mode` on the `Create Pipeline Run` page and pa
{
"plugin": "feishu",
"options": {
"numOfDaysToCollect" : 80,
"rateLimitPerSecond" : 5
"connectionId": 1,
"numOfDaysToCollect" : 80
}
}
]
Expand All @@ -47,18 +54,18 @@ To collect data, select `Advanced Mode` on the `Create Pipeline Run` page and pa

You can also trigger data collection by making a POST request to `/pipelines`.
```
curl --location --request POST 'localhost:8080/pipelines' \
curl 'http://localhost:8080/pipelines' \
--header 'Content-Type: application/json' \
--data-raw '
{
"name": "feishu 20211126",
"tasks": [[{
"plugin": "feishu",
"options": {
"numOfDaysToCollect" : 80,
"rateLimitPerSecond" : 5
"connectionId": 1,
"numOfDaysToCollect" : 80
}
}]]
}
'
```
```
56 changes: 25 additions & 31 deletions docs/Plugins/gitee.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,51 +6,42 @@ description: >

## Summary

## Configuration

### Provider (Datasource) Connection
The connection aspect of the configuration screen requires the following key fields to connect to the **Gitee API**. As gitee is a _single-source data provider_ at the moment, the connection name is read-only as there is only one instance to manage. As we continue our development roadmap we may enable _multi-source_ connections for gitee in the future.

- **Connection Name** [`READONLY`]
- ⚠️ Defaults to "**Gitee**" and may not be changed.
- **Endpoint URL** (REST URL, starts with `https://` or `http://`)
- This should be a valid REST API Endpoint eg. `https://gitee.com/api/v5/`
- ⚠️ URL should end with`/`
- **Auth Token(s)** (Personal Access Token)
- For help on **Creating a personal access token**
- Provide at least one token for Authentication with the . This field accepts a comma-separated list of values for multiple tokens. The data collection will take longer for gitee since they have a **rate limit of 2k requests per hour**. You can accelerate the process by configuring _multiple_ personal access tokens.

"For API requests using `Basic Authentication` or `OAuth`
This plugin collects `Gitee` data through [Gitee Openapi](https://gitee.com/api/v5/swagger).

## Configuration

If you have a need for more api rate limits, you can set many tokens in the config file and we will use all of your tokens.

For an overview of the **gitee REST API**, please see official [gitee Docs on REST](https://gitee.com/api/v5/swagger)

Click **Save Connection** to update connection settings.
In order to fully use this plugin, you will need to get `token` on the Gitee website.

A connection should be created before you can collection any data. Currently, this plugin supports creating connection by requesting `connections` API:

### Provider (Datasource) Settings
Manage additional settings and options for the gitee Datasource Provider. Currently there is only one **optional** setting, *Proxy URL*. If you are behind a corporate firewall or VPN you may need to utilize a proxy server.
```
curl 'http://localhost:8080/plugins/gitee/connections' \
--header 'Content-Type: application/json' \
--data-raw '
{
"name": "gitee",
"endpoint": "https://gitee.com/api/v5/",
"proxy": "http://localhost:1080",
"rateLimitPerHour": 20000,
"token": "<YOUR_TOKEN>"
}
'
```

**gitee Proxy URL [ `Optional`]**
Enter a valid proxy server address on your Network, e.g. `http://your-proxy-server.com:1080`

Click **Save Settings** to update additional settings.

### Regular Expression Configuration
Define regex pattern in .env
- GITEE_PR_BODY_CLOSE_PATTERN: Define key word to associate issue in pr body, please check the example in .env.example
## Collect data from Gitee

## Sample Request
In order to collect data, you have to compose a JSON looks like following one, and send it by selecting `Advanced Mode` on `Create Pipeline Run` page:

1. Configure-UI Mode
```json
[
[
{
"plugin": "gitee",
"options": {
"connectionId": 1,
"repo": "lake",
"owner": "merico-dev"
}
Expand All @@ -66,6 +57,7 @@ and if you want to perform certain subtasks.
"plugin": "gitee",
"subtasks": ["collectXXX", "extractXXX", "convertXXX"],
"options": {
"connectionId": 1,
"repo": "lake",
"owner": "merico-dev"
}
Expand All @@ -77,14 +69,15 @@ and if you want to perform certain subtasks.
2. Curl Mode:
You can also trigger data collection by making a POST request to `/pipelines`.
```
curl --location --request POST 'localhost:8080/pipelines' \
curl 'http://localhost:8080/pipelines' \
--header 'Content-Type: application/json' \
--data-raw '
{
"name": "gitee 20211126",
"tasks": [[{
"plugin": "gitee",
"options": {
"connectionId": 1,
"repo": "lake",
"owner": "merico-dev"
}
Expand All @@ -94,7 +87,7 @@ curl --location --request POST 'localhost:8080/pipelines' \
```
and if you want to perform certain subtasks.
```
curl --location --request POST 'localhost:8080/pipelines' \
curl 'http://localhost:8080/pipelines' \
--header 'Content-Type: application/json' \
--data-raw '
{
Expand All @@ -103,6 +96,7 @@ curl --location --request POST 'localhost:8080/pipelines' \
"plugin": "gitee",
"subtasks": ["collectXXX", "extractXXX", "convertXXX"],
"options": {
"connectionId": 1,
"repo": "lake",
"owner": "merico-dev"
}
Expand Down
7 changes: 3 additions & 4 deletions docs/Plugins/github.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,8 +28,6 @@ Here are some examples metrics using `GitHub` data:

## Configuration
- Configuring GitHub via [config-ui](/UserManuals/ConfigUI/GitHub.md).
- Configuring GitHub via `.env`
- GITHUB_PR_BODY_CLOSE_PATTERN: Define key word to associate issue in PR body, please check the example in .env.example

## Sample Request
To collect data, select `Advanced Mode` on the `Create Pipeline Run` page and paste a JSON config like the following:
Expand All @@ -40,6 +38,7 @@ To collect data, select `Advanced Mode` on the `Create Pipeline Run` page and pa
{
"plugin": "github",
"options": {
"connectionId": 1,
"repo": "lake",
"owner": "merico-dev"
}
Expand All @@ -50,19 +49,19 @@ To collect data, select `Advanced Mode` on the `Create Pipeline Run` page and pa

You can also trigger data collection by making a POST request to `/pipelines`.
```
curl --location --request POST 'localhost:8080/pipelines' \
curl 'http://localhost:8080/pipelines' \
--header 'Content-Type: application/json' \
--data-raw '
{
"name": "github 20211126",
"tasks": [[{
"plugin": "github",
"options": {
"connectionId": 1,
"repo": "lake",
"owner": "merico-dev"
}
}]]
}
'
```
<br/><br/><br/>
5 changes: 3 additions & 2 deletions docs/Plugins/jenkins.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,9 @@ To collect data, select `Advanced Mode` on the `Create Pipeline Run` page and pa
[
{
"plugin": "jenkins",
"options": {}
"options": {
"connectionId": 1
}
}
]
]
Expand All @@ -43,4 +45,3 @@ To collect data, select `Advanced Mode` on the `Create Pipeline Run` page and pa
## Relationship between job and build

Build is kind of a snapshot of job. Running job each time creates a build.
<br/><br/><br/>
26 changes: 26 additions & 0 deletions docs/Plugins/refdiff.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,32 @@ curl -v -XPOST http://localhost:8080/pipelines --data @- <<'JSON'
}
JSON
```
Or if you prefered calculating latest releases
```
curl -v -XPOST http://localhost:8080/pipelines --data @- <<'JSON'
{
"name": "test-refdiff",
"tasks": [
[
{
"plugin": "refdiff",
"options": {
"repoId": "github:GithubRepo:384111310",
"tagsPattern": "v\d+\.\d+.\d+",
"tagsLimit": 10,
"tagsOrder": "reverse semver",
"tasks": [
"calculateCommitsDiff",
"calculateIssuesDiff",
"calculatePrCherryPick",
]
}
}
]
]
}
JSON
```

## Development

Expand Down
33 changes: 20 additions & 13 deletions versioned_docs/version-v0.12/Plugins/feishu.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,16 +10,23 @@ This plugin collects Feishu meeting data through [Feishu Openapi](https://open.f

## Configuration

In order to fully use this plugin, you will need to get app_id and app_secret from a Feishu administrator (for help on App info, please see [official Feishu Docs](https://open.feishu.cn/document/ukTMukTMukTM/ukDNz4SO0MjL5QzM/auth-v3/auth/tenant_access_token_internal)),
then set these two parameters via Dev Lake's `.env`.
In order to fully use this plugin, you will need to get `app_id` and `app_secret` from a Feishu administrator (for help on App info, please see [official Feishu Docs](https://open.feishu.cn/document/ukTMukTMukTM/ukDNz4SO0MjL5QzM/auth-v3/auth/tenant_access_token_internal)),

### By `.env`

The connection aspect of the configuration screen requires the following key fields to connect to the Feishu API. As Feishu is a single-source data provider at the moment, the connection name is read-only as there is only one instance to manage. As we continue our development roadmap we may enable multi-source connections for Feishu in the future.
A connection should be created before you can collection any data. Currently, this plugin supports creating connection by requesting `connections` API:

```
FEISHU_APPID=app_id
FEISHU_APPSCRECT=app_secret
curl 'http://localhost:8080/plugins/feishu/connections' \
--header 'Content-Type: application/json' \
--data-raw '
{
"name": "feishu",
"endpoint": "https://open.feishu.cn/open-apis/vc/v1/",
"proxy": "http://localhost:1080",
"rateLimitPerHour": 20000,
"appId": "<YOUR_APP_ID>",
"appSecret": "<YOUR_APP_SECRET>"
}
'
```

## Collect data from Feishu
Expand All @@ -33,8 +40,8 @@ To collect data, select `Advanced Mode` on the `Create Pipeline Run` page and pa
{
"plugin": "feishu",
"options": {
"numOfDaysToCollect" : 80,
"rateLimitPerSecond" : 5
"connectionId": 1,
"numOfDaysToCollect" : 80
}
}
]
Expand All @@ -47,18 +54,18 @@ To collect data, select `Advanced Mode` on the `Create Pipeline Run` page and pa

You can also trigger data collection by making a POST request to `/pipelines`.
```
curl --location --request POST 'localhost:8080/pipelines' \
curl 'http://localhost:8080/pipelines' \
--header 'Content-Type: application/json' \
--data-raw '
{
"name": "feishu 20211126",
"tasks": [[{
"plugin": "feishu",
"options": {
"numOfDaysToCollect" : 80,
"rateLimitPerSecond" : 5
"connectionId": 1,
"numOfDaysToCollect" : 80
}
}]]
}
'
```
```
Loading