-
Notifications
You must be signed in to change notification settings - Fork 707
cloud: add doc for Data APP GitHub integration #13674
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
ti-chi-bot
merged 44 commits into
pingcap:release-6.5
from
qiancai:data_app_github_integration
Jun 5, 2023
Merged
Changes from all commits
Commits
Show all changes
44 commits
Select commit
Hold shift + click to select a range
5556cf8
add part of draft
qiancai ec4e75c
Update data-service-manage-github-integration.md
qiancai e20c0af
add the steps of github connection
qiancai d10d8cc
Update data-service-manage-github-integration.md
qiancai d78d9f0
add steps about importing configurations of an existing Data App
qiancai a833c62
update the files
qiancai c3103fc
update the steps according to the UI
qiancai d35c621
Apply suggestions from code review
qiancai 4de61c0
Update steps based on UI
qiancai 441da0b
Update data-service-manage-data-app.md
qiancai 519f528
Apply suggestions from code review
qiancai 92d2655
Update TOC-tidb-cloud.md
qiancai 6f89b99
TiDB Cloud Data Services -> TiDB Cloud Data Service
qiancai f9e0b2c
resolve comments
qiancai 609d67d
add app config
qiancai 262e19e
Update data-service-app-config.md
qiancai e57da45
Update data-service-app-config.md
qiancai 774705c
update the content
qiancai a64cd9c
Update data-service-app-config-files.md
qiancai e78cbd3
update the files according the latest UI design
qiancai 8def354
wording updates
qiancai 4706d43
updated the steps of GitHub connection
qiancai 0591579
use svg for icon
qiancai a55e8e0
rename the file
qiancai b7a389d
resolve comments
qiancai 1f9140a
fix the anchors
qiancai cc12897
Update TOC-tidb-cloud.md
qiancai 40d95fa
Apply suggestions from code review
qiancai e5a890d
Merge branch 'release-6.5' into data_app_github_integration
qiancai b36135b
add release note
qiancai 294220f
Update tidb-cloud-release-notes.md
qiancai c657db2
**Deployment** tab -> **Deployments** tab
qiancai 5ab8e70
**Unlink** -> **Disconnect**
qiancai 3a490b4
update release notes
qiancai 276143f
Apply suggestions from code review
qiancai c2fd871
Apply suggestions from code review
qiancai 07635f5
Apply suggestions from code review
qiancai 10e97fb
Automatic deployment -> Auto Sync & Deployment
qiancai b795745
resolve comments
qiancai 87fb6e2
fix typos
qiancai 685db8f
Apply suggestions from code review
qiancai f3b8cf6
fix a broken link
qiancai 40bc6ed
Update TOC-tidb-cloud.md
qiancai 95fec39
fix broken links
qiancai File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,205 @@ | ||
| --- | ||
| title: Data App Configuration Files | ||
| summary: This document describes the configuration files of Data App in TiDB Cloud. | ||
| --- | ||
|
|
||
| # Data App Configuration Files | ||
|
|
||
| This document describes the configuration files of a [Data App](/tidb-cloud/tidb-cloud-glossary.md#data-app) in TiDB Cloud. | ||
|
|
||
| If you have [connected your Data App to GitHub](/tidb-cloud/data-service-manage-github-connection.md), you can find the configuration files of your Data App in your specified directory on GitHub as follows: | ||
|
|
||
| ``` | ||
| ├── <Your Data App directory> | ||
| │ ├── data_sources | ||
| │ │ └── cluster.json | ||
| │ ├── dataapp_config.json | ||
| │ ├── http_endpoints | ||
| │ │ ├── config.json | ||
| │ │ └── sql | ||
| │ │ ├── <method>-<endpoint-path1>.sql | ||
| │ │ ├── <method>-<endpoint-path2>.sql | ||
| │ │ └── <method>-<endpoint-path3>.sql | ||
| ``` | ||
|
|
||
| ## Data source configuration | ||
|
|
||
| The data source of a Data App comes from its linked TiDB clusters. You can find the data source configuration in `data_sources/cluster.json`. | ||
|
|
||
| ``` | ||
| ├── <Your Data App directory> | ||
| │ ├── data_sources | ||
| │ │ └── cluster.json | ||
| ``` | ||
|
|
||
| For each Data App, you can link to one or multiple TiDB Serverless clusters. | ||
|
|
||
| The following is an example configuration of `cluster.json`. In this example, there are two linked clusters for this Data App. | ||
|
|
||
| ```json | ||
| [ | ||
| { | ||
| "cluster_id": <Cluster ID1> | ||
| }, | ||
| { | ||
| "cluster_id": <Cluster ID2> | ||
| } | ||
| ] | ||
| ``` | ||
|
|
||
| The field description is as follows: | ||
|
|
||
| | Field | Type | Description | | ||
| |---------|---------|--------------| | ||
| | `cluster_id` | Integer | The ID of your TiDB Serverless cluster. You can get it from the URL of your cluster. For example, if your cluster URL is `https://tidbcloud.com/console/clusters/1234567891234567890/overview`, your cluster ID is `1234567891234567890`. | | ||
|
|
||
| ## Data App configuration | ||
|
|
||
| The properties of a Data App contain the App ID, name, and type. You can find the properties in the `dataapp_config.json` file. | ||
|
|
||
| ``` | ||
| ├── <Your Data App directory> | ||
| │ ├── dataapp_config.json | ||
| ``` | ||
|
|
||
| The following is an example configuration of `dataapp_config.json`. | ||
|
|
||
| ```json | ||
| { | ||
| "app_id": "<Data App ID>", | ||
| "app_name": "<Data App name>", | ||
| "app_type": "dataapi" | ||
| } | ||
| ``` | ||
|
|
||
| The description of each field is as follows: | ||
|
|
||
| | Field | Type | Description | | ||
| |------------|--------|--------------------| | ||
| | `app_id` | String | The Data App ID. Do not change this field unless your `dataapp_config.json` file is copied from another Data App and you want to update it to the ID of your current Data App. Otherwise, the deployment triggered by this modification will fail. | | ||
| | `app_name` | String | The Data App name. | | ||
| | `app_type` | String | The Data App type, which can only be `"dataapi"`. | | ||
|
|
||
| ## HTTP endpoint configuration | ||
|
|
||
| In your Data App directory, you can find endpoint configurations in `http_endpoints/config.json` and the SQL files in `http_endpoints/sql/<method>-<endpoint-name>.sql`. | ||
|
|
||
| ``` | ||
| ├── <Your Data App directory> | ||
| │ ├── http_endpoints | ||
| │ │ ├── config.json | ||
| │ │ └── sql | ||
| │ │ ├── <method>-<endpoint-path1>.sql | ||
| │ │ ├── <method>-<endpoint-path2>.sql | ||
| │ │ └── <method>-<endpoint-path3>.sql | ||
| ``` | ||
|
|
||
| ### Endpoint configuration | ||
|
|
||
| For each Data App, there can be one or multiple endpoints. You can find the configurations of all endpoints for a Data App in `http_endpoints/config.json`. | ||
|
|
||
| The following is an example configuration of `config.json`. In this example, there are two endpoints for this Data App. | ||
|
|
||
| ```json | ||
| [ | ||
| { | ||
| "name": "<Endpoint name1>", | ||
| "description": "<Endpoint description1>", | ||
| "method": "<HTTP method1>", | ||
| "endpoint": "<Endpoint path1>", | ||
| "data_source": { | ||
| "cluster_id": <Cluster ID1> | ||
| }, | ||
| "params": [], | ||
| "settings": { | ||
| "timeout": <Endpoint timeout>, | ||
| "row_limit": <Maximum rows> | ||
| }, | ||
| "sql_file": "<SQL file directory1>", | ||
| "type": "sql_endpoint", | ||
| "return_type": "json" | ||
| }, | ||
| { | ||
| "name": "<Endpoint name2>", | ||
| "description": "<Endpoint description2>", | ||
| "method": "<HTTP method2>", | ||
| "endpoint": "<Endpoint path2>", | ||
| "data_source": { | ||
| "cluster_id": <Cluster ID2> | ||
| }, | ||
| "params": [ | ||
| { | ||
| "name": "<Parameter name>", | ||
| "type": "<Parameter type>", | ||
| "required": <0 | 1>, | ||
| "default": "<Parameter default value>" | ||
| } | ||
| ], | ||
| "settings": { | ||
| "timeout": <Endpoint timeout>, | ||
| "row_limit": <Maximum rows> | ||
| }, | ||
| "sql_file": "<SQL file directory2>", | ||
| "type": "sql_endpoint", | ||
| "return_type": "json" | ||
| } | ||
| ] | ||
| ``` | ||
|
|
||
| The description of each field is as follows: | ||
|
|
||
| | Field | Type | Description | | ||
| |---------------|--------|-------------| | ||
| | `name` | String | The endpoint name. | | ||
| | `description` | String | (Optional) The endpoint description. | | ||
| | `method` | String | The HTTP method of the endpoint. You can use `GET` to query data or use `POST` to insert data. | | ||
| | `endpoint` | String | The unique path of the endpoint in the Data App. Only letters, numbers, underscores (`_`), and slashes (`/`) are allowed in the path, which must start with a slash (`/`) and end with a letter, number, or underscore (`_`). For example, `/my_endpoint/get_id`. The length of the path must be less than 64 characters.| | ||
| | `cluster_id` | String | The ID of the TiDB cluster for your endpoint. You can get it from the URL of your TiDB cluster. For example, if your cluster URL is `https://tidbcloud.com/console/clusters/1234567891234567890/overview`, the cluster ID is `1234567891234567890`. | | ||
| | `params` | Array | The parameters used in the endpoint. By defining parameters, you can dynamically replace the parameter value in your queries through the endpoint. In `params`, you can define one or multiple parameters. For each parameter, you need to define its `name`, `type`, `required`, and `default` fields. If your endpoint does not need any parameters. You can leave `params` empty such as `"params": []`. | | ||
| | `params.name` | String | The name of the parameter. The name can only include letters, digits, and underscores (`_`) and must start with a letter or an underscore (`_`). | | ||
| | `params.type` | String | The data type of the parameter. Supported values are `string`, `number`, and `boolean`. When using a `string` type parameter, you do not need to add quotation marks (`'` or `"`). For example, `foo` is valid for the `string` type and is processed as `"foo"`, whereas `"foo"` is processed as `"\"foo\""`.| | ||
| | `params.required` | Integer | Specifies whether the parameter is required in the request. Supported values are `0` (not required) and `1` (required). The default value is `0`. | | ||
| | `params.default` | String | The default value of the parameter. Make sure that the value matches the type of parameter you specified. Otherwise, the endpoint returns an error. | | ||
| | `timeout` | Integer | The timeout for the endpoint in milliseconds, which is `5000` by default. You can set it to an integer from `1` to `30000`. | | ||
| | `row_limit` | Integer | The maximum number of rows that the endpoint returns, which is `50` by default. You can set it to an integer from `1` to `2000`. | | ||
| | `sql_file` | String | The SQL file directory for the endpoint. For example, `"sql/GET-v1.sql"`. | | ||
| | `type` | String | The type of the endpoint, which can only be `"sql_endpoint"`. | | ||
| | `return_type` | String | The response format of the endpoint, which can only be `"json"`. | | ||
|
|
||
| ### SQL file configuration | ||
|
|
||
| The SQL file of an endpoint specifies the SQL statements to query data through the endpoint. You can find the endpoint SQL files of a Data App in the `http_endpoints/sql/` directory. For each endpoint, there should be a corresponding SQL file. | ||
|
|
||
| The name of a SQL file is in the `<method>-<endpoint-path>.sql` format, where `<method>` and `<endpoint-path>` must match the `method` and `endpoint` configuration in [`http_endpoints/config.json`](#endpoint-configuration). | ||
|
|
||
| In the SQL file, you can write statements such as table join queries, complex queries, and aggregate functions. The following is an example SQL file. | ||
|
|
||
| ```sql | ||
| /* Getting Started: | ||
| Enter "USE {database};" before entering your SQL statements. | ||
| Type "--your question" + Enter to try out AI-generated SQL queries in the TiDB Cloud console. | ||
| Declare a parameter like "Where id = ${arg}". | ||
| */ | ||
| USE sample_data; | ||
| SELECT | ||
| rank, | ||
| company_name, | ||
| FROM | ||
| global_fortune_500_2018_2022 | ||
| WHERE | ||
| country = ${country}; | ||
| ``` | ||
|
|
||
| When writing a SQL file, pay attention to the following: | ||
|
|
||
| - At the beginning of the SQL file, you need to specify the database in the SQL statements. For example, `USE database_name;`. | ||
|
|
||
| - To define a parameter of the endpoint, you can insert it as a variable placeholder like `${variable-name}` to the SQL statement. | ||
|
|
||
| In the preceding example, `${country}` is used as a parameter of the endpoint. With this parameter, you can specify a desired country to query in your endpoint curl command. | ||
|
|
||
| > **Note:** | ||
| > | ||
| > - The parameter name is case-sensitive. | ||
| > - The parameter cannot be a table name or column name. | ||
| > - The parameter name in the SQL file match the parameter name configured in [`http_endpoints/config.json`](#endpoint-configuration). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.