[Feature][doc][Connector-V2][Common] Add Common connector documentation #5453
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
PTAL: @liugddx @EricJoy2048 |
EricJoy2048
reviewed
Oct 16, 2023
| | Name | Type | Required | Default | Description | | ||
| |-------------------|--------|----------|---------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | ||
| | source_table_name | String | No | - | When `source_table_name` is not specified, the current plug-in processes the data set `dataset` output by the previous plugin in the configuration file <br/> When `source_table_name` is specified, the current plug-in is processing the data set corresponding to this parameter. | | ||
| | parallelism | Int | No | - | When `parallelism` is not specified, the `parallelism` in env is used by default. <br/>When parallelism is specified, it will override the parallelism in env. | |
There was a problem hiding this comment.
At present, we do not support setting the parallelism for Sink separately. I think we can remove this parameter
3 tasks
EricJoy2048
reviewed
Jan 12, 2024
| | parallelism | int | no | - | | ||
| | Name | Type | Required | Default | Description | | ||
| |-------------------|--------|----------|---------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | ||
| | source_table_name | String | No | - | When `source_table_name` is not specified, the current plug-in processes the data set `dataset` output by the previous plugin in the configuration file <br/> When `source_table_name` is specified, the current plug-in is processing the data set corresponding to this parameter. | |
There was a problem hiding this comment.
Some important caveat should be added: source_table_name and result_table_name are only used to represent the upstream and downstream relationships of the connector. They should not be treated as in-memory datasets. No data is stored in them.
| | parallelism | int | no | - | | ||
| | Name | Type | Required | Default | Description | | ||
| |-------------------|--------|----------|---------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | ||
| | result_table_name | String | No | - | When `result_table_name` is not specified, the data processed by this plugin will not be registered as a data set `(dataStream/dataset)` that can be directly accessed by other plugins, or called a temporary table `(table)` <br/>When `result_table_name` is specified, the data processed by this plugin will be registered as a data set `(dataStream/dataset)` that can be directly accessed by other plugins, or called a temporary table `(table)` . The data set `(dataStream/dataset)` registered here can be directly accessed by other plugins by specifying `source_table_name` . | |
| | source_table_name | string | no | - | | ||
| | Name | Type | Required | Default | Description | | ||
| |-------------------|--------|----------|---------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | ||
| | result_table_name | String | No | - | When `source_table_name` is not specified, the current plug-in processes the data set `(dataset)` output by the previous plug-in in the configuration file; <br/>When `source_table_name` is specified, the current plugin is processing the data set corresponding to this parameter. | |
|
PTAL : @Carl-Zhou-CN @hailin0 @EricJoy2048 |
EricJoy2048
reviewed
Apr 23, 2024
|
|
||
| When `result_table_name` is specified, the data processed by this plugin will be registered as a data set `(dataset)` that can be directly accessed by other plugins, or called a temporary table `(table)` . The dataset registered here can be directly accessed by other plugins by specifying `source_table_name` . | ||
| transform { |
There was a problem hiding this comment.
Need add a example for Multiple Tables.
There was a problem hiding this comment.
Need add a example for Multiple Tables.
@EricJoy2048 Do you have a specific reference case let me add
EricJoy2048
approved these changes
Jun 14, 2024
hailin0
approved these changes
Jun 15, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Purpose of this pull request
Check list
New License Guide
release-note.