[HUDI-5102] source operator(monitor and reader) support user uid #7011
Conversation
source operator(monitor and reader) support user uid
1032851561
changed the title
nsivabalan
added
priority:minor
|
@hudi-bot run azure |
| .setParallelism(1) | ||
| .keyBy(MergeOnReadInputSplit::getFileId) | ||
| .transform("split_reader", typeInfo, factory) | ||
| .uid("uid_split_reader_" + conf.getString(FlinkOptions.TABLE_NAME)) | ||
| .setParallelism(conf.getInteger(FlinkOptions.READ_TASKS)); | ||
| return new DataStreamSource<>(source); |
There was a problem hiding this comment.
Can we reuse the util method: opItentifier ?
There was a problem hiding this comment.
Of course, I have optimized it.
There was a problem hiding this comment.
Can you apply the review patch though ~
There was a problem hiding this comment.
Did you notice that i have pasted a patch under the comments, you can apply the patch with command:
git apply xxx.patchreuse the util method readOpIdentifier
|
fix_op_uid.patch.zip and change your commit title into: |
1032851561
changed the title
1032851561
changed the title
1032851561
changed the title
|
@hudi-bot run azure |
|
Hi, can you rebase with the latest master and force push with the branch ? |
I have create new branch "fix_op_uid" and pull request again #7085 |
|
Close because it is fixed in #7085. |
source operator(monitor and reader) support user uid
Tips
What is the purpose of the pull request
https://issues.apache.org/jira/browse/HUDI-5090
Brief change log
refer to org.apache.hudi.sink.utils.Pipelines#hoodieStreamWrite , add user uid for StreamReadMonitoringFunction and StreamReadOperator
Verify this pull request
(Please pick either of the following options)
This pull request is a trivial rework / code cleanup without any test coverage.
(or)
This pull request is already covered by existing tests, such as (please describe tests).
(or)
This change added tests and can be verified as follows:
(example:)
Committer checklist
Has a corresponding JIRA in PR title & commit
Commit message is descriptive of the change
CI is green
Necessary doc changes done or have another open PR
For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.