[FLINK-25820] Introduce Table Store Flink Source #15
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
tsreaper
reviewed
Jan 29, 2022
Comment on lines
65
to
78
| private static void incrementCharArrayByOne(char[] array, int pos) { | ||
| char c = array[pos]; | ||
| c++; | ||
|
|
||
| if (c > '9') { | ||
| c = '0'; | ||
| incrementCharArrayByOne(array, pos - 1); | ||
| } | ||
| array[pos] = c; | ||
| } |
Contributor
There was a problem hiding this comment.
pos might be -1 if this method is called too many times. In this case we should recreate array and increment its length by 1.
Contributor
Author
There was a problem hiding this comment.
Just throw readable exception is OK... There are to many splits...
| public RecordsWithSplitIds<RecordAndPosition<RowData>> fetch() throws IOException { | ||
| checkSplitOrStartNext(); | ||
|
|
||
| // pool first, avoid thread safety issues |
Contributor
There was a problem hiding this comment.
Not thread safety. This is because batches can only be fetched one by one, so we use a pool to restrict the reader thread not to fetch too many batches at the same time.
Contributor
Author
There was a problem hiding this comment.
pool first, pool size is 1, the underlying implementation does not allow multiple batches to be read at the same time
Comment on lines
66
to
69
| assertSplit(splits.get(0), "0000000001", 2, 0, Arrays.asList("f3", "f4", "f5")); | ||
| assertSplit(splits.get(1), "0000000002", 2, 1, Collections.singletonList("f6")); | ||
| assertSplit(splits.get(2), "0000000003", 1, 0, Arrays.asList("f0", "f1")); | ||
| assertSplit(splits.get(3), "0000000004", 1, 1, Collections.singletonList("f2")); |
Contributor
There was a problem hiding this comment.
Add cases to test carrying.
Contributor
Author
There was a problem hiding this comment.
What do you mean?
Contributor
There was a problem hiding this comment.
I mean add cases for example 9 -> 10 or 99 -> 100. See https://en.wikipedia.org/wiki/Carry_(arithmetic)
JingsongLi
force-pushed
the
store_source
branch
from
916136c to
e1ebf5d
Compare
JingsongLi
force-pushed
the
store_source
branch
from
e1ebf5d to
aeff369
Compare
JingsongLi
force-pushed
the
store_source
branch
from
aeff369 to
9965a9b
Compare
tsreaper
approved these changes
Feb 14, 2022
Alibaba-HZY
pushed a commit
to Alibaba-HZY/incubator-paimon
that referenced
this pull request
Apr 11, 2023
parent 2a62022 author Alibaba-HZY <daowu.hzy@cainiao.com> 1681226598 +0800 committer Alibaba-HZY <daowu.hzy@cainiao.com> 1681226783 +0800 # 这是一个 13 个提交的组合。 parent 2a62022 author Alibaba-HZY <daowu.hzy@cainiao.com> 1681226598 +0800 committer Alibaba-HZY <daowu.hzy@cainiao.com> 1681226730 +0800 # 这是一个 8 个提交的组合。tree 46928987599714b071489a7b6d4957049e6ded7a parent 2a62022 author Alibaba-HZY <daowu.hzy@cainiao.com> 1681226598 +0800 committer Alibaba-HZY <daowu.hzy@cainiao.com> 1681226598 +0800 [core] Add streaming read from option (apache#778) # 这是提交说明 #8： [core] add test (apache#778) # 这是提交说明 apache#11： Update paimon-core/src/main/java/org/apache/paimon/CoreOptions.java Co-authored-by: Nicholas Jiang <programgeek@163.com> # 这是提交说明 apache#12： Update paimon-core/src/main/java/org/apache/paimon/CoreOptions.java Co-authored-by: Nicholas Jiang <programgeek@163.com> # 这是提交说明 apache#13： Update paimon-core/src/main/java/org/apache/paimon/CoreOptions.java Co-authored-by: Nicholas Jiang <programgeek@163.com> # 这是提交说明 apache#15： Update paimon-flink/paimon-flink-common/src/main/java/org/apache/paimon/flink/AbstractFlinkTableFactory.java Co-authored-by: Nicholas Jiang <programgeek@163.com> # 这是提交说明 apache#16： Update paimon-flink/paimon-flink-common/src/main/java/org/apache/paimon/flink/AbstractFlinkTableFactory.java Co-authored-by: Nicholas Jiang <programgeek@163.com> # 这是提交说明 apache#17： Update paimon-flink/paimon-flink-common/src/main/java/org/apache/paimon/flink/AbstractFlinkTableFactory.java Co-authored-by: Nicholas Jiang <programgeek@163.com> # 这是提交说明 apache#18： [core] add test (apache#778) # 这是提交说明 apache#19： merger sdda 这是一个 3 个提交的组合。 [core] commit1 (apache#778) [core] commit2 (apache#778) [core] commit3 (apache#778) [core] do commit 1(apache#778) [core] do commit 2(apache#778) # 这是提交说明 apache#20： sdda 这是一个 3 个提交的组合。 [core] commit1 (apache#778) [core] commit2 (apache#778) [core] commit3 (apache#778)
YetiCuzMountain
pushed a commit
to YetiCuzMountain/paimon
that referenced
this pull request
Jul 9, 2025
YetiCuzMountain
pushed a commit
to YetiCuzMountain/paimon
that referenced
this pull request
Jul 9, 2025
[apache#15] when creating paimon table encontered with a duplicated managed hive... See merge request realtime/paimon!6
YetiCuzMountain
pushed a commit
to YetiCuzMountain/paimon
that referenced
this pull request
Jul 9, 2025
…s exception should...
YetiCuzMountain
pushed a commit
to YetiCuzMountain/paimon
that referenced
this pull request
Jul 9, 2025
[apache#15][IDEV-REALTM-44] if table exist but instead of paimon, this exception should... See merge request realtime/paimon!7
tub
pushed a commit
to tub/paimon
that referenced
this pull request
Jan 13, 2026
…ry_main PRODENAB-149 fix infinite loop when a field is a unparseable value us…
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Introduce FLIP-27 source implementation for table file store.