Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(source): build connector when calling next #1746

Merged
merged 6 commits into from
Apr 11, 2022
Merged

Conversation

tabVersion
Copy link
Contributor

What's changed and what's your intention?

PLEASE DO NOT LEAVE THIS EMPTY !!!

Please explain IN DETAIL what the changes are in this PR and why they are needed:

in prev impl, connection to upstream is established when source is registered. Here to change that into a future and await when calling next, which means no connection is built until there is data demand.

  • Summarize your change (mandatory)
  • How does this PR work? Need a brief introduction for the changed logic (optional)
  • Describe clearly one logical change and avoid lazy messages (optional)
  • Describe any limitations of the current code (optional)

Checklist

  • I have written necessary docs and comments
  • I have added necessary unit tests and integration tests

Refer to a related PR or issue link (optional)

@CLAassistant
Copy link

CLAassistant commented Apr 11, 2022

CLA assistant check
All committers have signed the CLA.

@github-actions github-actions bot added the type/fix Bug fix label Apr 11, 2022
Copy link
Member

@BugenZhao BugenZhao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Comment on lines +44 to +45
/// the future that builds stream_reader. It is required because source should not establish
/// connections to the upstream before `next` is called
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can avoid this when we migrate SourceExecutor to executor v2. 😁

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

next time for sure 👍

The migration will happen after making source running!

Signed-off-by: tabVersion <tabvision@bupt.icu>
@skyzh skyzh enabled auto-merge (squash) April 11, 2022 03:27
@codecov
Copy link

codecov bot commented Apr 11, 2022

Codecov Report

Merging #1746 (db232ea) into main (266ca24) will decrease coverage by 0.00%.
The diff coverage is 65.62%.

@@            Coverage Diff             @@
##             main    #1746      +/-   ##
==========================================
- Coverage   71.16%   71.16%   -0.01%     
==========================================
  Files         600      600              
  Lines       77727    77743      +16     
==========================================
+ Hits        55317    55328      +11     
- Misses      22410    22415       +5     
Flag Coverage Δ
rust 71.16% <65.62%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
src/stream/src/executor/source.rs 75.35% <65.62%> (+0.98%) ⬆️
src/meta/src/manager/stream_clients.rs 88.88% <0.00%> (-7.41%) ⬇️
src/connector/src/filesystem/file_common.rs 80.35% <0.00%> (-0.45%) ⬇️
.../src/executor/managed_state/aggregation/extreme.rs 90.13% <0.00%> (-0.28%) ⬇️

📣 Codecov can now indicate which changes are the most critical in Pull Requests. Learn more

@skyzh
Copy link
Contributor

skyzh commented Apr 11, 2022

image

setup timed out, need merge #1752 first

@skyzh skyzh merged commit 1bad1af into main Apr 11, 2022
@skyzh skyzh deleted the tab/source-build-async branch April 11, 2022 07:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/fix Bug fix
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants