Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(frontend): introduce scheduler error #3156

Merged
merged 2 commits into from
Jun 13, 2022
Merged

feat(frontend): introduce scheduler error #3156

merged 2 commits into from
Jun 13, 2022

Conversation

lmatz
Copy link
Contributor

@lmatz lmatz commented Jun 13, 2022

What's changed and what's your intention?

As per the title.

Checklist

- [ ] I have written necessary docs and comments
- [ ] I have added necessary unit tests and integration tests

  • All checks passed in ./risedev check (or alias, ./risedev c)

Refer to a related PR or issue link (optional)

#2366

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

license-eye has totally checked 842 files.

Valid Invalid Ignored Fixed
840 1 1 0
Click to see the invalid file list
  • src/frontend/src/scheduler/error.rs

src/frontend/src/scheduler/error.rs Show resolved Hide resolved
@lmatz lmatz force-pushed the lz/error branch 2 times, most recently from caf0cd1 to 792f629 Compare June 13, 2022 04:58
@codecov
Copy link

codecov bot commented Jun 13, 2022

Codecov Report

Merging #3156 (3bca26e) into main (3bca26e) will not change coverage.
The diff coverage is n/a.

❗ Current head 3bca26e differs from pull request most recent head 9976542. Consider uploading reports for the commit 9976542 to get more accurate results

@@           Coverage Diff           @@
##             main    #3156   +/-   ##
=======================================
  Coverage   73.66%   73.66%           
=======================================
  Files         739      739           
  Lines      101792   101792           
=======================================
  Hits        74986    74986           
  Misses      26806    26806           
Flag Coverage Δ
rust 73.66% <0.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

📣 Codecov can now indicate which changes are the most critical in Pull Requests. Learn more

@lmatz lmatz marked this pull request as ready for review June 13, 2022 06:15
src/frontend/src/scheduler/distributed/query_manager.rs Outdated Show resolved Hide resolved
src/frontend/src/scheduler/distributed/stage.rs Outdated Show resolved Hide resolved
src/frontend/src/scheduler/distributed/query.rs Outdated Show resolved Hide resolved
src/frontend/src/scheduler/distributed/query.rs Outdated Show resolved Hide resolved
@@ -176,7 +181,7 @@ impl QueryResultFetcher {
.await?;
let mut stream = compute_client.get_data(self.task_output_id.clone()).await?;
while let Some(response) = stream.next().await {
yield DataChunk::from_protobuf(response.to_rw_result()?.get_record_batch()?)?
yield DataChunk::from_protobuf(response.map_err(TonicStatus)?.get_record_batch()?)?;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We may also use an RpcError variant for this and implement From<..> for this so there's no need to map_err here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yield returns RwError, tonic::Status -> SchedulerError::RpcError -> RwError, at least one map is needed I guess?

src/frontend/src/scheduler/distributed/stage.rs Outdated Show resolved Hide resolved
src/frontend/src/scheduler/local.rs Outdated Show resolved Hide resolved
@BugenZhao BugenZhao requested a review from skyzh June 13, 2022 07:06
@lmatz lmatz force-pushed the lz/error branch 2 times, most recently from 4891da6 to 2925733 Compare June 13, 2022 09:19
@liurenjie1024
Copy link
Contributor

Should we introduct a FrontendError rather a SchedulerError? Or do you plan to move query scheduler to another crate?

@lmatz lmatz force-pushed the lz/error branch 2 times, most recently from 96c7f9a to cf4746f Compare June 13, 2022 09:46
@lmatz
Copy link
Contributor Author

lmatz commented Jun 13, 2022

Should we introduce a FrontendError rather than a SchedulerError? Or do you plan to move the query scheduler to another crate?

In the long run, I believe it is very likely to do so. Scheduling is quite basic right now.
Maybe one can have both so that refactoring will be easier later.
BindError also stays in the ErrorCode for the moment.

Copy link
Member

@BugenZhao BugenZhao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We may also introduce an error type for rpc_client and refine some cases. NTFS😄

@lmatz lmatz merged commit 1b749b3 into main Jun 13, 2022
@lmatz lmatz deleted the lz/error branch June 13, 2022 11:35
@BugenZhao BugenZhao mentioned this pull request Jun 14, 2022
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants