Skip to content

[improvement] Simplify scan node logic: removing _materialized_blocks…#11060

Closed
jacktengg wants to merge 3 commits intoapache:masterfrom
jacktengg:refactor-scan-node-queue
Closed

[improvement] Simplify scan node logic: removing _materialized_blocks…#11060
jacktengg wants to merge 3 commits intoapache:masterfrom
jacktengg:refactor-scan-node-queue

Conversation

@jacktengg
Copy link
Contributor

… and transfer thread

Proposed changes

Issue Number: close #xxx

Problem Summary:

Describe the overview of changes.

Checklist(Required)

  1. Does it affect the original behavior: (Yes/No/I Don't know)
  2. Has unit tests been added: (Yes/No/No Need)
  3. Has document been added or modified: (Yes/No/No Need)
  4. Does it need to update dependencies: (Yes/No)
  5. Are there any changes that cannot be rolled back: (Yes/No)

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@jacktengg jacktengg force-pushed the refactor-scan-node-queue branch 2 times, most recently from e3fac25 to eb39071 Compare July 25, 2022 12:39
@jacktengg jacktengg force-pushed the refactor-scan-node-queue branch from eb39071 to 3e27ef6 Compare August 2, 2022 08:08
@jacktengg jacktengg marked this pull request as ready for review August 2, 2022 08:09
eos = true;
status = Status::Cancelled("Cancelled");
LOG(INFO) << "Scan thread cancelled, cause query done, maybe reach limit.";
LOG(INFO) << "Scan thread cancelled.";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this log can be removed.

@morningman
Copy link
Contributor

Could you please explain more about the threading model of this olap scan node.
For example:

  1. How to determine the concurrency.
  2. How to submit the scanner thread.
  3. How to transfer the block from scanner to scan node.

So that other developer can easily understand its logic

@jacktengg jacktengg force-pushed the refactor-scan-node-queue branch 2 times, most recently from 1c5e09f to d1b4201 Compare August 3, 2022 08:00
@jacktengg jacktengg force-pushed the refactor-scan-node-queue branch from d1b4201 to 3ab680c Compare August 4, 2022 02:18
@morningman morningman closed this Sep 18, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants