Skip to content

[feature](tools)profile viewer#14429

Merged
yiguolei merged 2 commits intoapache:masterfrom
englefly:profile-viewer
Nov 21, 2022
Merged

[feature](tools)profile viewer#14429
yiguolei merged 2 commits intoapache:masterfrom
englefly:profile-viewer

Conversation

@englefly
Copy link
Contributor

@englefly englefly commented Nov 20, 2022

Proposed changes

It is a painful work to read profile, especially there are multi-parallel instances.
This tool helps us to grasp the main information of profile in a graphical view.

The profile is represented by a tree.
Sql operation nodes contains operation type(join, scan...), its node id, its fragment id. The number on the arrow edge means how many rows output by child node. This tool will sum the output rows of the same node in multi-parallel instances, that is if there are 4 parallel instance, and each ScanNode on lineitem table output 10 rows, the label on the arrow beginning with ScanNode(lineitem) is 40.

Here is a demo for tpch Q2
tpch q2 profile viewer

Issue Number: close #xxx

Problem summary

Describe your changes.

Checklist(Required)

  1. Does it affect the original behavior:
    • Yes
    • No
    • I don't know
  2. Has unit tests been added:
    • Yes
    • No
    • No Need
  3. Has document been added or modified:
    • Yes
    • No
    • No Need
  4. Does it need to update dependencies:
    • Yes
    • No
  5. Are there any changes that cannot be rolled back:
    • Yes (If Yes, please explain WHY)
    • No

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

Copy link
Contributor

@morningman morningman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Better to add a document for this tool.
you can add a document on Doris website to just point to the README file of this tools.
So that you can just maintain the README content in future, and no need to modify
the doc on website every time you update this tool.

@hello-stephen
Copy link
Contributor

hello-stephen commented Nov 20, 2022

TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 35.44 seconds
load time: 431 seconds
storage size: 17152530732 Bytes
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20221120161208_clickbench_pr_49419.html

Copy link
Contributor

@yiguolei yiguolei left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@yiguolei yiguolei merged commit 0613ccd into apache:master Nov 21, 2022
FreeOnePlus pushed a commit to FreeOnePlus/doris that referenced this pull request Nov 27, 2022
It is a painful work to read profile, especially there are multi-parallel instances.
This tool helps us to grasp the main information of profile in a graphical view.

The profile is represented by a tree.
Sql operation nodes contains operation type(join, scan...), its node id, its fragment id. The number on the arrow edge means how many rows output by child node. This tool will sum the output rows of the same node in multi-parallel instances, that is if there are 4 parallel instance, and each ScanNode on lineitem table output 10 rows, the label on the arrow beginning with ScanNode(lineitem) is 40.

Here is a demo for tpch Q2
tpch q2 profile viewer

Issue Number: close #xxx
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants