Skip to content

[feature](Nereids): Add subgraph enumerator#14291

Merged
jackwener merged 1 commit intoapache:masterfrom
keanji-x:add-subgraph-enumerator
Nov 18, 2022
Merged

[feature](Nereids): Add subgraph enumerator#14291
jackwener merged 1 commit intoapache:masterfrom
keanji-x:add-subgraph-enumerator

Conversation

@keanji-x
Copy link
Copy Markdown
Contributor

@keanji-x keanji-x commented Nov 15, 2022

Signed-off-by: xiejiann jianxie0@gmail.com

Proposed changes

Issue Number: close #xxx

Add subgraph enumerator to find the best plan

Problem summary

For DPHyp, we need an enumerator for all csg-cmp pairs to find the best plan

Checklist(Required)

  1. Does it affect the original behavior:
    • Yes
    • No
    • I don't know
  2. Has unit tests been added:
    • Yes
    • No
    • No Need
  3. Has document been added or modified:
    • Yes
    • No
    • No Need
  4. Does it need to update dependencies:
    • Yes
    • No
  5. Are there any changes that cannot be rolled back:
    • Yes (If Yes, please explain WHY)
    • No

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@keanji-x keanji-x marked this pull request as draft November 15, 2022 11:20
@hello-stephen
Copy link
Copy Markdown
Contributor

hello-stephen commented Nov 16, 2022

TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 34.95 seconds
load time: 473 seconds
storage size: 17150399788 Bytes
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20221117083734_clickbench_pr_47781.html

@keanji-x keanji-x force-pushed the add-subgraph-enumerator branch from 281ed49 to 7e699a5 Compare November 16, 2022 05:59
@keanji-x keanji-x marked this pull request as ready for review November 16, 2022 06:04
@keanji-x keanji-x force-pushed the add-subgraph-enumerator branch from 7e699a5 to c2a6f18 Compare November 17, 2022 02:56
Signed-off-by: xiejiann <jianxie0@gmail.com>
@keanji-x keanji-x force-pushed the add-subgraph-enumerator branch from c2a6f18 to ab9a051 Compare November 17, 2022 05:57
* This is a class for iterating all true subset of a bitset, referenced in
* https://groups.google.com/forum/#!msg/rec.games.chess/KnJvBnhgDKU/yCi5yBx18PQJ
*/
public class SubsetIterator implements Iterable<BitSet> {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Future ticket: use hand-write bitmap instead of bitset.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I will do it in next pr. I plan to implement it by generics. That is if the tables less 64 , we use long. Otherwise we use bitset.

Comment on lines +189 to +193
private BitSet cloneBitSet(BitSet bitSet) {
BitSet cloned = new BitSet();
cloned.or(bitSet);
return cloned;
}
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
private BitSet cloneBitSet(BitSet bitSet) {
BitSet cloned = new BitSet();
cloned.or(bitSet);
return cloned;
}
private BitSet newBitSet(BitSet ...bitSets) {
BitSet cloned = new BitSet();
for (BitSet bitSet : bitSets) {
cloned.or(bitSet);
}
return cloned;
}

Comment on lines +84 to +86
BitSet newCsg = new BitSet();
newCsg.or(csg);
newCsg.or(subset);
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

newBitSet()

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Nov 17, 2022
@github-actions
Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

1 similar comment
@github-actions
Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

1 similar comment
@github-actions
Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

@jackwener jackwener merged commit 9e25aa8 into apache:master Nov 18, 2022
FreeOnePlus pushed a commit to FreeOnePlus/doris that referenced this pull request Nov 27, 2022
Add subgraph enumerator to find the best plan

For DPHyp, we need an enumerator for all csg-cmp pairs to find the best plan
@keanji-x keanji-x deleted the add-subgraph-enumerator branch November 30, 2022 05:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. area/nereids reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants