Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Distributed FMM with complete tree structure on each rank #8

Closed
wants to merge 273 commits into from
Closed

Distributed FMM with complete tree structure on each rank #8

wants to merge 273 commits into from

Conversation

gaohao95
Copy link
Contributor

@gaohao95 gaohao95 commented Oct 22, 2019 •
edited
Loading

TODO List:

Must do:

  • Integrate pytest with MPI for CI.
  • Remove exception tag of this branch from CI jobs.
  • Merge constantone test case into test_distributed.py.
  • Add a documentation webpage.
  • Adapt calculate_pot for Sumpy.

Maybe:

  • Make all public API inside __init__.py.
  • Think about whether we can decrease device memory usage for box_mpole_is_used.
  • Unified array with both device and host pointer.

gaohao95 and others added 30 commits November 19, 2017 20:28
@gaohao95
Fix bugs and renaming
1df5400
@gaohao95
Generate level starts
c520319
@gaohao95
Step 2.1 and 2.2
86117a4
@gaohao95
Change CI configuration file
82ee486
@gaohao95
Change CI configuration file
58acf53
@gaohao95
Use two tranversal objects
ebfac89
@gaohao95
Remaining steps of FMM
e224d27
@gaohao95
Fix bug
c36b00f
@gaohao95
Fix bugs
a06d013
@gaohao95
Rearrange interfaces
ef77b4b
@gaohao95
Add more timing
09c20b0
@gaohao95
Use Elementwise kernel for local particle generation
e2e199c
@gaohao95
Pass radii to the local tree
521fe68
@gaohao95
Support target with scale
e3652cb
@gaohao95
Add an argument for traversal builder to use existing box extent
a48a058
@gaohao95
Improve load balancing by DFS order and load prediction
cabadd9
@mattwala
Implement an all-reduce communication pattern.
ac80df8
@mattwala
Minor updates to allreduce.
105f598
@mattwala
Merge branch 'distributed-fmm-global' of gitlab.tiker.net:inducer/box…
0186047 
…tree into distributed-fmm-global
@mattwala
Add a test for the allreduce.
9c084f2
@mattwala
Undo changes to test_tree.py
1384f6e
@mattwala
flake8 fix
88a4c14
@mattwala
s/ValueError/RuntimeError
06d04da
@mattwala
Implement a utility class to compress dense matrices into CSR lists.
ffc4a35
@mattwala
Compute box_to_users.
df00afb
@mattwala
flake8 fix
7c5f986
@mattwala
Move AllReduceCommPattern to boxtree.tools
dc4e890
@mattwala
Generalize MatrixCompressor to compress both lists and matrices.
eb8930e
@mattwala
MaskCompressorKernel: Fix handling of transposed arrays.
9465d68
@mattwala
Remove print statements.
8014525
inducer
inducer reviewed Jun 17, 2021
View reviewed changes
.gitlab-ci.yml Show resolved Hide resolved
boxtree/fmm.py Outdated Show resolved Hide resolved
boxtree/distributed/__init__.py Show resolved Hide resolved
boxtree/distributed/__init__.py Outdated Show resolved Hide resolved
boxtree/distributed/__init__.py Show resolved Hide resolved
@inducer inducer force-pushed the no-tree-in-wrangler branch 2 times, most recently from ddb2065 to 5d13e04 Compare July 1, 2021 03:36
inducer
inducer reviewed Jul 1, 2021
View reviewed changes
Copy link
Owner

@inducer inducer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi Hao, thanks for working on this! Here's a bit more progress on review.

FYI, the "tree-dependent data" PRs:

are done and should be landing soon.

boxtree/distributed/__init__.py Outdated Show resolved Hide resolved
boxtree/distributed/partition.py Show resolved Hide resolved
boxtree/distributed/partition.py Outdated Show resolved Hide resolved
boxtree/distributed/partition.py Outdated Show resolved Hide resolved
boxtree/distributed/partition.py Outdated Show resolved Hide resolved
boxtree/tools.py Outdated Show resolved Hide resolved
boxtree/traversal.py Outdated Show resolved Hide resolved
boxtree/traversal.py Outdated Show resolved Hide resolved
boxtree/tools.py Show resolved Hide resolved
boxtree/fmm.py Outdated Show resolved Hide resolved
inducer
inducer reviewed Jul 1, 2021
View reviewed changes
Copy link
Owner

@inducer inducer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(as discussed during meeting)

boxtree/distributed/calculation.py Outdated Show resolved Hide resolved
inducer
inducer reviewed Aug 12, 2021
View reviewed changes
boxtree/distributed/__init__.py Outdated
calibration_params = \
FMMCostModel.get_unit_calibration_params()

# We need to construct a wrangler in order to access `level_nterms`
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ wrangler being created just for that purpose

inducer
inducer reviewed Oct 12, 2021
View reviewed changes
Copy link
Owner

@inducer inducer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unbelievable. I actually remembered to submit comments this time.

boxtree/tools.py Outdated Show resolved Hide resolved
boxtree/tools.py Show resolved Hide resolved
inducer
inducer reviewed Nov 2, 2021
View reviewed changes
Copy link
Owner

@inducer inducer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! A few more comments below, largely minor. In terms of review progress, I've now read everything except

  • local_traversal
  • local_tree
  • partition
  • traversal

doc/distributed.rst Outdated Show resolved Hide resolved
boxtree/fmm.py Outdated Show resolved Hide resolved
boxtree/fmm.py Show resolved Hide resolved
boxtree/fmm.py Outdated Show resolved Hide resolved
boxtree/fmm.py Outdated Show resolved Hide resolved
boxtree/distributed/__init__.py Show resolved Hide resolved
boxtree/distributed/__init__.py Show resolved Hide resolved
boxtree/distributed/calculation.py Outdated Show resolved Hide resolved
boxtree/distributed/calculation.py Outdated Show resolved Hide resolved
boxtree/distributed/partition.py Outdated Show resolved Hide resolved
@inducer
Copy link
Owner

inducer commented Dec 14, 2021

FYI I just rebased inducer/sumpy#70 to bring it up to date with respect to the latest sumpy (actually just redid the patch, the merge didn't really work).

@inducer inducer deleted the branch inducer:no-tree-in-wrangler December 14, 2021 17:37
@inducer inducer closed this Dec 14, 2021
@inducer
Copy link
Owner

inducer commented Dec 14, 2021

Whoops! I did not mean to close this. I think what happened here is that I merged the wrangler refactor (#29), which served as the base branch of this, and because that branch got deleted, it closed this PR. @gaohao95, could you open a new PR off of your branch that targets main and links to this? Sorry about the fuss.

@gaohao95 gaohao95 mentioned this pull request Dec 15, 2021
Merged
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@inducer inducer inducer left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@gaohao95 @inducer @mattwala