Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[autoparallel] adapt solver with resnet #1583

Merged

Conversation

YuliangLiu0306
Copy link
Contributor

##What is new?
This PR change the following things:

  1. The old version liveness analyser add the parameters and buffers into the memory cost, but it does not has enough information about the sharding strategy. Therefore, we only provide the liveness cycle of nodes, and leave the memory cost computation work to the op handler or strategy constructor.
  2. implement the solver class with the pulp module. Now, it could find dp strategy for resnet with sufficient memory budget and mixed strategy if the memory budget is not enough for the dp strategy.
  3. Add BatchNorm handler to search the strategy for resent.
  4. add the weight and bias memory cost into total memory cost for all handlers.

@FrankLeeeee FrankLeeeee merged commit 82d4376 into hpcaitech:main Sep 13, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants