Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Only populate grad accumulator to var mapping for find_unused_parameters=True in DDP #45942

Closed
wants to merge 5 commits into from

Commits on Oct 7, 2020

  1. Only populate grad accumulator to var mapping for find_unused_paramet…

    …ers=True in DDP
    
    We only need to keep track of this for traversing the autograd graph
    when find_unused_parameters=True. Without that, we populate and keep this
    mapping in memory, which occupies sizeof(pointer) * number of grad accumulators
    of extra memory.
    
    Differential Revision: [D24154407](https://our.internmc.facebook.com/intern/diff/D24154407/)
    
    [ghstack-poisoned]
    rohan-varma committed Oct 7, 2020
    Configuration menu
    Copy the full SHA
    64b2203 View commit details
    Browse the repository at this point in the history
  2. Update on "Only populate grad accumulator to var mapping for find_unu…

    …sed_parameters=True in DDP"
    
    
    We only need to keep track of this for traversing the autograd graph
    when find_unused_parameters=True. Without that, we populate and keep this
    mapping in memory, which occupies sizeof(pointer) * number of grad accumulators
    of extra memory.
    
    Also renames the variable to something more meaningful. 
    
    Differential Revision: [D24154407](https://our.internmc.facebook.com/intern/diff/D24154407/)
    
    [ghstack-poisoned]
    rohan-varma committed Oct 7, 2020
    Configuration menu
    Copy the full SHA
    c8eaa63 View commit details
    Browse the repository at this point in the history

Commits on Oct 8, 2020

  1. Update on "Only populate grad accumulator to var mapping for find_unu…

    …sed_parameters=True in DDP"
    
    
    We only need to keep track of this for traversing the autograd graph
    when find_unused_parameters=True. Without that, we populate and keep this
    mapping in memory, which occupies sizeof(pointer) * number of grad accumulators
    of extra memory.
    
    Also renames the variable to something more meaningful. 
    
    Differential Revision: [D24154407](https://our.internmc.facebook.com/intern/diff/D24154407/)
    
    [ghstack-poisoned]
    rohan-varma committed Oct 8, 2020
    Configuration menu
    Copy the full SHA
    3f19cf2 View commit details
    Browse the repository at this point in the history

Commits on Oct 11, 2020

  1. Update on "Only populate grad accumulator to var mapping for find_unu…

    …sed_parameters=True in DDP"
    
    
    We only need to keep track of this for traversing the autograd graph
    when find_unused_parameters=True. Without that, we populate and keep this
    mapping in memory, which occupies sizeof(pointer) * number of grad accumulators
    of extra memory.
    
    Also renames the variable to something more meaningful. 
    
    Differential Revision: [D24154407](https://our.internmc.facebook.com/intern/diff/D24154407/)
    
    [ghstack-poisoned]
    rohan-varma committed Oct 11, 2020
    Configuration menu
    Copy the full SHA
    8763c08 View commit details
    Browse the repository at this point in the history

Commits on Oct 13, 2020

  1. Update on "Only populate grad accumulator to var mapping for find_unu…

    …sed_parameters=True in DDP"
    
    
    We only need to keep track of this for traversing the autograd graph
    when find_unused_parameters=True. Without that, we populate and keep this
    mapping in memory, which occupies sizeof(pointer) * number of grad accumulators
    of extra memory.
    
    Also renames the variable to something more meaningful. 
    
    Differential Revision: [D24154407](https://our.internmc.facebook.com/intern/diff/D24154407/)
    
    [ghstack-poisoned]
    rohan-varma committed Oct 13, 2020
    Configuration menu
    Copy the full SHA
    1e29d71 View commit details
    Browse the repository at this point in the history