Skip to content

Zero Optimizer crashes if a param_group has fewer number of elements than the data parallel size #129

@samyam

Description

@samyam

If a param_group has fewer parameter elements than the size of data parallelism, then certain GPUs will have empty partitions causing it to crash

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions