Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Format Python code with psf/black push #284

Closed
wants to merge 2 commits into from

Conversation

github-actions[bot]
Copy link
Contributor

@github-actions github-actions bot commented Oct 15, 2021

There appear to be some python formatting errors in a05e4e3. This pull request
uses the psf/black formatter to fix these issues.

@todo
Copy link

todo bot commented Oct 15, 2021

remove the dependency on torch process group

# TODO remove the dependency on torch process group
if not dist.is_initialized():
torch.distributed.init_process_group(
backend="nccl",
store=_default_store,
rank=get_rank(),


This comment was generated by todo based on a TODO comment in 255d0fc in #284. cc @BaguaSys.

@todo
Copy link

todo bot commented Oct 15, 2021

combine **inplace API

# TODO combine **inplace API
def alltoall_inplace(
tensor: torch.Tensor,
comm=comm.WORLD,
):
"""The in-place version of :func:`alltoall`."""


This comment was generated by todo based on a TODO comment in 255d0fc in #284. cc @BaguaSys.

@todo
Copy link

todo bot commented Oct 15, 2021

(wayi@): Remove this field since SPMD is no longer supported,

# TODO(wayi@): Remove this field since SPMD is no longer supported,
# and also remove all the relevant unnecessary loops.
# Module replication within process (single-process multi device)
self._module_copies = [self.module]
self.bagua_init(optimizers, algorithm, process_group)


This comment was generated by todo based on a TODO comment in 255d0fc in #284. cc @BaguaSys.

@todo
Copy link

todo bot commented Oct 15, 2021

@shjwudp add support for reporting tensor completion order

# TODO: @shjwudp add support for reporting tensor completion order
# so that the autotune service does not rely on tensor registration
# order
rsp = self._bagua_autotune_client.report_metrics(
model_name=self.bagua_module_name,
rank=env.get_rank(),


This comment was generated by todo based on a TODO comment in 255d0fc in #284. cc @BaguaSys.

@todo
Copy link

todo bot commented Oct 15, 2021

sanity check

// TODO: sanity check
let dtype = torch_tensor
.getattr("dtype")
.expect("must pass valid torch tensor")
.repr()?
.to_string();


This comment was generated by todo based on a TODO comment in 255d0fc in #284. cc @BaguaSys.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants