Add Hpu to the rebuild component list

[jeejakp12]

Numpy array is chosen to be the rebuild component for
HPU. so add it to the backend list.

Signed-off-by: Ayman Yousefayousef@habana.ai
Signed-off-by: Jeeja jeejakp@habana.ai

Fixes #ISSUE_NUMBER

[facebook-github-bot]

🔗 Helpful links

• 🧪  See artifacts and rendered test results at hud.pytorch.org/pr/74738
• Need help or want to give feedback on the CI? Visit our office hours

💊 CI failures summary and remediations

As of commit f72a888 (more details on the Dr. CI page):

---

💚 💚 Looks good so far! There are no failures yet. 💚 💚

---

This comment was automatically generated by Dr. CI (expand for details).

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

[albanD]

Would it be possible to have a test for this? To make sure it works as expected and doesn't break in the future?

[jeejakp12]

Would it be possible to have a test for this? To make sure it works as expected and doesn't break in the future?

@albanD the backend tests are in not in fork, for cpu or gpu this is not valid. any suggestion on where to add it. Below is the test that was reproducing the issue for hpu backend tensor(torch.save cause the problem)
import os
import torch
import torch.distributed as dist
import torch.multiprocessing as mp
import torch.nn as nn
import torch.optim as optim
from torch.distributed.optim import ZeroRedundancyOptimizer
from torch.nn.parallel import DistributedDataParallel as DDP
from habana_frameworks.torch.utils.library_loader import load_habana_module
load_habana_module()
import habana_frameworks.torch.core.hccl

def example(rank, world_size, use_zero):
torch.manual_seed(0)

os.environ['MASTER_ADDR'] = 'localhost'
os.environ['MASTER_PORT'] = '29500'
# create default process group
dist.init_process_group("hccl", rank=rank, world_size=world_size)
rank = torch.distributed.get_rank()
# create local model
model = nn.Sequential(*[nn.Linear(2000, 2000).to('hpu') for _ in range(2)])

# construct DDP model
ddp_model = DDP(model)

# define loss function and optimizer
loss_fn = nn.MSELoss()
if use_zero:
    optimizer = ZeroRedundancyOptimizer(
        ddp_model.parameters(),
        optimizer_class=torch.optim.Adam,
        lr=0.01
    )

# forward pass
outputs = ddp_model(torch.randn(20, 2000).to('hpu'))
labels = torch.randn(20, 2000).to('hpu')
# backward pass
loss_fn(outputs, labels).backward()

# update parameters
optimizer.step()
if use_zero:
  optimizer.consolidate_state_dict()
print(f"params sum is: {sum(model.parameters()).sum()}")

def main():
world_size = 2
print("=== Using ZeroRedundancyOptimizer ===")
mp.spawn(example,
args=(world_size, True),
nprocs=world_size,
join=True)

if name=="main":
main()

[albanD]

Well if there is no CI config that supports this, then this is ok. And your testing will handle it.

[albanD]

@pytorchbot merge this please

[github-actions]

Hey @jeejakp12.
You've committed this PR, but it does not have both a 'release notes: ...' and 'topics: ...' label. Please add one of each to the PR. The 'release notes: ...' label should represent the part of PyTorch that this PR changes (fx, autograd, distributed, etc) and the 'topics: ...' label should represent the kind of PR it is (not user facing, new feature, bug fix, perf improvement, etc). The list of valid labels can be found here for the 'release notes: ...' and here for the 'topics: ...'.
For changes that are 'topic: not user facing' there is no need for a release notes label.