Skip to content

Commit

Permalink
[DTensor] Replaced neg dim normalization with assert in helper (#114141)
Browse files Browse the repository at this point in the history
This is a replacement for #113922. I think we can still leave the check for negative shard dimension in `compute_local_shape_and_global_offset` and replace the normalization logic with an assert. This should provide us a stack trace to see which user-facing API did not normalize the dim as expected.
Pull Request resolved: #114141
Approved by: https://github.com/wanchaol
ghstack dependencies: #113919, #113924, #114134, #113925, #113930
  • Loading branch information
awgu authored and pytorchmergebot committed Nov 21, 2023
1 parent d70857b commit fb25fd6
Showing 1 changed file with 4 additions and 2 deletions.
6 changes: 4 additions & 2 deletions torch/distributed/_tensor/_utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -145,8 +145,10 @@ def compute_global_tensor_info(
if placement.is_shard():
shard_placement = cast(Shard, placement)
if shard_placement.dim < 0:
# normalize shard dim to be positive
shard_placement.dim += len(tensor_shape)
raise AssertionError(
"Shard placements should have negative dims normalized in "
f"the user-facing APIs: {shard_placement}"
)
shard_dim = shard_placement.dim

assert (
Expand Down

0 comments on commit fb25fd6

Please sign in to comment.