Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Added support hardswish function #100

Merged
merged 12 commits into from
Apr 2, 2024
Merged

Added support hardswish function #100

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
e659c35
Added support for HardSwish function
shaharelys Mar 28, 2024
4109be3
Adjust test directives for HardSwish to expect failure for integer ty…
shaharelys Mar 28, 2024
39c2a55
Merge branch 'main' into added_support_hardswish_function
Borda Mar 30, 2024
4ea0abc
Merge branch 'main' into added_support_hardswish_function
t-vi Apr 1, 2024
cb9d941
Merge remote-tracking branch 'upstream/main' into added_support_hards…
shaharelys Apr 1, 2024
4f1e1b4
Refine hardswish tests to exclude integer types on CUDA
shaharelys Apr 1, 2024
90515e3
Merge branch 'added_support_hardswish_function' of https://github.com…
shaharelys Apr 1, 2024
f804a90
Merge remote-tracking branch 'upstream/main' into added_support_hards…
shaharelys Apr 1, 2024
fa58968
Refine hardswish to support only floating point dtypes and update OpInfo
shaharelys Apr 1, 2024
50c8480
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Apr 1, 2024
a460236
Implement dtype check for hardswish to only allow floating point types
shaharelys Apr 1, 2024
7d0a9bb
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Apr 1, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
19 changes: 1 addition & 18 deletions thunder/tests/opinfos.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1608,32 +1608,15 @@ def hardswish_error_generator(op, device, dtype=torch.float32, **kwargs):
sample_input_generator=elementwise_unary_generator,
error_input_generator=hardswish_error_generator,
torch_reference=_elementwise_unary_torch(torch.nn.functional.hardswish),
dtypes=(datatypes.floating,),
test_directives=(
# PyTorch does not support bool for both CPU and CUDA hardswish
DecorateInfo(
pytest.mark.xfail,
"test_core_vs_torch_consistency",
dtypes=(datatypes.bool8,),
),
# PyTorch does not support complex types for both the CPU and CUDA hardswish
DecorateInfo(
pytest.mark.xfail,
"test_core_vs_torch_consistency",
dtypes=(datatypes.complexfloating,),
),
# PyTorch does not support CPU Half hardswish
DecorateInfo(
pytest.mark.xfail,
"test_core_vs_torch_consistency",
dtypes=(datatypes.float16,),
devicetypes=(devices.DeviceType.CPU,),
),
# PyTorch does not support integer types for both the CPU and CUDA hardswish
DecorateInfo(
pytest.mark.xfail,
"test_core_vs_torch_consistency",
dtypes=(datatypes.int16, datatypes.int32, datatypes.int64, datatypes.int8, datatypes.uint8),
),
# TODO: we might have a tolerance issue here with hardsiwsh, a function of relu6
DecorateInfo(
pytest.mark.xfail(strict=False),
Expand Down
3 changes: 3 additions & 0 deletions thunder/torch/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1211,6 +1211,9 @@ def relu6(a: TensorProxy, /, inplace: bool = False) -> TensorLike:
@torchsymbol(torch.nn.functional.hardswish, id="torch.hardswish", is_method=False)
def hardswish(a: TensorProxy, /, inplace: bool = False) -> TensorLike:
t-vi marked this conversation as resolved.
Show resolved Hide resolved
utils.check(not inplace, lambda: f"hardswish only supports inplace=False", exception_type=NotImplementedError)
utils.check(a.dtype in (torch.float16, torch.float32, torch.float64),
t-vi marked this conversation as resolved.
Show resolved Hide resolved
lambda: f"hardswish only supports floating point dtypes, got {a.dtype}",
exception_type=TypeError)
return a * relu6(a + 3) / 6


Expand Down