Skip to content

[Fix] permute fusion number of forward output and backward input is not match#161

Merged
wangye805 merged 1 commit into
ROCm:devfrom
RuibinCheung:fix_permute
Apr 1, 2025
Merged

[Fix] permute fusion number of forward output and backward input is not match#161
wangye805 merged 1 commit into
ROCm:devfrom
RuibinCheung:fix_permute

Conversation

@RuibinCheung
Copy link
Copy Markdown
Contributor

@RuibinCheung RuibinCheung commented Mar 31, 2025

Description

Fix permute fusion number of forward output and backward input is not match when inp.numel() is zero.

This is a nvidia's upstream code bug. And it was fixed on PR NVIDIA/TransformerEngine#1468. But this PR involves a feature which relates to PTX. It can't cherry-pick it directly now.

It will return two tensor when expert of MoE dispatched token is 0 but backward function only accepts one tensor. It will raise a runtime error because the number of parameters and arguments is not matched.

Type of change

  • Documentation change (change only to the documentation, either a fix or a new content)
  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Infra/Build change
  • Code refractor

Changes

Please list the changes introduced in this PR:

  • Change A
  • Change B

Checklist:

  • I have read and followed the contributing guidelines
  • The functionality is complete
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes

@RuibinCheung RuibinCheung changed the title [Fix permute fusion number of forward output and backward input is not match [Fix[ permute fusion number of forward output and backward input is not match Mar 31, 2025
@RuibinCheung
Copy link
Copy Markdown
Contributor Author

@wangye805 PTAL

@RuibinCheung RuibinCheung changed the title [Fix[ permute fusion number of forward output and backward input is not match [Fix] permute fusion number of forward output and backward input is not match Apr 1, 2025
@wangye805 wangye805 self-requested a review April 1, 2025 04:23
Copy link
Copy Markdown
Collaborator

@wangye805 wangye805 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@wangye805 wangye805 merged commit 3d2a780 into ROCm:dev Apr 1, 2025
@RuibinCheung RuibinCheung deleted the fix_permute branch April 1, 2025 08:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants