Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AD Cleanup: Use a temporary accumulation inst such as IRAccumulate instead of explicitly loading, adding and storing differential values in reverse mode #4211

Open
saipraveenb25 opened this issue May 22, 2024 · 1 comment
Assignees
Labels
goal:forward looking Feature needed at a later date, not connected to a specific use case. kind:cleanup tech debt and rough edges kind:enhancement a desirable new feature, option, or behavior

Comments

@saipraveenb25
Copy link
Collaborator

We currently accumulate derivatives in a somewhat explicit manner by loading the current differential value, adding the new derivative contribution and then storing this back.

This makes our code fairly messy, and impedes potential optimizations, since accumulation instructions (i.e. +=) can be reordered, but standard loads and stores can't.
We can simply lower these instructions into standard load/add/store instructions after optimizations are performed.
Further, this allows Slang to explicitly emit accumulate instructions on targets that support it (for further downstream optimizations).

Note: IRAccumulate can also have a corresponding IRConsume which would read the derivative and write back a 0. This wouldn't be too helpful other than code clarity, since they can't be reordered. However subsequent consume instructions to the same variable/address can be removed as an optimization (though this is likely to occur anyway with standard loads/stores)

@saipraveenb25 saipraveenb25 added kind:enhancement a desirable new feature, option, or behavior kind:cleanup tech debt and rough edges goal:forward looking Feature needed at a later date, not connected to a specific use case. labels May 22, 2024
@saipraveenb25 saipraveenb25 added this to the Q3 2024 (Summer) milestone May 22, 2024
@saipraveenb25 saipraveenb25 self-assigned this May 22, 2024
@csyonghe
Copy link
Collaborator

This is a good idea and it will keep our IR more compact and this is good for compiler perf as well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
goal:forward looking Feature needed at a later date, not connected to a specific use case. kind:cleanup tech debt and rough edges kind:enhancement a desirable new feature, option, or behavior
Projects
None yet
Development

No branches or pull requests

2 participants