[Inductor] Fix constant folding and extern kernel mutation tracking bugs

[oulgen]

Stack from ghstack (oldest at bottom):

• apply #115291
• -> [Inductor] Fix constant folding and extern kernel mutation tracking bugs #115908

This PR fixes two bugs

1. Constant folding a triton kernel results in the kernel's inputs to be returned back without any modification. Disable constant folding for triton kernels. Need more investigation
2. NoneLayout buffers should not be deleted as they do not exist

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @aakhundov @ColinPeppler

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/115908

• 📄 Preview Python docs built from this PR
• 📄 Preview C++ docs built from this PR
• ❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 2d221e8 with merge base bf62511 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[aakhundov]

Thanks for the fix!

[oulgen]

So this PR results in matmul input buffers to live longer
OLD LEFT, NEW RIGHT

the difference is
OLD

x = EMPTY()
y = matrix multiply (x...)
z = reinterp(x)
del x

NEW

x = prev_z
del prev_x
y = matrix_multiply(some_old_value...)
z = empty()

so in the new one z unnecessarily lives until the next matrix multiply

[oulgen]

@aakhundov also discovered that
so triton_poi_fused__to_copy_add_mul_pow_tanh_backward_10 used to be:

def triton_(in_out_ptr0, in_ptr0, in_ptr1, xnumel, XBLOCK : tl.constexpr):
    …
    tmp0 = tl.load(in_out_ptr0 + (x0), None).to(tl.float32)
    …
    tl.store(in_out_ptr0 + (x0), tmp28, None)

and now it is

def triton_(in_ptr0, in_ptr1, in_ptr2, out_ptr0, xnumel, XBLOCK : tl.constexpr):
    …
    tmp0 = tl.load(in_ptr0 + (x0), None).to(tl.float32)
    …
    tl.store(out_ptr0 + (x0), tmp28, None)  

[oulgen]

Mutation tracking looks good

[oulgen]

@pytorchbot merge

[pytorchmergebot]

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here