Add non-recursive module.to_empty option

[mikaylagawarecki]

Fixes #97049, related to #104187

Stack from ghstack:

• -> Add non-recursive module.to_empty option #104197

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/104197

• 📄 Preview Python docs built from this PR
• 📄 Preview C++ docs built from this PR
• ❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit a0116cd:
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[albanD]

SGTM
Note that this will break any Module that re-defines _apply with the current signature. No one should be doing that but there is a non-zero probability it is happening in the wild.

[mikaylagawarecki]

@pytorchbot merge

[mikaylagawarecki]

@albanD ah that is true, hm is there any way to workaround this?

(just to check it only breaks those overrides of _apply that don't define *args and **kwargs is that correct?)

Upon looking it seems like we do override _apply within nn modules let me fix these

[pytorchmergebot]

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

[pytorchmergebot]

The merge job was canceled. If you believe this is a mistake,then you can re trigger it through pytorch-bot.

[mikaylagawarecki]

this override technically already has the recurse=False behavior and never calls super()._apply, but adding the kwarg here just so nn.ModuleWrapper can compose with nn.Module.to_empty()

[mikaylagawarecki]

@pytorchbot merge

[pytorchmergebot]

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

[albanD]

(just to check it only breaks those overrides of _apply that don't define *args and **kwargs is that correct?)

Correct!
Should we change these to *args, **kwargs btw?

[mikaylagawarecki]

@albanD should we also make that change for other methods other than _apply? I feel as if that might apply to most of our nn.Module methods which are probably overriden in some of our core modules.

I can do this in a follow up if that makes sense!

[albanD]

Yes that can be a follow up for sure.
Also our internal Module should not override these these days I suspect and should be using the proper hooks!

[mikaylagawarecki]

Hmm true, does that mean we should have hooks for private _apply 🤔