Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run examine automatically when hit unsupported operator failure #270

Open
parthmannan opened this issue Apr 24, 2024 · 0 comments
Open

Run examine automatically when hit unsupported operator failure #270

parthmannan opened this issue Apr 24, 2024 · 0 comments
Labels
enhancement New feature or request

Comments

@parthmannan
Copy link
Collaborator

🚀 Feature

examine is a very useful and clean method that analyzes the network and reports the unsupported operators. When Thunder hits an unsupported operator, it would be very useful to have Thunder automatically run examine internally with the model and the inputs it already has and report the examine output along with the error.

Motivation

As Thunder grows in adoption, we are seeing it being used on models with operators which are not currently supported. Today, Thunder throws an error that's not very clear that the error is attributed to an unsupported operator.
Example, see comment from a contributor below -- #93 (comment)

Another motivation for this comes from more complex libraries like NeMo where the model builder and the dataloader are not often in the same piece of code and is not easily accessible as they use PyTorch Lightning Trainer etc. While there is an option to create dummy inputs to run examine, this creates a layer of friction between adoption. Often, the engineer running the model is not the same as the one who built the model and may not even have the understanding of the input data expected by the model.

CC - @mruberry @tfogal

@parthmannan parthmannan added the enhancement New feature or request label Apr 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants