Conversation
|
@wconstab you think it makes sense to merge this? Or just leave it as a standalone thing. |
|
@jansel yea I'd definitely merge it in. I will review now. I'm wondering if this is something we'd want to run every nightly or at some interval, or just for manual use. |
|
|
||
| def run_model(model_class, model_path, device): | ||
| m = model_class(device=device) | ||
| def test_fx_profile(self): |
There was a problem hiding this comment.
I guess this is a correctness test rather than something you're using to gather data. Did you confirm if this gets invoked in CI as you wrote it? I think we run all tests from test.py.
There was a problem hiding this comment.
Correct, this is just a correctness test. I verified it locally, I did not check the CI though.
wconstab
left a comment
wconstab
left a comment
There was a problem hiding this comment.
lgtm, and I confirmed there is one new test (157 on your branch vs 156 master) being run in CI although the test names aren't logged.
Depending on what people want to do with this we can talk about automating it in CI to collect stats or whatever. But seems good/useful as-is so please merge.
3 participants
Perhaps others would find this useful. Results as of last week.