Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support torch.compile #47

Open
oxysoft opened this issue Jun 13, 2023 · 0 comments
Open

Support torch.compile #47

oxysoft opened this issue Jun 13, 2023 · 0 comments

Comments

@oxysoft
Copy link

oxysoft commented Jun 13, 2023

In #40 there is some discussion about supporting torch.compile and I'd like to create an issue for it in case anyone comes here looking for the same. I am working towards real-time applications, so any speed I can scrape is a massive gain for me. Must break the 1 FPS barrier!

I've made this benchmark below and would love to extend it. In particular I wonder if the speedup would be additive or multiplicative.

GPU Model Optimizations Speed (it/s)
RTX 3090 ControlNet(HED+TemporalNet+Depth) Raw 5.76
RTX 3090 ControlNet(HED+TemporalNet+Depth) TomeSD 37.5% 6.13
RTX 3090 ControlNet(HED+TemporalNet+Depth) Compile reduce-overhead 6.30
RTX 3090 ControlNet(HED+TemporalNet+Depth) Compile max-autotune 6.50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant