New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add proper timing for model inference. #13

Open

DennisSmolek opened this issue Jul 17, 2024 · 0 comments

Owner

DennisSmolek commented Jul 17, 2024

Right now timing execution actually just times submission to the GPU. Not the actual work.

There are various strategies for this we can employ.

We should also time input and output handling better so we can see where in the flow speed improvements would be possible

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment