Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Keep track of stream usage for arrays #633

Merged
merged 1 commit into from
May 17, 2024
Merged

Keep track of stream usage for arrays #633

merged 1 commit into from
May 17, 2024

Conversation

pxl-th
Copy link
Collaborator

@pxl-th pxl-th commented May 17, 2024

  • Add Managed struct that keeps track of TLS streams and correctly switches between them. This in turn allows us to not use global stream in finalizers.
    Ref: Track CuArray stream usage CUDA.jl#2236
    Allows us to use tls streams in finalizers instead of global one.
  • Remove REPL hook.

Performance on Nerf.jl benchmark (1k training steps & 10 rendered 1080p frames):

  • 0.8.11:
Training
 38.445143 seconds (3.35 M allocations: 145.687 MiB, 0.15% gc time)
Rendering
141.926297 seconds (26.43 M allocations: 930.844 MiB, 0.17% gc time)
  • This PR:
Training
 34.165407 seconds (3.24 M allocations: 153.949 MiB, 0.15% gc time)
Rendering
116.225133 seconds (27.26 M allocations: 1.039 GiB, 0.22% gc time, 0.04% compilation time)

#628 also had an impact on performance.

- Add Managed struct that keeps track of TLS streams and correctly
    switches between them. This in turn allows us to **not** use global
    stream in finalizers.
- Remove REPL hook.
@pxl-th pxl-th merged commit 24ad256 into master May 17, 2024
3 checks passed
@pxl-th pxl-th deleted the pxl-th/managed branch May 17, 2024 09:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant