Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Does enqueueV3 require the stream synchronization before the call? #3925

Closed
fortminors opened this issue Jun 5, 2024 · 1 comment
Closed

Comments

@fortminors
Copy link

I am reading the description of the enqueueV3 function, it states

Modifying or releasing memory that has been registered for the tensors before stream
synchronization or the event passed to setInputConsumedEvent has been being triggered results in undefined
behavior

Does that mean that after I preprocess my data, making it ready for the inference, I have to invoke cudaStreamSynchronize before I call enqueueV3?
I expect that I need no synchronization because all the preprocessing followed by inference is done asynchronously on the same cudaStream_t, however I just wanted to make sure that my understanding is correct and there will be no undefined behavior if I have no synchronization in this case

P.S. I am aware that I would need to cudaStreamSynchronize after inference before I move data to host and get results

@lix19937
Copy link

lix19937 commented Jun 7, 2024

Does enqueueV3 require the stream synchronization before the call? #3925

No, if use the same stream from preprocess to infer.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants