fix crash during torch model reload with inter op thread parameter set #158
Conversation
snadampal
changed the title
There was a problem hiding this comment.
During model reload the call to set inter_op value will be skipped altogether, however, nothing will be logged.
From the user perspective, it might create a confusion as to what inter_op value is used internally by Pytorch backend.
I would suggest taking LOG_MESSAGE out of the conditon, so inter_op_thread_count value is always logged whether setting it skipped or not.
torch allows setting the interop thread count only once and before any inter-op parallel work is started because it creates threadpool statically. To follow this constraint, check the interop thread count and set only if it's not yet set.
snadampal
force-pushed
the
fix_crash_during_model_reload
branch
from
46e7f68 to
b5a7410
Compare
|
@gerilya thanks for the feedback. I have made the LOG_MESSAGE unconditional so the |
| } else { | ||
| if (inter_op_thread_count > 0) { | ||
| if ((inter_op_thread_count > 0) && | ||
| (inter_op_thread_count != at::get_num_interop_threads()) && |
There was a problem hiding this comment.
This can still allow a repeated call if two model config files set the number of interop threads to different values. (See #159 for a safer solution.)
There was a problem hiding this comment.
Good. thanks for fixing the multiple config file scenario as well!
|
closing this in favor of #159 |
torch allows setting the interop thread count only once and before any inter-op parallel work is started because it creates threadpool statically.
To follow this constraint, check the interop thread count and set only if it's not yet set.