PackedFunc usage in generated code should be thread-safe #4387
Description
See the discussion at https://discuss.tvm.ai/t/tvm-deadlocking-when-calling-python-packedfunc-from-generated-code/4657.
In the custom datatypes framework I'm building, one feature I'd love to incorporate is the ability to implement datatype operations purely in Python. The way I've currently implemented this takes in a Python function implementing a datatype operator (e.g. my_custom_dtype_add), wraps it in a PackedFunc, and then, at codegen, when TVM sees an add with the custom type, it inserts a tvm_call_packed_lowered intrinsic calling the PackedFunc.
The above discuss thread describes my experimentation with this -- it works, if I run TVM single-threaded. Tianqi suspects that the problem is that calls to PackedFuncs in generated code are not thread-safe, due to how they handle argument and return allocation.
This issue is meant to open the discussion on making generated PackedFunc calls thread-safe. I'm sure this has been discussed before -- please link previous discussions and conclusions!