[FIX,RPC] Skip RPC tests when using multiprocessing's spawn method

[tkonolige]

The rpc tests are broken when running under pytest with multiprocessing using spawn. I suspect this is because pytest tests each function in a separate process and does not import the full module.

@tqchen

[tqchen]

I think it is fine as it is. A better way is to also fix the testcases so that we don't have to rely on the forking behavior in many rpc tests

[junrushao]

Thanks for the contribution! I didn't actually understand where the issue comes from? Is it because some of the register_funcs are not correctly executed?

[tqchen]

This is because some of the register_func are in local function scope instead of global, so they won't be executed right after spawn. Globally registered packedfunc will be preserved.

If we move most of the rpc testing function registration to tvm.testing. Or enable a env setup function that passes to the spawn method(via cloud pickle) which registers the function during rpc startup, it should work.

[tkonolige]

This is because some of the register_func are in local function scope instead of global, so they won't be executed right after spawn. Globally registered packedfunc will be preserved

This is what I thought initially, but something weirder is going on. The globally registered packedfunc works when you run the test script on its own (without pytest). But when running under pytest, the registered functions cannot be found. I commented about it in the source code:

# tkonolige: The issue as I understand it is this: multiprocessing's spawn
 # method launches a new process and then imports the relevant modules. This
 # means that all registered functions must exist at the top level scope. In
 # this file they are, so all is well when we run this file directly.
 # However, when run under pytest, the functions aren't registered on the
 # server. I believe this is because pytest is also using multiprocessing to
 # run individual functions. Somewhere along the way, the imports are being
 # lost, so the server ends up not registering the functions.

[tqchen]

I think this is because of the location of the "main" file. The python spawn might automatically run the global reg in the main file. but not other places. Pytest itself have a different main file.

Putting the registeration under tvm.testing means they are imported regardless. Alternatively, passing an startup env callback to RPCServer would mean the startup env function get called explicitly(and registers the function). My guess is that the startup env might be an attractive approach we can try.

[tqchen]

merge this for now, we should re-enable after we rewrite test to use startup env callback