AsyncIO: Different Server/Client Reader Limits

[drewrisinger]

Bug Report

One-Line Summary

Async co-routines have different reader limits. Related: #285

Issue Details

Steps to Reproduce

AsyncioClient:

artiq/artiq/protocols/pc_rpc.py

Lines 203 to 204 in 8f9858b

	self.__reader, self.__writer = \
	await asyncio.open_connection(host, port, limit=100*1024*1024)

AsyncioServer:

artiq/artiq/protocols/asyncio_server.py

Lines 27 to 29 in 8f9858b

	self.server = await asyncio.start_server(self._handle_connection,
	host, port,
	limit=4*1024*1024)

Expected Behavior

I would expect these limits to be the same. Is there any reason why they aren't? If I'm transferring this much data, is PYON not the right tool and I should use something else (like a client as mentioned in developing an ndsp.rst)

Actual (undesired) Behavior

This caused me some errors when sending large numpy arrays (~2 MB of raw data) via PYON RPC. Actual code is too complex, but changing the AsyncioServer limit to ~16 MB fixed it for me.

[sbourdeauducq]

No good reason. If you have better numbers, test them, and send a PR, I will merge it.
Note that the two code snippets you mention are not necessarily always talking to each other; the first one is for pc_rpc only whereas the second one factors out some generic asyncio server boilerplate. Look at the file names and not just the class names.

[sbourdeauducq]

@drewrisinger Did you figure out better numbers and can you send a PR?

[drewrisinger]

I used 16 MB for asyncio_server and that worked for my data. Think this is highly data-dependent, and I don't have a good handle on when it's worth increasing the limit vs moving to a different non-PYON protocol. The raw data that the numpy array was constructed from was ~2-4 MB of text, so it makes sense that 4 MB was too tight.