Issue
I came across this issue when trying to run code that uses the sympy library invoked via a webserver in Snekbox. It seems that if there is a memory-intensive request, Snekbox will begin returning "137" out-of-memory errors if it receives multiple of them at the same time.
For an import to a large library like networkx, pandas, or sympy, merely two requests that import these at the same time will cause either one (typical) or both (rare) requests to fail.
Environment
Operating System: Microsoft Windows 10 Education Version 10.0.18363 Build 18363
Total Physical Memory: 31.9 GB
Docker version: Docker version 19.03.13, build 4484c46d9d
Steps to reproduce
- Run snekbox
docker run --ipc=none --privileged -p 8060:8060 ghcr.io/python-discord/snekbox
- Run the following code with one of the "CODE" lines uncommented.
import json
import urllib.request
from multiprocessing.dummy import Pool
PROCESSES = 3
# CODE = "import networkx"
# CODE = "import pandas"
# CODE = "import sympy"
CODE = "import time; ' ' * 33000000; time.sleep(0.1)"
def run_code_in_snekbox(code):
body = {'input': code}
req = urllib.request.Request("http://localhost:8060/eval")
req.add_header('Content-Type', 'application/json; charset=utf-8')
json_data = json.dumps(body).encode('utf-8')
req.add_header('Content-Length', str(len(json_data)))
with urllib.request.urlopen(req, json_data) as f:
response = f.read()
return json.loads(response.decode('utf-8'))
if __name__ == '__main__':
arguments = [CODE] * PROCESSES
with Pool(PROCESSES) as p:
print(p.map(run_code_in_snekbox, arguments))
- Observe that one request will succeed while the other two will fail:
[{'stdout': '', 'returncode': 0}, {'stdout': '', 'returncode': 137}, {'stdout': '', 'returncode': 137}]Container logs:
2020-12-07 00:57:27,616 | 7 | snekbox.nsjail | INFO | Executing code...
2020-12-07 00:57:27,622 | 8 | snekbox.nsjail | INFO | Executing code...
2020-12-07 00:57:27,641 | 7 | snekbox.nsjail | INFO | pid=2447 ([STANDALONE MODE]) terminated with signal: SIGKILL (9), (PIDs left: 0)
2020-12-07 00:57:27,642 | 7 | snekbox.nsjail | INFO | nsjail return code: 137
2020-12-07 00:57:27,644 | 7 | snekbox.nsjail | INFO | Executing code...
2020-12-07 00:57:27,668 | 7 | snekbox.nsjail | INFO | pid=2451 ([STANDALONE MODE]) terminated with signal: SIGKILL (9), (PIDs left: 0)
2020-12-07 00:57:27,668 | 7 | snekbox.nsjail | INFO | nsjail return code: 137
2020-12-07 00:57:27,750 | 8 | snekbox.nsjail | INFO | pid=2449 ([STANDALONE MODE]) exited with status: 0, (PIDs left: 0)
2020-12-07 00:57:27,751 | 8 | snekbox.nsjail | INFO | nsjail return code: 0
Desired behavior
If snekbox can service one of these requests, I'm hoping that there can be a way to make it scale and service all of them even if that means wait times.
Issue
I came across this issue when trying to run code that uses the sympy library invoked via a webserver in Snekbox. It seems that if there is a memory-intensive request, Snekbox will begin returning "137" out-of-memory errors if it receives multiple of them at the same time.
For an import to a large library like networkx, pandas, or sympy, merely two requests that import these at the same time will cause either one (typical) or both (rare) requests to fail.
Environment
Operating System: Microsoft Windows 10 Education Version 10.0.18363 Build 18363
Total Physical Memory: 31.9 GB
Docker version: Docker version 19.03.13, build 4484c46d9d
Steps to reproduce
[{'stdout': '', 'returncode': 0}, {'stdout': '', 'returncode': 137}, {'stdout': '', 'returncode': 137}]Container logs:
Desired behavior
If snekbox can service one of these requests, I'm hoping that there can be a way to make it scale and service all of them even if that means wait times.