[Config?] cuOpt docker container upload file size limit?

[algomia]

Hi all

I am experiencing a problem with the cuOpt server API when the inputs get large. Details on my setup below. I have created a zlib'ed file (containing an input for the LP solver) sized 2.9 GB. To reproduce the problem I run the command:

wget --header="Content-type: application/zlib" --header="Accept: application/json" --post-file <path-to-zip> https://<cuopt-server-address>/cuopt/request
If the size of the zip is small enough, e.g., 200 MB or so, it all works fine. But the large files don't make it to the cuOpt process: the logs remain empty in that case. However wget can connect, but it can't send anything. It reports:

--2026-06-03 15:54:40--  https://9doyh1d7ut5pix-5000.proxy.runpod.net/cuopt/request
Resolving 9doyh1d7ut5pix-5000.proxy.runpod.net (9doyh1d7ut5pix-5000.proxy.runpod.net)... 104.18.6.228, 104.18.7.228, 2606:4700::6812:6e4, ...
Connecting to 9doyh1d7ut5pix-5000.proxy.runpod.net (9doyh1d7ut5pix-5000.proxy.runpod.net)|104.18.6.228|:443... connected.
Retrying.

--2026-06-03 15:55:11--  (try: 2)  https://9doyh1d7ut5pix-5000.proxy.runpod.net/cuopt/request
Connecting to 9doyh1d7ut5pix-5000.proxy.runpod.net (9doyh1d7ut5pix-5000.proxy.runpod.net)|104.18.6.228|:443... connected.
Retrying.

--2026-06-03 15:55:43--  (try: 3)  https://9doyh1d7ut5pix-5000.proxy.runpod.net/cuopt/request
Connecting to 9doyh1d7ut5pix-5000.proxy.runpod.net (9doyh1d7ut5pix-5000.proxy.runpod.net)|104.18.6.228|:443... connected.
Retrying.

--2026-06-03 15:56:16--  (try: 4)  https://9doyh1d7ut5pix-5000.proxy.runpod.net/cuopt/request
Connecting to 9doyh1d7ut5pix-5000.proxy.runpod.net (9doyh1d7ut5pix-5000.proxy.runpod.net)|104.18.6.228|:443... connected.
Retrying.

--2026-06-03 15:56:50--  (try: 5)  https://9doyh1d7ut5pix-5000.proxy.runpod.net/cuopt/request
Connecting to 9doyh1d7ut5pix-5000.proxy.runpod.net (9doyh1d7ut5pix-5000.proxy.runpod.net)|104.18.6.228|:443... connected.
Retrying.

--2026-06-03 15:57:25--  (try: 6)  https://9doyh1d7ut5pix-5000.proxy.runpod.net/cuopt/request
Connecting to 9doyh1d7ut5pix-5000.proxy.runpod.net (9doyh1d7ut5pix-5000.proxy.runpod.net)|104.18.6.228|:443... connected.
Retrying.

--2026-06-03 15:58:01--  (try: 7)  https://9doyh1d7ut5pix-5000.proxy.runpod.net/cuopt/request
Connecting to 9doyh1d7ut5pix-5000.proxy.runpod.net (9doyh1d7ut5pix-5000.proxy.runpod.net)|104.18.6.228|:443... connected.
Retrying.

--2026-06-03 15:58:38--  (try: 8)  https://9doyh1d7ut5pix-5000.proxy.runpod.net/cuopt/request
Connecting to 9doyh1d7ut5pix-5000.proxy.runpod.net (9doyh1d7ut5pix-5000.proxy.runpod.net)|104.18.6.228|:443... connected.
Retrying.

--2026-06-03 15:59:16--  (try: 9)  https://9doyh1d7ut5pix-5000.proxy.runpod.net/cuopt/request
Connecting to 9doyh1d7ut5pix-5000.proxy.runpod.net (9doyh1d7ut5pix-5000.proxy.runpod.net)|104.18.6.228|:443... connected.
Retrying.

Setup:

• Hosted at runpod
• GPU: A100 SXM x1
• Container image: nvidia/cuopt:latest-cuda12.9-py3.13
• Port: 5000
• Pod volume: 50 GB
• cuOpt server: cuopt server version 26.04.000

Does anybody have an idea here?

Cheers,
Alex

[rgsl888prabhu]

I just ran it through an agent, please let me know if this helps

  Diagnosis: it's the runpod proxy, not cuOpt

  Your endpoint is *.proxy.runpod.net, which resolves to 104.18.6.228 / 104.18.7.228 — Cloudflare IPs. runpod's HTTP proxy is Cloudflare‑fronted, and that path enforces a request‑body size limit (Cloudflare's default is ~100 MB; runpod's proxy is in that same ballpark).
  That's the whole signature of what you're seeing:

  - wget completes the TLS handshake (connected) but can never get the body accepted → endless Retrying.
  - ~200 MB works, 2.9 GB doesn't → a body‑size ceiling in the proxy.
  - cuOpt logs stay empty → the request never reaches the cuOpt process at all.

  So nothing is wrong with cuOpt here — the large POST body is being dropped upstream by the proxy. Two ways to fix it, in order of preference.

  Fix 1 (recommended): pass the data by file, not in the HTTP body

  cuOpt server has a feature built precisely for this — submit the problem via the filesystem so the HTTP body stays tiny and the proxy limit is irrelevant.

  1. Start the server with a mounted data directory (and a result dir for the large response too):
  docker run --gpus all -p 5000:5000 \
    -e CUOPT_DATA_DIR=/cuopt_data \
    -e CUOPT_RESULT_DIR=/cuopt_results \
    -v /host/path/data:/cuopt_data \
    -v /host/path/results:/cuopt_results \
    nvidia/cuopt:latest-cuda12.9-py3.13
  2. Put your problem.zlib in /host/path/data.
  3. Reference it by filename in a header — the body is empty:
  curl -X POST https://<addr>/cuopt/request \
    -H "CUOPT-DATA-FILE: problem.zlib" \
    -H "CUOPT-RESULT-FILE: result.zlib" \
    -H "Content-Type: application/zlib" \
    -H "Accept: application/json"
  3. The server reads CUOPT_DATA_DIR/problem.zlib directly (CUOPT-DATA-FILE header), so nothing large traverses the proxy.

  With the self‑hosted Python client this is just filepath=True:
  client = CuOptServiceSelfHostClient(ip=..., port=...)
  sol = client.get_LP_solve("problem.zlib", filepath=True, output="result.zlib")
  (internally it sets CUOPT-DATA-FILE and sends an empty body — see cuopt_self_host_client.py). Note results: when CUOPT_RESULT_DIR is set, results larger than CUOPT_MAX_RESULT KB (default 250) are written to that dir instead of returned over HTTP — important, since a 2.9 GB
  input likely produces a large response that would hit the same proxy limit on the way back.
  
  Fix 2: bypass the runpod HTTP proxy

  If you must send the body over the wire, don't use *.proxy.runpod.net. Use runpod's direct TCP port mapping (the pod's public IP + mapped port) — that avoids the Cloudflare body cap and lets a normal POST carry the payload. Fix 1 is still better for 2.9 GB.

  Two things worth checking regardless

  - RAM, not disk. A 2.9 GB zlib decompresses to a much larger JSON that the server parses in memory. Your "50 GB pod volume" is disk; make sure the instance has enough host RAM to hold the decompressed/parsed model, or you'll OOM after the upload succeeds.
  - For an LP, consider MPS instead of zlib'd JSON. MPS is typically far more compact than JSON for the same LP and is supported by the LP path — that alone may shrink your input dramatically and sidestep the size problem.

  Want me to package this up as a reply on the GitHub issue (and, if this matches a real gap, propose a docs note in the server skill/quick-start about CUOPT_DATA_DIR for large inputs behind size‑limited proxies)?

[algomia]

Hi @rgsl888prabhu

Thank you very much for the analysis and suggestions! I have first tried "Fix 2", as it was quicker to follow. Indeed, then the file gets transferred and cuOpt starts its work. However - as assumed in the analysis - I ran into OOM issues. Even the following setup could not finish:

GPU: B300 1x
vCPU: 32 (Intel(R) Xeon(R) 6767P)
Memory: 251 GB
Container disk: 50 GB

Here are also the logs:

• Container log:
  container-logs.txt
• System log:
  system-logs.txt

It's a bit strange that no explicit OOM message is logged. The server just says that the process is aborted:

Response status code 500 Internal Server Error
{"error":"73cdcecf-f774-4def-abf2-8b748802cfed was aborted","error_result":true}

I guess that there is not much we can do about it, except for somehow reducing/decomposing the LP ...

Best,
Alex

[rgsl888prabhu]

Hi @rgsl888prabhu

Thank you very much for the analysis and suggestions! I have first tried "Fix 2", as it was quicker to follow. Indeed, then the file gets transferred and cuOpt starts its work. However - as assumed in the analysis - I ran into OOM issues. Even the following setup could not finish:

GPU: B300 1x vCPU: 32 (Intel(R) Xeon(R) 6767P) Memory: 251 GB Container disk: 50 GB

Here are also the logs:

• Container log:
  container-logs.txt
• System log:
  system-logs.txt

It's a bit strange that no explicit OOM message is logged. The server just says that the process is aborted:

Response status code 500 Internal Server Error
{"error":"73cdcecf-f774-4def-abf2-8b748802cfed was aborted","error_result":true}

I guess that there is not much we can do about it, except for somehow reducing/decomposing the LP ...

Best, Akex

Thank you Alex, may be I will follow-up on OOM logging issue.

[algomia]

Btw, here is a description as to when and how to bypass the runpod proxy:

https://www.runpod.io/blog/runpod-proxy-guide