Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OOM error on large JSON files #14337

Closed
aduh95 opened this issue Aug 1, 2024 · 2 comments
Closed

OOM error on large JSON files #14337

aduh95 opened this issue Aug 1, 2024 · 2 comments
Assignees

Comments

@aduh95
Copy link

aduh95 commented Aug 1, 2024

I did this

curl --config - <<'EOF'
request = "POST"
fail-with-body
verbose
show-error
url = "http://172.16.22.1:12507/"
header = "Expect:"
insecure
max-time = "300"
data-urlencode = "transloadit@/path/to/data.json"
globoff
create-dirs
max-redirs = "5"
max-filesize = "128849018880"
EOF

I'd rather not share data.json, but I can disclose it's a 16M JSON file.

I expected the following

Note: Unnecessary use of -X or --request, POST is already inferred.
*   Trying 172.16.22.1:12507...

Instead got curl: <stdin>:9: 'data-urlencode' out of memory

curl/libcurl version

git bisect pointed me to f37840a. I can confirm that reverting this commit does solve the issue for me.

operating system

Reproduced with macOS 14.5 and Linux 6.8.0

@bagder
Copy link
Member

bagder commented Aug 1, 2024

Hm, right. It stops at 8MB now...

@bagder bagder self-assigned this Aug 1, 2024
@bagder
Copy link
Member

bagder commented Aug 1, 2024

I'll fix

bagder added a commit that referenced this issue Aug 1, 2024
Previously it would bail out of the generated data reached 8MB in
memory.

Reported-by: Antoine du Hamel
Fixes #14337
bagder added a commit that referenced this issue Aug 1, 2024
Previously it would bail out of the generated data reached 8MB in
memory.

Reported-by: Antoine du Hamel
Fixes #14337
Closes #14340
@bagder bagder closed this as completed in 1f61db5 Aug 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging a pull request may close this issue.

2 participants