Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Valid cURL command fails to be imported into HTTP node with "Couldn’t import cURL command" error #9768

Closed
restyler opened this issue Jun 15, 2024 · 6 comments

Comments

@restyler
Copy link

restyler commented Jun 15, 2024

Bug Description

I have a rather complex JSON payload in my cURL which fails to be imported into n8n.

curl -X POST -H 'X-RapidAPI-Key: YOUR-KEY' \
-H 'content-type: application/json' \
-H 'X-RapidAPI-Host: scrapeninja.p.rapidapi.com' \
-d '{"url":"https://news.ycombinator.com","method":"GET","retryNum":1,"geo":"us","extractor":"// define function which accepts body and cheerio as args\nfunction extract(input, cheerio) {\n  let top = \"gg\";\n  return { items };\n}"}' \
"https://scrapeninja.p.rapidapi.com/scrape"

If I remove = equals sign ( let top = \\"gg\\";) from the JSON, or whole extractor property, it works fine.

n8n throws {"code":400,"message":"Invalid cURL command"} for rest curl import.

2024-06-15 at 09 13
Here is what is sent to n8n curl-to-json endpoint

What is interesting is that curlconverter@3.21.0 (which is used in n8n) does not have any issues parsing this payload when I call it directly: (UPD: I was wrong! Initially I thought the problem occurs due to incorrect quotes escaping, but it was due to = sign)

2024-06-15 at 09 22

So the problem is located somewhere in n8n /rest/curl-to-json endpoint (I confirmed this by digging through n8n code).

To Reproduce

  1. create new HTTP node
  2. try to Import cURL
  3. get my cURL command from bug description
  4. receive the error

Expected behavior

the HTTP node should be created successfully

Operating System

Ubuntu Linux 22

n8n Version

1.45.1

Node.js Version

v20.13.1

Database

SQLite (default)

Execution mode

main (default)

@restyler
Copy link
Author

restyler commented Jun 15, 2024

I hacked curl.controller.js to return real nodejs exception message, and now I see that the exception error is {"code":400,"message":"Unterminated string in JSON at position 234"}

@restyler
Copy link
Author

restyler commented Jun 15, 2024

@netroy I noticed you’ve been working with n8n cURL service recently and I’d really value your opinion on this. Could you please take a look when you have a moment? Thanks!

Some background: I run a web scraper builder which generates JS / cURL commands to run this web scraper - and cURL is a perfect method to embed complex web scrapers into n8n.

2024-06-15 at 10 36

@restyler
Copy link
Author

restyler commented Jun 15, 2024

ok after some digging I realised it's not nested quotes, it's = sign. I have updated the issue description accordingly.

2024-06-15 at 11 05

so if I remove equals sign - it is imported correctly.

@restyler
Copy link
Author

restyler commented Jun 15, 2024

Ok I have isolated the problem - it is indeed a curlconverter@3 issue.
curlconverter@4 (4.10.1) encodes the equal sign properly while v3 trims it poorly and adds single quotes which breaks n8n.

restyler added a commit to restyler/n8n that referenced this issue Jun 15, 2024
Fix equal sign encoding in "Import from cURL" feature.

n8n-io#9768
@Joffcom
Copy link
Member

Joffcom commented Jun 17, 2024

Hey @restyler,

Thanks for the report, As there is a PR for this as well I am going to close this issue and we can continue the conversation on the PR.

@Joffcom Joffcom closed this as not planned Won't fix, can't repro, duplicate, stale Jun 17, 2024
@restyler
Copy link
Author

restyler commented Jul 5, 2024

It turned out that the problem is in query-string package which is used in generators/json.js dependency of the curlconverter.
https://github.com/curlconverter/curlconverter/blob/v3.21.0/generators/json.js#L48

this is where the string gets corrupted.
The funny thing is that latest query-string@7 contains the error, and query-string@8 is ESM-only, pretty much like latest curlconverter.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants