Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Spreadsheet File exports/imports CSV with BOM #7443

Closed
ed-parsadanyan opened this issue Oct 14, 2023 · 3 comments
Closed

Spreadsheet File exports/imports CSV with BOM #7443

ed-parsadanyan opened this issue Oct 14, 2023 · 3 comments
Labels
in linear Issue or PR has been created in Linear for internal review

Comments

@ed-parsadanyan
Copy link

Describe the bug
When converting a JSON into CSV via Spreadsheet File, an output binary contains a UTF-8 BOM in the beginning. The same goes for importing CSV binary into JSON, as the BOM is preserved in the output.

This makes subsequent work with such files tedious and unreliable, as BOM bytes are completely hidden from the view.

After digging into the code, I noticed that CSV is converted via a 'csv-parse' library:
https://github.com/n8n-io/n8n/blob/57c609384af7c583c213c639add7170e0bbb24fd/packages/nodes-base/nodes/SpreadsheetFile/v2/SpreadsheetFileV2.node.ts

It seems that this library has an additional bom option that can handle these cases:
https://csv.js.org/parse/options/bom/

I would be great to provide more clarity regarding BOM or at least automatically get rid of these bytes.

To Reproduce
Steps to reproduce the behavior:

  1. Create a JSON
  2. Convert to CSV via Spreadsheet File node
  3. Convert back to JSON via another Spreadsheet node.
  4. See error:

image

Expected behavior
A consistent back and forth conversion of CSV files

Environment (please complete the following information):

  • OS: Ubuntu Linux 22.04
  • n8n Version 1.8.2
  • Node.js Version NA, docker version
  • Database system Postgres
  • Operation mode main
@ed-parsadanyan
Copy link
Author

So, as of now I was able to beat this via 2 Move Binary nodes. Just in case someone will stumble upon the same issue:
image

@Joffcom
Copy link
Member

Joffcom commented Oct 16, 2023

Hey @ed-parsadanyan,

Thanks for the report, It took a bit of time to create a spreadsheet file that reproduced the issue but I have confirmed the issue and have created an internal ticket which we will be tracking as NODE-861 to get this resolved.

@Joffcom Joffcom added the in linear Issue or PR has been created in Linear for internal review label Oct 16, 2023
netroy added a commit that referenced this issue Oct 18, 2023
This adds support for
1. custom delimiters
2. reading offsets to avoid having to read a large CSV all at once
3. excluding byte-order-mark

NODE-861
#7443
@netroy
Copy link
Member

netroy commented Oct 18, 2023

@ed-parsadanyan This has been fixed in n8n@1.12.0.

@netroy netroy closed this as completed Oct 18, 2023
elsmr pushed a commit that referenced this issue Oct 19, 2023
This adds support for
1. custom delimiters
2. reading offsets to avoid having to read a large CSV all at once
3. excluding byte-order-mark

NODE-861
#7443
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
in linear Issue or PR has been created in Linear for internal review
Projects
None yet
Development

No branches or pull requests

3 participants