Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[backend] Fixed issue with skip char line parameter in csv mapper (#5764) #5962

Merged
merged 2 commits into from
Feb 15, 2024

Conversation

JeremyCloarec
Copy link
Contributor

Proposed changes

The issue related to this PR mentioned a parsing error when parsing a csv file with the skip char line character at the start of a csv field line.
The error came from the fact that we were trying to skip the lines after the parse was done, but the csv couldn't be parsed because it wasn't recognized as a proper csv file in the raw parser.
I moved the handling of the skip line char to the native parser, using the comment and comment_no_infix options.

Related issues

Checklist

  • I consider the submitted work as finished
  • I tested the code for its functionality
  • I wrote test cases for the relevant uses case
  • I added/update the relevant documentation (either on github or on notion)
  • Where necessary I refactored code to improve the overall quality

Further comments

If this is a relatively large or complex change, kick off the discussion by explaining why you chose the solution you did and what alternatives you considered, etc...

@JeremyCloarec JeremyCloarec added the filigran team use to identify PR from the Filigran team label Feb 14, 2024
@labo-flg labo-flg self-requested a review February 15, 2024 08:11
Copy link

codecov bot commented Feb 15, 2024

Codecov Report

Attention: 2 lines in your changes are missing coverage. Please review.

Comparison is base (a42bf77) 65.44% compared to head (d3544f4) 65.82%.
Report is 3 commits behind head on master.

Files Patch % Lines
...-platform/opencti-graphql/src/parser/csv-parser.ts 80.00% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #5962      +/-   ##
==========================================
+ Coverage   65.44%   65.82%   +0.38%     
==========================================
  Files         539      539              
  Lines       63055    63060       +5     
  Branches     5052     5509     +457     
==========================================
+ Hits        41267    41512     +245     
+ Misses      21788    21548     -240     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@JeremyCloarec JeremyCloarec merged commit 2e7eadf into master Feb 15, 2024
8 checks passed
@JeremyCloarec JeremyCloarec deleted the issues/5764 branch February 15, 2024 09:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
filigran team use to identify PR from the Filigran team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

CSV Mapper skip char, not working when csv fields are surrounded by double quote
3 participants