Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: broken CSV (#2426) #7318

Merged
merged 1 commit into from
Sep 7, 2022
Merged

fix: broken CSV (#2426) #7318

merged 1 commit into from
Sep 7, 2022

Conversation

CharlesNepote
Copy link
Member

@CharlesNepote CharlesNepote commented Sep 7, 2022

Should fix #2426.

Script not tested, in itself, but I have done some successful tests with:
perl -ne 'print if /(\xE2\x80\xA8|\xE2\x80\xA9|[\000-\007\013-\037])/' en.openfoodfacts.org.products.csv

I suggest to control in production.

Should solve #2426. Script not tested, in itself, but I have done some successful tests with:
`perl -ne 'print if /(\xE2\x80\xA8|\xE2\x80\xA9|[\000-\007\013-\037])/' en.openfoodfacts.org.products.csv`

I suggest to control in production.
@sonarcloud
Copy link

sonarcloud bot commented Sep 7, 2022

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
No Duplication information No Duplication information

@CharlesNepote CharlesNepote changed the title Fix broken CSV (#2426) fix: broken CSV (#2426) Sep 7, 2022
@stephanegigandet
Copy link
Contributor

Looks good to me

@stephanegigandet stephanegigandet merged commit 4c62929 into main Sep 7, 2022
@stephanegigandet stephanegigandet deleted the CharlesNepote-patch-2 branch September 7, 2022 16:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: No status
Development

Successfully merging this pull request may close these issues.

CSV dataset broken due to control chars
2 participants