Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add default encoding for CSV #1025

Merged
merged 7 commits into from
Jun 12, 2024
Merged

Add default encoding for CSV #1025

merged 7 commits into from
Jun 12, 2024

Conversation

D-system
Copy link
Contributor

ActiveStorage fetch files as ASCII-8BIT. The CSV fails to open any files with non ASCII characters. With this change the CSV is open with Encoding.default_external which use the ENV["LANG"].

test/models/maintenance_tasks/csv_task_test.rb Outdated Show resolved Hide resolved
test/models/maintenance_tasks/csv_task_test.rb Outdated Show resolved Hide resolved
app/models/maintenance_tasks/task.rb Outdated Show resolved Hide resolved
test/models/maintenance_tasks/csv_task_test.rb Outdated Show resolved Hide resolved
test/models/maintenance_tasks/csv_task_test.rb Outdated Show resolved Hide resolved
test/models/maintenance_tasks/csv_task_test.rb Outdated Show resolved Hide resolved
@D-system
Copy link
Contributor Author

@etiennebarrie I updated from your review.
TIL Rails forced the encoding to UTF-8

@D-system
Copy link
Contributor Author

@etiennebarrie any other issue I can address for the UTF-8 in the CSV?

@etiennebarrie
Copy link
Member

Rebase to fix CI errors

D-system and others added 7 commits June 12, 2024 18:15
ActiveStorage fetch files as ASCII-8BIT. The CSV fails to open any files
with non ASCII characters. With this change the CSV is open with
`Encoding.default_external` which use the `ENV["LANG"]`.
Update the tests in accordance.
Remove the encoding override.
Co-authored-by: Étienne Barrié <etienne.barrie@gmail.com>
@D-system
Copy link
Contributor Author

@etiennebarrie thank you, I rebased and tested rubocop locally, all greens (in ruby 3.3.0)

Copy link
Member

@etiennebarrie etiennebarrie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@etiennebarrie etiennebarrie merged commit f455890 into Shopify:main Jun 12, 2024
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants