Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change another solution to handle utf-8 string #59

Merged
merged 3 commits into from
May 31, 2015

Conversation

ilake
Copy link
Contributor

@ilake ilake commented May 28, 2015

The original solution will cause character lost like

# Chinese character will be replaced
"中文".encode('UTF-8', 'binary', invalid: :replace, undef: :replace, replace: '').strip.gsub(/[\r\n\f]+/, "\n")
=>  ""

# New solution
str = "中文"
str.encode!("UTF-16", "UTF-8", invalid: :replace, undef: :replace, replace: "")
=> "\uFEFF\u4E2D\u6587"
str.encode!("UTF-8", "UTF-16").strip.gsub(/[\r\n\f]+/, "\n")
=> "中文"

@gottfrois
Copy link
Owner

nice thx, this is really obscure to me

@ilake
Copy link
Contributor Author

ilake commented May 28, 2015

totally understand :)

@coveralls
Copy link

Coverage Status

Coverage increased (+0.0%) to 98.92% when pulling ad8c616 on ilake:master into 4a60511 on gottfrois:master.

@gottfrois
Copy link
Owner

can you please add a spec for this?

it { expect(action).to eq(result) }

@ilake
Copy link
Contributor Author

ilake commented May 28, 2015

Sure

@coveralls
Copy link

Coverage Status

Coverage decreased (-0.06%) to 98.85% when pulling 3a4fdef on ilake:master into 4a60511 on gottfrois:master.

@gottfrois
Copy link
Owner

failing, you probably want to change the file encoding ;)

@coveralls
Copy link

Coverage Status

Coverage increased (+0.0%) to 98.92% when pulling adcef38 on ilake:master into 4a60511 on gottfrois:master.

@ilake
Copy link
Contributor Author

ilake commented May 30, 2015

passed :)

gottfrois added a commit that referenced this pull request May 31, 2015
Change another solution to handle utf-8 string
@gottfrois gottfrois merged commit ae27324 into gottfrois:master May 31, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants