Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle console output that is improperly encoded #121

Merged
merged 2 commits into from
Jun 14, 2019

Conversation

mattbrictson
Copy link
Owner

Not sure why, but in some cases the data received from SSHKit claims to be encoded as UTF-8 but is actually invalid. This causes gsub to blow up with ArgumentError: invalid byte sequence in UTF-8 when airbrussh calls strip_ascii_color.

Work around this by detecting invalid encoding and stripping out the offending code points from string before calling gsub.

Fixes #120

@dbackeus can you test using this branch to see if this resolves your issue? 🙏

@mattbrictson mattbrictson force-pushed the handle-invalid-utf8 branch 3 times, most recently from 8fdaa7e to a49a34c Compare June 13, 2019 20:05
@dbackeus
Copy link

dbackeus commented Jun 13, 2019

I can confirm that I no longer get the exception 👍

Also now that I don't get the exception any more I noticed that the output coming out of rsync is actually Ho?\#210vding-65x42.png rather than Hövding-65x42.png. Maybe that's a clue as to why we were seeing the exception in the first place?

Seems like rsync might be the main culprit.

Not sure why, but in some cases the data received from SSHKit claims to
be encoded as UTF-8 but is actually invalid. This causes `gsub` to blow
up with `ArgumentError: invalid byte sequence in UTF-8` when airbrussh
calls `strip_ascii_color`.

Work around this by detected invalid encoding and stripping out the
offending code points from string before calling `gsub`.
@mattbrictson
Copy link
Owner Author

I don't know where the badly encoded string is coming from but at least airbrussh is now able to deal with it 😄

@mattbrictson mattbrictson merged commit 7b38769 into master Jun 14, 2019
@mattbrictson mattbrictson deleted the handle-invalid-utf8 branch June 14, 2019 00:00
@mattbrictson mattbrictson restored the handle-invalid-utf8 branch June 14, 2019 00:00
@dbackeus
Copy link

Nice. Hope to see a new rubygem release soon. Will point my Gemfile to the master branch in the meantime.

@mattbrictson mattbrictson deleted the handle-invalid-utf8 branch August 9, 2020 01:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

"invalid byte sequence in UTF-8" when rsyncing files with international character
2 participants