-
Notifications
You must be signed in to change notification settings - Fork 2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Decode to utf8 #20
Decode to utf8 #20
Conversation
fec549d
to
1e8cfd3
Compare
stringutil/decoder_test.go
Outdated
|
||
func TestDecodeToUTF8(t *testing.T) { | ||
for _, testcase := range []struct { | ||
Value string |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
does it need to be a public variable?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
you squash those two lines, in, out string
stringutil/decoder.go
Outdated
return s | ||
} | ||
|
||
s, err := charmap.ISO8859_1.NewDecoder().String(s) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
should we store the decoder in a variable instead of instanciate it every time
stringutil/decoder.go
Outdated
s, err := charmap.ISO8859_1.NewDecoder().String(s) | ||
|
||
if err != nil { | ||
return "" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
should we log the error ?
stringutil/decoder.go
Outdated
"github.com/upfluence/pkg/log" | ||
) | ||
|
||
var default_decoder = charmap.ISO8859_1.NewDecoder() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
defaultDecoder
4e6f92b
to
8112541
Compare
stringutil/decoder.go
Outdated
var defaultDecoder = charmap.ISO8859_1.NewDecoder() | ||
|
||
func DecodeToUTF8(s string) string { | ||
if s == "" { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
is it really useful?
stringutil/decoder_test.go
Outdated
{"test", "test"}, | ||
{"testé", "testé"}, | ||
{"test\xc3", "testÃ"}, | ||
{"\xc3", "Ã"}, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
you need to also test the case where it's a non standard set of bytes
No description provided.