UTF-8 error in the compression model

[berkcoker]

I was having a utf-8 error until I included 'rb' in line 62 of file compression/image_encoder/encoder.py. It is quick fix:)

[ali01]

Thanks for the heads up, @berkcoker!
@nickj-google, are you able to make the fix and test?

Thanks,
Ali

[nmjohn]

Yeah, I can fix this. I want to spend a little time digging into what broke between then and now as well.

[nmjohn]

@berkcoker, What version of TensorFlow are you running?

I'm not seeing the issue on 1.0.1.

[d-hoke]

@nmjohn - Are you using a file that contains invalid utf-8 sequences? If your image files don't contain invalid utf-8 sequences, then you won't encounter the outright error. (I have wondered if the possibility exists of data being silently corrupted from conversions from valid utf-8 that gets transformed into something else internally.)

I have encountered the same issue as @berkcoker, and addressed it similarly (with versions of models cloned in week prior to this post, and tensorflow-gpu installed via pip on windows python 3.5.2 - unfortunately not immediately at hand to see commit or version [edit] tensorflow-gpu==1.3.0, model last commit 09a32f3)

I think there may also be an issue in decoder.py writing out the files as text, rather than with the additional 'b', with the output file possibly having conversions performed on it, but not sure.

I'm guessing this issue may have resulted from differences between python2 and python3 regarding (unicode changes for) text vs bytes and file io...

[rzumer]

Hello,

Using the latest version of Tensorflow with Python 3.6, the example image provided in the repository cannot be read without the modification indicated above by @berkcoker. A similar modification also needed to be applied to the generated code file in order to support decoding.
While this may be presumptuous, I have submitted a PR, #3712, to resolve this issue, as it has been sitting for nearly a year with no resolution despite a simple fix being available. Images are not UTF-8 strings and should not be read as such. I have tested the encoding-decoding loop, and the output does seem correct. Here it is in case you would like to validate.

@nmjohn, @ali01, I would appreciate any comments regarding this issue and PR and assistance to get it pulled if you agree with the fix.

Thanks.

[rzumer]

This is fixed in commit dcfe009.

[karmel]

Sounds like this issue is resolved; closing. Please open a new issue if new problems arise.