UnicodeDecodeError: 'utf-8' codec can't decode byte 0xfc in position 187: invalid start byte

[philipp-classen]

I'm using version 1.5.7 of system-config-printer on Arch Linux (foomatic-db is installed):

• Select "Add -> LPD/LPR Host or Printer"
• Enter some IP as "Host"
• Forward

It hangs forever and prints the following errors to the console:

Traceback (most recent call last):
  File "/usr/share/system-config-printer/newprinter.py", line 899, in on_btnNPForward_clicked
    self.nextNPTab()
  File "/usr/share/system-config-printer/newprinter.py", line 1095, in nextNPTab
    self.getNetworkPrinterMakeModel ()
  File "/usr/share/system-config-printer/newprinter.py", line 1960, in getNetworkPrinterMakeModel
    line = stdout.decode ().strip ()
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xfc in position 187: invalid start byte

/var/log/cups/error_log contains no errors (or other messages).

---

Side-note: When I try the IP of another printer, I see a "Searching for driver" popup. It fails differently, though. The Traceback from above is not printed. Instead the dialog fails with this ambiguous message:

"CUPS server error: There was an error during the CUPS operation: 'Success'."

There is an error in /var/log/cups/error_log:
[cups-driverd] Bad driver information file "/usr/share/cups/model/foomatic-db-ppds/Kyocera/ReadMe.htm"!

However, I think it is unrelated to the problem. I only posted it in case it helps. Otherwise, you can safely ignore it.

[jpopelka]

Not able to reproduce here.

What do you see when you run
/usr/lib/cups/backend/snmp IP_address_of_printer

[philipp-classen]

Next time I have access to the printer, I'll try again. Presumably, in about two weeks.

[philipp-classen]

$ /usr/lib/cups/backend/snmp xxx.xxx.xxx.xxx
INFO: Using default SNMP Community public
network socket://xxx.xxx.xxx.xxx "RICOH Aficio MP 171" "RICOH Aficio MP 171" "MFG:RICOH;CMD:PJL,RCS,PCL,PCLXL,PCL,PCLXL;MDL:Aficio MP 171;STS:10033,0;CLS:PRINTER;DES:RICOH Aficio MP 171;" "München"

[jpopelka]

Thanks,
there's some problem with how the ü in München is encoded.

The ü in Python is either b'\xc3\xbc' or u'\xfc'.

We use subprocess.Popen(["/usr/lib/cups/backend/snmp", host]).communicate()
and expect it returns sequence of bytes where the ü is encoded as b'\xc3\xbc'.
For some reason it returns bytes (as expected), but with ü encoded as b'\xfc'. And as you can see above b'\xfc' is nonsense.

The only explanation I have is bug in your Python version, because when I on my machine replace /usr/lib/cups/backend/snmp with a shell script returning the same as the snmp backend on your machine, everything works OK here.

[twaugh]

b'\xfc' is ü encoded as ISO-8859-1 instead of UTF-8. That's just an incorrect value (it has to be UTF-8, RFC 2571 says so).

I think there are two bugs here:

• We aren't trying to decode SNMP output as UTF-8, but should.
• When decoding SNMP output fails, we need to handle it gracefully.

[jpopelka]

We aren't trying to decode SNMP output as UTF-8, but should.

Well, the decode() in Python 3 uses utf-8 codec by default, as can be seen in the traceback. But specifying it won't do any harm.

[jpopelka]

b'\xfc' is ü encoded as ISO-8859-1 instead of UTF-8.

You're right as always :-) but how did you get that ?

[twaugh]

It was a guess, verified with echo ü | iconv -f utf-8 -t iso-8859-1 | od -tx1.

[philipp-classen]

Very nice. :-) If I can help you by trying out a potential patch, just let me know.

[jpopelka]

b'\xfc' is ü encoded as ISO-8859-1 instead of UTF-8.

Right, because when it has the same value in unicode as well as encoded it must be iso-8859-1 encoded, because "ISO-8859-1 was incorporated as the first 256 code points of Unicode." (https://en.wikipedia.org/wiki/ISO/IEC_8859-1#Similar_character_sets)

[jpopelka]

Philipp, if you need the fix now, you can change your /usr/share/system-config-printer/newprinter.py according to e4b9c62