Avoid UnicodeDecoderError: use binary stream for xml files, let SAX determine the encoding

[StefanBruens]

Currently zone_reader from zone.py:

firewalld/src/firewall/core/io/zone.py

Line 699 in 10bcda6

with open(name, "r") as f:

and *_reader from the other io classes open the file as character stream, which fails as soon as e.g. the "C" locale is used and the xml files contains non-ASCII character.

SAX allows to feed a binary stream into the parser, and let SAX determine the encoding from the XML encoding pseudo attribute.

The following code works with both Python2 and Python3:

#! /usr/bin/python3
# -*- coding: utf-8 -*-

import xml.sax as sax
import codecs

name = "./foo.xml"
parser = sax.make_parser()

with open(name, "rb") as f:
  source = sax.InputSource(None)
  source.setByteStream(f)
  try:
    parser.parse(source)
  except Exception as e:
    print(e)

This is able to parse the following XML file:

<?xml version="1.0" encoding="utf-8"?>
<foo>
  <a>
    asdf
  </a>
  <b>
    äöü
  </b>
</foo>

[erig0]

Can you tell me what version of python you're using? I can reproduce this on 3.4 and 3.6, but not 3.7

$ LC_ALL=C python3.4 foo.py
'ascii' codec can't decode byte 0xc3 in position 99: ordinal not in range(128)

$ LC_ALL=C python3.6 foo.py
'ascii' codec can't decode byte 0xc3 in position 99: ordinal not in range(128)

$ LC_ALL=C python3.7 foo.py
works

[StefanBruens]

Currently 3.6.5, not sure which version I used in march.

On 3.7 it probably works cause of https://docs.python.org/3/whatsnew/3.7.html#whatsnew37-pep538

[erig0]

On 3.7 it probably works cause of https://docs.python.org/3/whatsnew/3.7.html#whatsnew37-pep538

Indeed. Thanks for the pointer.