Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

A couple of questions about default-zpublisher-encoding #571

Closed
jugmac00 opened this issue Apr 28, 2019 · 12 comments

Comments

@jugmac00
Copy link
Member

commented Apr 28, 2019

When revising #564 @dataflake made me aware of the default-zpublisher-encoding option.

I have a couple of questions /annotations about that one.

  1. The only documentation about that option seems to be the CHANGES.rst and the NEWS.rst.

Both refer to zope.conf.

I installed Zope via
$ bin/buildout -c zope-ecosystem.cfg
and under etc/ there is only a wsgi.conf and a zope.ini, no zope.conf.

Afaik zope.conf came with ZServer, which is no longer the default, and zope.ini / wsgi.conf comes with the new wsgi way.

If that's correct, we should update the news.rst.

  1. Can anybody please explain what is the "wsgischema.xml" for?

<key name="default-zpublisher-encoding"
datatype=".default_zpublisher_encoding"
default="utf-8">
<description>
This key controls what character set is used to encode unicode
data that reaches ZPublisher without any other specified encoding.
</description>
</key>

Is this a config file or only documentation?

I'd assume it's a config file, as it gets loaded here..

class ZopeWSGIOptions(object):
"""ZopeWSGIOptions parses a ZConfig schema and config file.
"""
configfile = None
confighandlers = None
configroot = None
schema = None
schemadir = os.path.dirname(os.path.abspath(__file__))
schemafile = 'wsgischema.xml'

  1. Wrong docstring

Having a look at...

@implementer(IUnicodeEncodingConflictResolver)
class Z2UnicodeEncodingConflictResolver(object):
""" This resolver tries to lookup the encoding from the
'default-zpublisher-encoding' setting in the Zope configuration
file and defaults to sys.getdefaultencoding().
"""
def __init__(self, mode='strict'):
self.mode = mode
def resolve(self, context, text, expression):
if isinstance(text, text_type):
return text
try:
return text.decode('ascii')
except UnicodeDecodeError:
encoding = ZPublisher.HTTPRequest.default_encoding
try:
return text.decode(encoding, errors=self.mode)
except UnicodeDecodeError:
# finally try the old management_page_charset default
return text.decode('iso-8859-15', errors=self.mode)

I'd think the docstring is wrong.

a) the encoding is not used from "default-zpublisher-encoding" but from
"ZPublisher.HTTPRequest.default_encoding" which is hardcoded to "utf-8."

# This may get overwritten during configuration
default_encoding = 'utf-8'

b) the fallback is "iso-8859-15", and not "sys.getdefaultencoding".

c) Does the class even get used?

The class gets used at the bottom of the module

StrictUnicodeEncodingConflictResolver = \
Z2UnicodeEncodingConflictResolver('strict')
ReplacingUnicodeEncodingConflictResolver = \
Z2UnicodeEncodingConflictResolver('replace')
IgnoringUnicodeEncodingConflictResolver = \
Z2UnicodeEncodingConflictResolver('ignore')

but these new identifiers are only used for tests.

So it looks like the code could be removed.

I have done a quick search on github - no results except the tests. Or is that a public API we'd have to deprecate?

@dataflake

This comment has been minimized.

Copy link
Member

commented Apr 28, 2019

  1. The setting applies both to ZServer and WSGI setups and it predates WSGI by a lot. It makes no sense to go back and change all references to from zope.conf to wsgi.conf

  2. wsgischema.xml describes what you can put into the wsgi.conf file as legal configuration settings. The exact same thing exists in the ZServer package for ZServer setups

  3. ZPublisher.HTTPRequest.default_encoding gets manipulated during startup and is set to the value of default-zpublisher-encoding. The docstring is obviously wrong, just go change it.

Don't get hung up about something as inconsequential as a small class that doesn't get used, please. There's more important stuff to do, really.

@d-maurer

This comment has been minimized.

Copy link
Contributor

commented Apr 28, 2019

@dataflake

This comment has been minimized.

Copy link
Member

commented Apr 28, 2019

What the configuration name is called, be it wsgi.conf or zope.conf, was just convention. I believe the intent was to say "this is a zope configuration for WSGI setups only", because it is likely to fail when starting a ZServer instance with it. There are some configuration keys that differ.

I'm open to switch the naming so that mkwsgiinstance creates a zope.conf and a wsgi.ini as opposed to a wsgi.conf and a zope.ini, but is this really important?

@d-maurer

This comment has been minimized.

Copy link
Contributor

commented Apr 28, 2019

@jamadden

This comment has been minimized.

Copy link
Member

commented Apr 29, 2019

Zope relies on ZConfig for its configuration.

ZConfig gets the configuration options applicable for a given application or application component from an XML file (usually called component[s].xml or ***schmema.xml). The schema has (typically) detailed documentation for each option.

Beginning with ZConfig 3.2, ZConfig ships with a Sphinx extension that can automatically publish that documentation. (For example, that's how ZConfig's logging documentation is generated.)

.. zconfig:: Zope2.Startup
    :file: wsgischema.xml
@dataflake

This comment has been minimized.

Copy link
Member

commented May 3, 2019

@jamadden Have you actually tried that ZConfig Sphinx directive? It appears to expect files starting with <component> as it tries to use a parser subclass that can only deal with such a start tag. The correct parser is not used.

reading sources... [100%] config_keys
Exception occurred:
  File "/Users/jens/src/.eggs/ZConfig-3.4.0-py3.7.egg/ZConfig/_compat.py", line 79, in reraise
    raise value
ZConfig.SchemaError: Unknown document type schema (line 1, column 0)

Full traceback:

  File "/Users/jens/src/.eggs/ZConfig-3.4.0-py3.7.egg/ZConfig/sphinx.py", line 161, in run
    True, self.options.get('file'))
  File "/Users/jens/src/.eggs/ZConfig-3.4.0-py3.7.egg/ZConfig/_schema_utils.py", line 321, in load_schema
    schema = ZConfig.loader.loadSchemaFile(schema_reader)
  File "/Users/jens/src/.eggs/ZConfig-3.4.0-py3.7.egg/ZConfig/loader.py", line 65, in loadSchemaFile
    return SchemaLoader().loadFile(file, url)
  File "/Users/jens/src/.eggs/ZConfig-3.4.0-py3.7.egg/ZConfig/loader.py", line 177, in loadFile
    return self.loadResource(r)
  File "/Users/jens/src/.eggs/ZConfig-3.4.0-py3.7.egg/ZConfig/loader.py", line 366, in loadResource
    schema = ZConfig.schema.parseResource(resource, self)
  File "/Users/jens/src/.eggs/ZConfig-3.4.0-py3.7.egg/ZConfig/schema.py", line 32, in parseResource
    xml.sax.parse(resource.file, parser)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/xml/sax/__init__.py", line 33, in parse
    parser.parse(source)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/xml/sax/expatreader.py", line 111, in parse
    xmlreader.IncrementalParser.parse(self, source)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/xml/sax/xmlreader.py", line 125, in parse
    self.feed(buffer)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/xml/sax/expatreader.py", line 217, in feed
    self._parser.Parse(data, isFinal)
  File "/private/tmp/python-20190327-98828-4pywv0/Python-3.7.3/Modules/pyexpat.c", line 417, in StartElement
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/xml/sax/expatreader.py", line 333, in start_element
    self._cont_handler.startElement(name, AttributesImpl(attrs))
  File "/Users/jens/src/.eggs/ZConfig-3.4.0-py3.7.egg/ZConfig/schema.py", line 112, in startElement
    getattr(self, "start_" + name)(attrs)
  File "/Users/jens/src/.eggs/ZConfig-3.4.0-py3.7.egg/ZConfig/schema.py", line 321, in start_import
    self.loadComponent(src)
  File "/Users/jens/src/.eggs/ZConfig-3.4.0-py3.7.egg/ZConfig/schema.py", line 327, in loadComponent
    xml.sax.parse(r.file, parser)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/xml/sax/__init__.py", line 33, in parse
    parser.parse(source)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/xml/sax/expatreader.py", line 111, in parse
    xmlreader.IncrementalParser.parse(self, source)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/xml/sax/xmlreader.py", line 125, in parse
    self.feed(buffer)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/xml/sax/expatreader.py", line 217, in feed
    self._parser.Parse(data, isFinal)
  File "/private/tmp/python-20190327-98828-4pywv0/Python-3.7.3/Modules/pyexpat.c", line 417, in StartElement
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/xml/sax/expatreader.py", line 333, in start_element
    self._cont_handler.startElement(name, AttributesImpl(attrs))
  File "/Users/jens/src/.eggs/ZConfig-3.4.0-py3.7.egg/ZConfig/schema.py", line 99, in startElement
    self.error("Unknown document type " + name)
  File "/Users/jens/src/.eggs/ZConfig-3.4.0-py3.7.egg/ZConfig/schema.py", line 472, in error
    raise_with_same_tb(self.initerror(ZConfig.SchemaError(message)))
  File "/Users/jens/src/.eggs/ZConfig-3.4.0-py3.7.egg/ZConfig/_compat.py", line 104, in raise_with_same_tb
    reraise(type(exception), exception, sys.exc_info()[2])
  File "/Users/jens/src/.eggs/ZConfig-3.4.0-py3.7.egg/ZConfig/_compat.py", line 79, in reraise
    raise value
ZConfig.SchemaError: Unknown document type schema (line 1, column 0)
@jamadden

This comment has been minimized.

Copy link
Member

commented May 3, 2019

I didn't realize it wasn't actually a component file when I made that above remark. It's true that directive currently just takes component files. I think there may be a workaround involving a dummy component file, but ideally the ZConfig directive would just handle that. (zopefoundation/ZConfig#59)

@jamadden

This comment has been minimized.

Copy link
Member

commented May 3, 2019

I've got a PR to ZConfig to make the directive handle schemas.

@dataflake

This comment has been minimized.

Copy link
Member

commented May 3, 2019

@jamadden Thank you! I'm not sure how active they are, though.

@jamadden

This comment has been minimized.

Copy link
Member

commented May 3, 2019

I wrote the original Sphinx extension plus this PR and I have PyPI access for ZConfig, so getting it released won't be an issue. We just need someone to swing by and review. I've had several different reviewers in that repo in the past, usually within a day or so.

@dataflake

This comment has been minimized.

Copy link
Member

commented May 3, 2019

Great - sorry, I had not clicked on the link to the PR, otherwise I wouldn't have said that.

@icemac icemac added this to To do in Zope 4 final release via automation May 6, 2019

@icemac icemac added this to the 4.0 final milestone May 6, 2019

Zope 4 final release automation moved this from To do to Done May 9, 2019

@dataflake

This comment has been minimized.

Copy link
Member

commented Jun 29, 2019

A configuration reference based on the schema is now live at https://zope.readthedocs.io/en/latest/operation.html#configuration-reference

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
5 participants
You can’t perform that action at this time.