Can SGMLParser properly handle <empty/> tags?

[once-off]

BPO	5498
Nosy	@ezio-melotti, @merwok
Files	sgml_error.py

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2010-02-05.16:00:44.619>
created_at = <Date 2009-03-17.11:19:33.579>
labels = ['easy', 'expert-XML', 'type-bug', 'extension-modules', 'invalid', 'library']
title = 'Can SGMLParser properly handle <empty/> tags?'
updated_at = <Date 2010-02-05.16:23:31.773>
user = 'https://bugs.python.org/once-off'

bugs.python.org fields:

activity = <Date 2010-02-05.16:23:31.773>
actor = 'ezio.melotti'
assignee = 'none'
closed = True
closed_date = <Date 2010-02-05.16:00:44.619>
closer = 'ezio.melotti'
components = ['Extension Modules', 'Library (Lib)', 'XML']
creation = <Date 2009-03-17.11:19:33.579>
creator = 'once-off'
dependencies = []
files = ['13348']
hgrepos = []
issue_num = 5498
keywords = ['easy']
message_count = 3.0
messages = ['83667', '98424', '98425']
nosy_count = 3.0
nosy_names = ['ezio.melotti', 'eric.araujo', 'once-off']
pr_nums = []
priority = 'normal'
resolution = 'not a bug'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue5498'
versions = ['Python 2.6']

[once-off]

The attached script (sgml_error.py) was designed to output XML files
unchanged, other than expanding <empty/> tags into an opening and
closing tag, such as <empty></empty>.

It seems the SGMLParser class recognizes an empty tag, but does not emit
the closing tag until the NEXT forward slash it sees. So everything from
the forward slash in <empty/> (even the closing angle bracket) until the
next forward slash is considered to be textual data. See the following
line output.

Have I missed something here (like a conscious design limitation on the
class, an error on my part, etc), or is this really a bug with the class?

C:\Python24\Lib>python sgmllib.py H:\input.xml
start tag: <root>
data: '\n '
start tag: <tag1>
end tag: </tag1>
data: '\n '
start tag: <tag2>
data: '>\n <tag3>hello<'
end tag: </tag2>
data: 'tag3>\n'
end tag: </root>

C:\Python24\Lib>python
ActivePython 2.4.3 Build 12 (ActiveState Software Inc.) based on
Python 2.4.3 (#69, Apr 11 2006, 15:32:42) [MSC v.1310 32 bit (Intel)] on
win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import sgml_error

Input:
<root>
<tag1></tag1>
<tag2/>
<tag3>hello</tag3>
</root>

Output:
<root>
<tag1></tag1>
<tag2>>
<tag3>hello<</tag2>tag3>
</root>

Expected:
<root>
<tag1></tag1>
<tag2></tag2>
<tag3>hello</tag3>
</root>

[merwok]

Hello

XML of the form <tag/> are an SGML hack, or more precisely the combination of two features of SGML. The forward slash closes the tag, and the following angle bracket is character data, not part of the tag.

The W3C validator uses a real SGML parser for HTML doctypes, and fails on XML-like /> constructs: http://validator.w3.org/check?uri=data%3Atext%2Fhtml%2C%3C!DOCTYPE+html+PUBLIC+%22-%2F%2FW3C%2F%2FDTD+HTML+4.01%2F%2FEN%22+%22http%3A%2F%2Fwww.w3.org%2FTR%2Fhtml4%2Fstrict.dtd%22%3E+%3Chtml%3E+%3Chead%3E+++%3Ctitle%3ETest%3C%2Ftitle%3E+++%3Cmeta+name%3Dtest+content%3Done%2F%3E+++%3Cmeta+name%3Dbug+content%3Dtwo%3E+%3C%2Fhead%3E+%3Cbody%3E+++%3Cp%3ETest%3C%2Fp%3E+%3C%2Fbody%3E+%3C%2Fhtml%3E&charset=%28detect+automatically%29&doctype=Inline&group=1&verbose=1

The complete explanation can be read at http://www.cs.tut.fi/~jkorpela/html/empty.html

In conclusion, sgmllib is right. Use an XML parser for XML or an HTML5 parser for HTML.

Kind regards

[merwok]

Damn, the URI got fubared :/ Anyway, I just wanted to give an example of the verbose error message, but the second link will contain enough explanation.

Regards