HTMLParser in Python 2.7 doesn't recognize image tags wrapped up in link tags

[Ari]

BPO	29276
Nosy	@zhangyangyu, @brendan-donegan

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2017-01-14.17:59:49.049>
created_at = <Date 2017-01-14.15:52:38.605>
labels = ['invalid', 'type-bug', 'library']
title = "HTMLParser in Python 2.7 doesn't recognize image tags wrapped up in link tags"
updated_at = <Date 2017-01-14.17:59:49.048>
user = 'https://bugs.python.org/Ari'

bugs.python.org fields:

activity = <Date 2017-01-14.17:59:49.048>
actor = 'Ari'
assignee = 'none'
closed = True
closed_date = <Date 2017-01-14.17:59:49.049>
closer = 'Ari'
components = ['Library (Lib)']
creation = <Date 2017-01-14.15:52:38.605>
creator = 'Ari'
dependencies = []
files = []
hgrepos = []
issue_num = 29276
keywords = []
message_count = 4.0
messages = ['285490', '285491', '285494', '285495']
nosy_count = 3.0
nosy_names = ['xiang.zhang', 'brendan-donegan', 'Ari']
pr_nums = []
priority = 'normal'
resolution = 'not a bug'
stage = None
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue29276'
versions = ['Python 2.7']

[Ari]

The following code produces incorrect results under Python 2.7.13. One would expect it to print 2 lines, "Encountered a start tag: a" and "Encountered a start tag: img". Yet it prints only "Encountered a start tag: a".

from HTMLParser import HTMLParser
class MyHTMLParser(HTMLParser):
    def handle_starttag(self, tag, attrs):
        print 'Encountered a start tag: %s' % tag
parser = MyHTMLParser()
parser.feed('<a href="http://somesite.com/large_image.jpg"><img src="http://somesite.com/small_image.jpg" width="800px" /></a>')

Python 3.5.2 produces correct results on the same input and prints the expected "Encountered a start tag: a" and "Encountered a start tag: img".

from html.parser import HTMLParser
class MyHTMLParser(HTMLParser):
    def handle_starttag(self, tag, attrs):
        print("Encountered a start tag:", tag)
parser = MyHTMLParser()
parser.feed('<a href="http://somesite.com/large_image.jpg"><img src="http://somesite.com/small_image.jpg" width="800px" /></a>')

[zhangyangyu]

I can get the expected behaviour with the lastest 2.7 build.

Python 2.7.13+ (2.7:0d4e0a736688, Jan 15 2017, 00:51:57) 
[GCC 5.2.1 20151010] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from HTMLParser import HTMLParser
>>> class MyHTMLParser(HTMLParser):
...     def handle_starttag(self, tag, attrs):
...         print 'Encountered a start tag: %s' % tag
... 
>>> parser = MyHTMLParser()
>>> parser.feed('<a href="http://somesite.com/large_image.jpg"><img src="http://somesite.com/small_image.jpg" width="800px" /></a>')
Encountered a start tag: a
Encountered a start tag: img

[brendan-donegan]

I even get the correct behaviour in 2.7.12:
Python 2.7.12+ (default, Sep 17 2016, 12:08:02) 
[GCC 6.2.0 20160914] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from HTMLParser import HTMLParser
>>> class MyHTMLParser(HTMLParser):
...     def handle_starttag(self, tag, attrs):
...         print("Encountered a start tag: %s" % tag)
... 
>>> parser = MyHTMLParser()
>>> parser.feed('<a href="http://somesite.com/large_image.jpg"><img src="http://somesite.com/small_image.jpg" width="800px" /></a>')
Encountered a start tag: a
Encountered a start tag: img

Ari, can you provide more info about the exact version and platform you are using?

[Ari]

Sorry, it was a false alarm. I had a poorly constructed testcase. I was trying to grab all image links from html markup while simultaneously substituting image links to somesite.com with links to anothersite.com. I had handle_starttag and handle_startendtag defined in my class, but all the image processing resided in handle_starttag, and I didn't realize that handle_starttag is not automatically called when handle_startendtag is executed. So <img src="http://somesite.com/small_image.jpg" width="800px" /> went straight to handle_startendtag, and it did nothing to process the image link because handle_starttag was never called. When I removed handle_startendtag definition, I got the link I wanted (but the absense of handle_startendtag totally mangled the markup I was trying to produce).