Python 3.0 beta 2 : json and urllib not working together?

[swaroopch]

BPO	3763
Nosy	@etrepum, @orsenthil, @pitrou
Superseder	bpo-4136: merge json library with latest simplejson 2.0.x
Files	yahoo_search.py

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2009-04-01.05:04:41.062>
created_at = <Date 2008-09-03.09:15:49.318>
labels = ['type-bug', 'library']
title = 'Python 3.0 beta 2 : json and urllib not working together?'
updated_at = <Date 2009-04-01.05:04:40.968>
user = 'https://bugs.python.org/swaroopch'

bugs.python.org fields:

activity = <Date 2009-04-01.05:04:40.968>
actor = 'orsenthil'
assignee = 'none'
closed = True
closed_date = <Date 2009-04-01.05:04:41.062>
closer = 'orsenthil'
components = ['Library (Lib)']
creation = <Date 2008-09-03.09:15:49.318>
creator = 'swaroopch'
dependencies = []
files = ['11354']
hgrepos = []
issue_num = 3763
keywords = []
message_count = 7.0
messages = ['72379', '72380', '72396', '72400', '72421', '84981', '84982']
nosy_count = 4.0
nosy_names = ['bob.ippolito', 'orsenthil', 'pitrou', 'swaroopch']
pr_nums = []
priority = 'normal'
resolution = 'wont fix'
stage = None
status = 'closed'
superseder = '4136'
type = 'behavior'
url = 'https://bugs.python.org/issue3763'
versions = ['Python 3.0']

[swaroopch]

Hi,

Running the attached program in Python 3.0 beta 2 gives the following error:

File "C:\Python30\lib\json\decoder.py", line 21, in linecol
lineno = doc.count('\n', 0, pos) + 1
TypeError: expected an object with the buffer interface

I can't figure out if there's an error in the program itself, but I
suspect this isn't working as expected.

Please let me know if there's anything I can do to help (if this is a
bug indeed).

Regards,
Swaroop

[swaroopch]

Adding full traceback:

$ python yahoo_search.py
Traceback (most recent call last):
  File "yahoo_search.py", line 35, in <module>
    for result in search(query)['Result']:
  File "yahoo_search.py", line 28, in search
    result = json.load(urllib.request.urlopen(url))
  File "C:\Python30\lib\json\__init__.py", line 267, in load
    parse_constant=parse_constant, **kw)
  File "C:\Python30\lib\json\__init__.py", line 307, in loads
    return _default_decoder.decode(s)
  File "C:\Python30\lib\json\decoder.py", line 322, in decode
    raise ValueError(errmsg("Extra data", s, end, len(s)))
  File "C:\Python30\lib\json\decoder.py", line 30, in errmsg
    lineno, colno = linecol(doc, pos)
  File "C:\Python30\lib\json\decoder.py", line 21, in linecol
    lineno = doc.count('\n', 0, pos) + 1
TypeError: expected an object with the buffer interface

[orsenthil]

On the code against the trunk, I am getting the following error:

Traceback (most recent call last):
  File "python3k_json.py", line 38, in <module>
    for result in search(query)['Result']:
  File "python3k_json.py", line 31, in search
    result = json.load(obj)
  File "/usr/local/lib/python3.0/json/__init__.py", line 267, in load
    parse_constant=parse_constant, **kw)
  File "/usr/local/lib/python3.0/json/__init__.py", line 307, in loads
    return _default_decoder.decode(s)
  File "/usr/local/lib/python3.0/json/decoder.py", line 319, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
TypeError: can't use a string pattern on a bytes-like object

Swaroop: What encoding would be the JSON File content? You can try by
passing the encoding argument to the load method. I tried latin1 and
ascii, did not help.

Few more things to note:

• The above TypeError was introduced by the fix of bpo-2834.
• There are also bugs open in other modules (shutil, imaplib) where
  problems with str<->bytes conversions are observed.

[swaroopch]

Hi Senthil,

I am not aware of what encoding is used.

An example of the content is
http://search.yahooapis.com/WebSearchService/V1/webSearch?query=byte+of+python&appid=jl22psvV34HELWhdfUJbfDQzlJ2B57KFS_qs4I8D0Wz5U5_yCI1Awv8.lBSfPhwr&results=20&start=1&output=json

( If the above link does not work properly, please use http://is.gd/2bbI )

When viewing this, Firefox says it is UTF-8.

[pitrou]

If you look at the headers of HTTP response, the encoding is utf-8. You
should also be able to get this information by calling the info() method
on the return value of urlopen().

[orsenthil]

This is superseded by bpo-4136.

[orsenthil]

This is superseded by bpo-4136.