inspect_response(response) yields incorrect response in IPython shell #396
Comments
Bug Confirmed. SPIDER: from scrapy.spider import BaseSpider
from scrapy.http import Request
from scrapy.shell import inspect_response
class I396Spider(BaseSpider):
name = 'i396'
start_urls = ('http://httpbin.org/stream/1', 'http://httpbin.org/stream/2')
def parse(self, response):
print response.body
inspect_response(response) OUTPUT:
|
IPython embedding code borrowed from pallets/werkzeug#85
IPython embedding code borrowed from pallets/werkzeug#85
This bug is still happening, with same spider just change: inspect_response(response) to inspect_response(response, self) Even if I inspect the second response, inspect_response returns data from the first one. scrapy version 1.2.1 output:
|
@Tarliton , thank you for reporting.
When running the test spider, after the 2nd Ctrl+D, response is still pointing to the 1st received response:
Note that using the standard python shell, it's working fine:
|
Same with Python 3:
|
@redapple as we can see, this only happens with ipython. I was debugging for a while and found this: file: @wraps(_embed_ipython_shell)
def wrapper(namespace=namespace, banner=''):
config = load_default_config()
# Always use .instace() to ensure _instance propagation to all parents
# this is needed for <TAB> completion works well for new imports
shell = InteractiveShellEmbed.instance(
banner1=banner, user_ns=namespace, config=config)
shell()
return wrapper So the wrapper for ipython calls the instance method. If we check the instance method, the doc string says this: file: @classmethod
def instance(cls, *args, **kwargs):
"""Returns a global instance of this class.
This method create a new instance if none have previously been created
and returns a previously created instance is one already exists.
The arguments and keyword arguments passed to this method are passed
on to the :meth:`__init__` method of the class upon instantiation.
Examples
--------
Create a singleton class using instance, and retrieve it::
>>> from traitlets.config.configurable import SingletonConfigurable
>>> class Foo(SingletonConfigurable): pass
>>> foo = Foo.instance()
>>> foo == Foo.instance()
True
Create a subclass that is retrived using the base class instance::
>>> class Bar(SingletonConfigurable): pass
>>> class Bam(Bar): pass
>>> bam = Bam.instance()
>>> bam == Bar.instance()
True
""" It's returning a global instance, so the first inspect_response is creating this shell and the next ones are reusing the same shell with old variables. Shall this be correct? |
Hm, I'm pretty new to embedded shell and all. |
I think it's working with this change:
I'll continue checking. |
that instance part was added here #2229 |
@Tarliton , indeed. It works in Scrapy 1.1.3 (the regression appears in Scrapy 1.2) |
The InteractiveShellEmbed class is a singleton and we need to drop the instance with its clear_instance() method to rebuild the instance from scratch with fresh environment for each subsequent drop in.
Thanks @ahlinc , I'll check on my end. |
[MRG+1] Fix #396 re-triggered issue
Example case (requires registration at example site, and even then would be hard to use as a use-case; modify to suit your needs): http://pastebin.com/GT8N893q
In the above example, the response.meta printout in after_submit callback does not match that within the inspect_response shell on the second iteration (the first is correct). It appears that inspect_response has a stale response the second time.
The text was updated successfully, but these errors were encountered: