inspect_response(response) yields incorrect response in IPython shell #396
Comments
|
Bug Confirmed. SPIDER: from scrapy.spider import BaseSpider
from scrapy.http import Request
from scrapy.shell import inspect_response
class I396Spider(BaseSpider):
name = 'i396'
start_urls = ('http://httpbin.org/stream/1', 'http://httpbin.org/stream/2')
def parse(self, response):
print response.body
inspect_response(response)OUTPUT: |
dangra
added a commit
that referenced
this issue
Oct 10, 2013
IPython embedding code borrowed from pallets/werkzeug#85
gcmalloc
added a commit
to gcmalloc/scrapy
that referenced
this issue
Oct 10, 2013
IPython embedding code borrowed from pallets/werkzeug#85
|
This bug is still happening, with same spider just change: inspect_response(response)to inspect_response(response, self)Even if I inspect the second response, inspect_response returns data from the first one. scrapy version 1.2.1 output: |
|
@Tarliton , thank you for reporting. When running the test spider, after the 2nd Ctrl+D, response is still pointing to the 1st received response: Note that using the standard python shell, it's working fine: |
redapple
changed the title
|
Same with Python 3: |
|
@redapple as we can see, this only happens with ipython. I was debugging for a while and found this: file: @wraps(_embed_ipython_shell)
def wrapper(namespace=namespace, banner=''):
config = load_default_config()
# Always use .instace() to ensure _instance propagation to all parents
# this is needed for <TAB> completion works well for new imports
shell = InteractiveShellEmbed.instance(
banner1=banner, user_ns=namespace, config=config)
shell()
return wrapperSo the wrapper for ipython calls the instance method. If we check the instance method, the doc string says this: file: @classmethod
def instance(cls, *args, **kwargs):
"""Returns a global instance of this class.
This method create a new instance if none have previously been created
and returns a previously created instance is one already exists.
The arguments and keyword arguments passed to this method are passed
on to the :meth:`__init__` method of the class upon instantiation.
Examples
--------
Create a singleton class using instance, and retrieve it::
>>> from traitlets.config.configurable import SingletonConfigurable
>>> class Foo(SingletonConfigurable): pass
>>> foo = Foo.instance()
>>> foo == Foo.instance()
True
Create a subclass that is retrived using the base class instance::
>>> class Bar(SingletonConfigurable): pass
>>> class Bam(Bar): pass
>>> bam = Bam.instance()
>>> bam == Bar.instance()
True
"""It's returning a global instance, so the first inspect_response is creating this shell and the next ones are reusing the same shell with old variables. Shall this be correct? |
|
Hm, I'm pretty new to embedded shell and all. |
|
I think it's working with this change: I'll continue checking. |
|
that instance part was added here #2229 |
|
@Tarliton , indeed. It works in Scrapy 1.1.3 (the regression appears in Scrapy 1.2) |
ahlinc
added a commit
to ahlinc/scrapy
that referenced
this issue
Nov 30, 2016
The InteractiveShellEmbed class is a singleton and we need to drop the instance with its clear_instance() method to rebuild the instance from scratch with fresh environment for each subsequent drop in.
|
Thanks @ahlinc , I'll check on my end. |
eliasdorneles
added a commit
that referenced
this issue
Nov 30, 2016
[MRG+1] Fix #396 re-triggered issue
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Example case (requires registration at example site, and even then would be hard to use as a use-case; modify to suit your needs): http://pastebin.com/GT8N893q
In the above example, the response.meta printout in after_submit callback does not match that within the inspect_response shell on the second iteration (the first is correct). It appears that inspect_response has a stale response the second time.
The text was updated successfully, but these errors were encountered: