Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

RecentlyUsedContainer is not threadsafe #7

Closed
christer opened this Issue · 4 comments

2 participants

Christer Sjöholm Andrey Petrov
Christer Sjöholm

I'm testing 1.0.1

As noted in the code, RecentlyUsedContainer is not threadsafe.

I'm getting: RuntimeError: deque mutated during iteration

Stacktrace:

  File "/home/chrsjo/src/arkenutils_omkomp/arken/hcap.py", line 153, in _request
    return self._pool.request(*args, **kwargs)
  File ".../urllib3-1.0.1-py2.7.egg/urllib3/request.py", line 65, in request
    **urlopen_kw)
  File ".../urllib3-1.0.1-py2.7.egg/urllib3/request.py", line 78, in request_encode_url
    return self.urlopen(method, url, **urlopen_kw)
  File ".../urllib3-1.0.1-py2.7.egg/urllib3/poolmanager.py", line 107, in urlopen
    conn = self.connection_from_url(url)
  File ".../urllib3-1.0.1-py2.7.egg/urllib3/poolmanager.py", line 98, in connection_from_url
    return self.connection_from_host(host, port=port, scheme=scheme)
  File ".../urllib3-1.0.1-py2.7.egg/urllib3/poolmanager.py", line 73, in connection_from_host
    pool = self.pools.get(pool_key)
  File "/home/chrsjo/.virtualenvs/arkenutils_omkomp/lib/python2.7/_abcoll.py", line 342, in get 
    return self[key]
  File ".../urllib3-1.0.1-py2.7.egg/urllib3/_collections.py", line 96, in __getitem__
    self._prune_invalidated_entries()
  File ".../urllib3-1.0.1-py2.7.egg/urllib3/_collections.py", line 77, in _prune_invalidated_entries
    self.access_log = deque(e for e in self.access_log if e.is_valid)
  File ".../urllib3-1.0.1-py2.7.egg/urllib3/_collections.py", line 77, in <genexpr>
    self.access_log = deque(e for e in self.access_log if e.is_valid)
RuntimeError: deque mutated during iteration

Christer Sjöholm

I quickly added some locking
christer@5923463

Andrey Petrov
Owner

Perfect, thank you @christer. I've been meaning to do this and I appreciate you pushing this through.

Looks good at first glance. I'll do a review later today and merge it in.

Andrey Petrov
Owner

I merged and ported over your changes to <Py26 (acquire/release instead of with) and made some other minor improvements here:

4a68690

I'm wondering whether we should expand the scope of the mutex to the entire methods rather than just that one deque. Hmm. Or perhaps I'll write some of the surrounding code a bit more defensively.

Andrey Petrov
Owner

Alright I reviewed the code and I feel fairly confident about it. I'll do a 1.0.2 release later today with these changes included.

Andrey Petrov shazow closed this
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.