Periodically able to cause moved functions to fail delattr()

[benjaminp]

Originally reported by: Joshua Harlow (Bitbucket: harlowja, GitHub: harlowja)

---

When running a piece of threaded code that directly uses six.moves.$XYZ:

For example:

https://github.com/openstack/taskflow/blob/master/taskflow/examples/jobboard_produce_consume_colors.py

I am periodically able to trigger an unusually occurrence with-in six (it doesn't occur on every run, but once every ~20 runs of the above example).

#!python

Traceback (most recent call last):
  File "/usr/lib64/python2.7/threading.py", line 810, in __bootstrap_inner
    self.run()
  File "/usr/lib64/python2.7/threading.py", line 763, in run
    self.__target(*self.__args, **self.__kwargs)
  File "taskflow/examples/jobboard_produce_consume_colors.py", line 140, in producer
    for i in six.moves.xrange(0, PRODUCER_UNITS):
  File "/home/josh/Dev/taskflow/.tox/py27/lib/python2.7/site-packages/six.py", line 90, in __get__
    delattr(obj.__class__, self.name)
AttributeError: xrange

Is it possible for the six.moves.xrange attribute to disappear somehow when running from daemon threads (although those daemon threads are joined on by the parent). Perhaps there is some underlying issue with the above six code?

This is with six==1.7.3 (if that matters).

Full environment @ http://paste.ubuntu.com/8495305/

---

• Bitbucket: https://bitbucket.org/gutworth/six/issue/98

[benjaminp]

Original comment by Joshua Harlow (Bitbucket: harlowja, GitHub: harlowja):

---

Also:

$ python --version
Python 2.7.8

[benjaminp]

Original comment by Joshua Harlow (Bitbucket: harlowja, GitHub: harlowja):

---

Another example occurrence:

#!python

Exception in thread Thread-6:
Traceback (most recent call last):
  File "/usr/lib64/python2.7/threading.py", line 810, in __bootstrap_inner
    self.run()
  File "/usr/lib64/python2.7/threading.py", line 763, in run
    self.__target(*self.__args, **self.__kwargs)
  File "taskflow/examples/jobboard_produce_consume_colors.py", line 109, in worker
    board.claim(job, name)
  File "/home/josh/Dev/taskflow/taskflow/jobs/backends/impl_zookeeper.py", line 533, in claim
    kazoo_utils.checked_commit(txn)
  File "/home/josh/Dev/taskflow/taskflow/utils/kazoo_utils.py", line 103, in checked_commit
    for op, result in six.moves.zip(txn.operations, results):
  File "/home/josh/Dev/taskflow/.tox/py27/lib/python2.7/site-packages/six.py", line 90, in __get__
    delattr(obj.__class__, self.name)
AttributeError: zip

[benjaminp]

Original comment by Alex Gaynor (Bitbucket: alex_gaynor, GitHub: Unknown):

---

The problem is _LazyDescr.__get__ isn't thread-safe. If two threads both enter this method concurrently, this issue will always be hit.

[benjaminp]

Original comment by Joshua Harlow (Bitbucket: harlowja, GitHub: harlowja):

---

A simple example that can be used to test this:

http://paste.ubuntu.com/8509049/

Eventually this will get

#!python

Exception in thread Thread-1:
Traceback (most recent call last):
  File "/home/y/lib/python2.6/threading.py", line 532, in __bootstrap_inner
    self.run()
  File "/home/y/lib/python2.6/threading.py", line 484, in run
    self.__target(*self.__args, **self.__kwargs)
  File "test.py", line 11, in a
    for (i, i) in six.moves.zip(stuff, stuff):
  File "/homes/harlowja/dev/os/taskflow/.venv/lib/python2.6/site-packages/six.py", line 92, in __get__
    delattr(obj.__class__, self.name)
AttributeError: zip

[benjaminp]

Original comment by Joshua Harlow (Bitbucket: harlowja, GitHub: harlowja):

---

I'm thinking we need to use imp.acquire_lock() in quite a few places to avoid this in six (one of those is in LazyDescr.__get__).

Thoughts?

[benjaminp]

Original comment by Alex Gaynor (Bitbucket: alex_gaynor, GitHub: Unknown):

---

It's not immediately obvious to me that the import lock is the right thing to hold here, the semantics around the import lock weird me out in general. I think each LazyDescr having its own lock is fine.

[benjaminp]

Original comment by Joshua Harlow (Bitbucket: harlowja, GitHub: harlowja):

---

Fair enough, let me see if I can get a patch up.

[benjaminp]

Merged in harlowja/six/fix-delattr (pull request #54)

Fix issue #98

[benjaminp]

Merged in harlowja/six/fix-delattr (pull request #54)

Fix issue #98