A Maze of Twisted Wormholes, None Alike

[davidlwillson]

Issue:

Lots of broken wormholes: mine, Rob's, now Dave's.

Symptom:

wormhole send and wormohole receive return an error.

$ wormhole send
Traceback (most recent call last):
  File "/usr/bin/wormhole", line 11, in <module>
    sys.exit(wormhole())
  File "/usr/lib/python2.7/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/lib/python2.7/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/lib/python2.7/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/lib/python2.7/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/click/decorators.py", line 27, in new_func
    return f(get_current_context().obj, *args, **kwargs)
  File "/usr/lib/python2.7/site-packages/wormhole/cli/cli.py", line 182, in send
    from . import cmd_send
  File "/usr/lib/python2.7/site-packages/wormhole/cli/cmd_send.py", line 7, in <module>
    from twisted.internet import reactor
  File "/usr/lib64/python2.7/site-packages/twisted/internet/reactor.py", line 38, in <module>
    from twisted.internet import default
  File "/usr/lib64/python2.7/site-packages/twisted/internet/default.py", line 56, in <module>
    install = _getInstallFunction(platform)
  File "/usr/lib64/python2.7/site-packages/twisted/internet/default.py", line 44, in _getInstallFunction
    from twisted.internet.epollreactor import install
  File "/usr/lib64/python2.7/site-packages/twisted/internet/epollreactor.py", line 24, in <module>
    from twisted.internet import posixbase
  File "/usr/lib64/python2.7/site-packages/twisted/internet/posixbase.py", line 18, in <module>
    from twisted.internet import error, udp, tcp
  File "/usr/lib64/python2.7/site-packages/twisted/internet/tcp.py", line 28, in <module>
    from twisted.internet._newtls import (
  File "/usr/lib64/python2.7/site-packages/twisted/internet/_newtls.py", line 21, in <module>
    from twisted.protocols.tls import TLSMemoryBIOFactory, TLSMemoryBIOProtocol
  File "/usr/lib64/python2.7/site-packages/twisted/protocols/tls.py", line 63, in <module>
    from twisted.internet._sslverify import _setAcceptableProtocols
  File "/usr/lib64/python2.7/site-packages/twisted/internet/_sslverify.py", line 38, in <module>
    TLSVersion.TLSv1_1: SSL.OP_NO_TLSv1_1,
AttributeError: 'module' object has no attribute 'OP_NO_TLSv1_1'

Fix:

http://stackoverflow.com/questions/42225773/install-scrapy-on-mac-successful-but-run-error

pip install Twisted==16.4.1

$ sudo pip install Twisted==16.4.1
Collecting Twisted==16.4.1
  Downloading Twisted-16.4.1.tar.bz2 (3.0MB)
    100% |████████████████████████████████| 3.0MB 406kB/s 
Requirement already satisfied: zope.interface>=3.6.0 in /usr/lib64/python2.7/site-packages (from Twisted==16.4.1)
Requirement already satisfied: setuptools in /usr/lib/python2.7/site-packages (from zope.interface>=3.6.0->Twisted==16.4.1)
Requirement already satisfied: six>=1.6.0 in /usr/lib/python2.7/site-packages (from setuptools->zope.interface>=3.6.0->Twisted==16.4.1)
Requirement already satisfied: appdirs>=1.4.0 in /usr/lib/python2.7/site-packages (from setuptools->zope.interface>=3.6.0->Twisted==16.4.1)
Requirement already satisfied: packaging>=16.8 in /usr/lib/python2.7/site-packages (from setuptools->zope.interface>=3.6.0->Twisted==16.4.1)
Requirement already satisfied: pyparsing in /usr/lib/python2.7/site-packages (from packaging>=16.8->setuptools->zope.interface>=3.6.0->Twisted==16.4.1)
Installing collected packages: Twisted
  Found existing installation: Twisted 17.1.0
    Uninstalling Twisted-17.1.0:
      Successfully uninstalled Twisted-17.1.0
  Running setup.py install for Twisted ... done
Successfully installed Twisted-16.4.1
[dlwillson@intecrowd-vboxws ~]$ wormhole send
Text to send:

Notes:

Even though that fixed a box, I remain confused about the actual problem, because another box has Twisted (17.1.0) and it works fine. (i.e. no downgrade needed)

[davidlwillson]

see also: scrapy/scrapy#2473

[davidlwillson]

Dave's fixed by downgrading Twisted and re-broke after upgrading Twisted.

[danselmi-intecrowd]

On Mac OSX using pip, wormhole broke when upgraded from 0.9.1 to 0.9.2 (though I'm sure it's a dependency that caused the breakage).

sudo -H pip install --ignore-installed magic-wormhole (resulting in 0..9.2 installed)

This fixes it:

sudo -H pip install --ignore-installed twisted==16.6.0

I don't know what that does because pip list shows the same output before/after that command, including twisted 17.1.0.

[davidlwillson]

Downgrading Twisted to v16 worked on Rob's machine, too.

[warner]

That sounds like you have an older pyOpenSSL, and the newer Twisted requires a newer one. I don't think it's magic-wormhole specific.

Could you try running just python -c "from twisted.internet import reactor" on that box, in both the twisted-17.1.0 case and the downgraded case? If everything is happy, that command will exit without printing anything, but if it's unhappy, then it should fail with that same "AttributeError" traceback you saw from invoking magic-wormhole. If that's the case, then at least we've removed magic-wormhole from the potential home of the problem.

It looks like twisted-17.1.0 was the first release to have a hard requirement that pyOpenSSL provides SSL.OP_NO_TLSv1_1. Twisted-16.6.0 and earlier used getattr(SSL, "OP_NO_TLSv1_1", 0) and could survive if pyOpenSSL didn't have it. Twisted correctly declares its dependency on the newer pyOpenSSL, but maybe some version of pip isn't respecting that declaration.

Try this:

% python
> import OpenSSL
> print OpenSSL
> print OpenSSL.__version__
> [name for name in dir(OpenSSL.SSL) if name.startswith("OP_NO_")]

that should tell you where pyOpenSSL is being imported from, what version it is, and a full list of all the OP_NO_ symbols it provides. That might give us a clue as to what's broken about it. My hunch is that you've got something ancient like pyOpenSSL-0.13.

Installing magic-wormhole directly into /usr/ with sudo pip means you're either using or replacing the other python libraries up there too. I don't know whether pip will see the dependency and upgrade your system pyOpenSSL for you, or if it will ignore it (and then you get runtime errors like this).

In general, my recommendation is to install magic-wormhole (and other python applications) into a virtualenv, where pip can install everything you need without affecting the rest of your system. I haven't found a nice way to do this for multiple users though (e.g. installing it once in /usr/local vs installing it with pipsi into ~/.local/ for every user on that box). You might check out Glyph's presentation at the last PyCon on the subject: https://www.youtube.com/watch?v=5BqAeN-F9Qs .

If you can't do that, then my recommendation is to upgrade the dependencies, rather than downgrade Twisted. I haven't yet looked at everything magic-wormhole needs from Twisted to determine what the oldest version we can possibly use is, but my usual assumption (not necessarily a good one) is it's at least whatever version of Twisted was present when I made the release. So running against a significantly older Twisted might run into other problems. If you discover those, let me know and I'll make magic-wormhole correctly declare a dependency upon the newer one. But the result will be the same: more demands from pesky software asking you to upgrade everything :).

[danselmi-intecrowd]

Thanks for the detailed explanation and link to the video. You're right that I have an old pyOpenSSL (0.13.1). Perhaps you could update your install instructions (for non-python developers) since pip install magic-wormhole wasn't very helpful to me.

I think I took two wrong turns trying to follow those directions. I used sudo. OK, I see that isn't a good idea. Because the Mac python install is protected from sudo, I used -ignore-installed and that results in getting a "successful" install that doesn't work.

So yeah, a virtualenv might solve my problem. But what I don't see (because the docs seem to be written for python developers) is, once I've installed wormhole in a virtualenv, where does my next python application go? Do I have to switch virtualenvs every time I want to run one or the other?

In the meantime, maybe installing python with homebrew allows me to use pip more easily. So I'll try that. (Turns out that I can brew install magic-wormhole so even better.)

[glyph]

Perhaps magic-wormhole's dependencies ought to be updated to include twisted[tls]>=something? twisted[tls] will unfortunately be satisfied even by versions of Twisted which don't include the tls extra, and pip install -U twisted sadly won't upgrade the TLS dependencies.

[glyph]

For what it's worth, this isn't really a bug in magic-wormhole, or indeed in twisted; it's actually a bug in Pip pypa/pip#988 that, I believe, is being worked on by a GSoC intern this year.

[danselmi-intecrowd]

I agree that this is a pip bug. So this issue could be closed.

My workaround was to install with homebrew, which saved me from pip and Apple's shipped python.

[warner]

I'm happy to increase our twisted[tls] dependency to something versioned. Any recommendations on what version would provoke the right down(up?)stream dependencies?

[warner]

We now depend upon twisted[tls] >= 17.5.0 (to ensure we've got access to an API that fixes #68). I think that ought to fix this as a side-effect. Closing it out.. thanks everyone for the help!