Crash in Debian sid, related to twisted version

[micahflee]

Downstream bug: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=793791

[ToBeReplaced]

Also downstream on Fedora 21+: https://bugzilla.redhat.com/show_bug.cgi?id=1246381 .
Twisted upgrade from 14 to 15 is the cause.

[quixotique]

Installing the latest version of txsocksx from GitHub fixed this problem. Now instead of TypeError: _getEndpoint() takes exactly 4 arguments (2 given) I get the following:

Tor Browser Launcher
By Micah Lee, licensed under MIT
version 0.2.0
https://github.com/micahflee/torbrowser-launcher
Updating over Tor
Checking for update
Downloading https://www.torproject.org/projects/torbrowser/RecommendedTBBVersions
Download error: Connection was refused by other side: 111: Connection refused. <class 'twisted.internet.error.ConnectionRefusedError'>

Starting the Tor service fixes this problem as well.

[micahflee]

Thanks @quixotique. And eek, gotta try to fix this soon. torbrowser-launcher is marked for autoremoval from Debian testing because of this bug. I'll see if I can do it tomorrow, but then I'll be on an airplane and then at hacker conferences for a week... so it may need to wait.

[niklasf]

This might even be a bug in Twisted, because twisted/twisted@b4d2e01#diff-477bf6af9f412b5a20b42bdddca60a82L1426 changed _getEndpoint() but not all calls (e.g. the line from this backtrace).

Similar: scrapy/scrapy#1034

Can confirm that upgrading to txsocksx-1.15.0.2 helps: habnabit/txsocksx#13

@quixotique What can I try to reproduce Download error: Connection was refused by other side: 111: Connection refused. <class 'twisted.internet.error.ConnectionRefusedError'>?

[micahflee]

I can confirm that it helps too. I can't ship my own txsocksx with torbrowser-launcher, but luckily yesterday Debian sid upgraded to python-txsocksx 1.15.0.2. It can now successfully make http requests over Tor! But now I'm running into a problem related to certificate pinning:

micah@debian:~$ torbrowser-launcher 
Tor Browser Launcher
By Micah Lee, licensed under MIT
version 0.2.0
https://github.com/micahflee/torbrowser-launcher
Creating GnuPG homedir /home/micah/.local/share/torbrowser/gnupg_homedir
Updating over Tor
Checking for update
Downloading https://www.torproject.org/projects/torbrowser/RecommendedTBBVersions
Checking to see if update is needed
TBB is not installed, attempting to install 4.5.3
Downloading https://dist.torproject.org/torbrowser/4.5.3/sha256sums.txt
Unhandled Error
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/twisted/web/_newclient.py", line 916, in dispatcher
    return func(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/twisted/web/_newclient.py", line 1472, in _finishResponse_WAITING
    self._giveUp(Failure(reason))
  File "/usr/lib/python2.7/dist-packages/twisted/web/_newclient.py", line 1525, in _giveUp
    self._disconnectParser(reason)
  File "/usr/lib/python2.7/dist-packages/twisted/web/_newclient.py", line 1513, in _disconnectParser
    parser.connectionLost(reason)
--- <exception caught here> ---
  File "/usr/lib/python2.7/dist-packages/twisted/web/_newclient.py", line 537, in connectionLost
    self.response._bodyDataFinished()
  File "/usr/lib/python2.7/dist-packages/twisted/web/_newclient.py", line 916, in dispatcher
    return func(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/twisted/web/_newclient.py", line 1161, in _bodyDataFinished_CONNECTED
    self._bodyProtocol.connectionLost(reason)
  File "/usr/lib/python2.7/dist-packages/torbrowser_launcher/launcher.py", line 375, in connectionLost
    self.all_done(reason)
  File "/usr/lib/python2.7/dist-packages/torbrowser_launcher/launcher.py", line 392, in response_finished
    self.run_task()
  File "/usr/lib/python2.7/dist-packages/torbrowser_launcher/launcher.py", line 314, in run_task
    self.attempt_update()
  File "/usr/lib/python2.7/dist-packages/torbrowser_launcher/launcher.py", line 541, in attempt_update
    self.build_ui()
  File "/usr/lib/python2.7/dist-packages/torbrowser_launcher/launcher.py", line 284, in build_ui
    self.start(None)
  File "/usr/lib/python2.7/dist-packages/torbrowser_launcher/launcher.py", line 293, in start
    self.run_task()
  File "/usr/lib/python2.7/dist-packages/torbrowser_launcher/launcher.py", line 318, in run_task
    self.download('signature', self.common.paths['sha256_url'], self.common.paths['sha256_file'])
  File "/usr/lib/python2.7/dist-packages/torbrowser_launcher/launcher.py", line 454, in download
    agent = SOCKS5Agent(reactor, VerifyTorProjectCert(self.common.paths['torproject_pem']), proxyEndpoint=torEndpoint)
  File "/usr/lib/python2.7/dist-packages/txsocksx/http.py", line 58, in __init__
    super(_SOCKSAgent, self).__init__(*a, **kw)
  File "/usr/lib/python2.7/dist-packages/txsocksx/http.py", line 37, in __init__
    'contextFactory must implement IPolicyForHTTPS')
exceptions.NotImplementedError: contextFactory must implement IPolicyForHTTPS

The crash is happening here: https://github.com/micahflee/torbrowser-launcher/blob/master/torbrowser_launcher/launcher.py#L454

And it seems like it has to do with the VerifyTorProjectCert class not implementing IPolicyForHTTPS:
https://github.com/micahflee/torbrowser-launcher/blob/master/torbrowser_launcher/launcher.py#L52

I'm about to head to the airport and won't be able to work on this for a week. With any luck Debian will update some more packages and the problem will just go away :) but if not any pull requests are appreciated.

[iseeareddoor]

The patch in the pull request works well (tested under debian sid) and seems largely cosmetic. Since this has been breaking the debian package for quite a while (at least in sid), merging and releasing this soonish would be nice.

[quixotique]

Hi @niklasf, I had configured torbrowser-launcher to only check for upgrades over Tor, and I normally used Vidalia to start and stop Tor. If I launched torbrowser-launcher without starting Tor first, then I would get the "Connection refused" error.

[shirishag75]

any updates to the patch gentleman ?

[kdeldycke]

I confirm this issue is fixed by the #196 pull-request.

See how I install and patch torbrowser-launcher on Kubuntu 15.10 in my dotfiles.

[micahflee]

Sorry I haven't merged it yet! Working on it now.

[micahflee]

I just merged #196. There's a new critical issue now (#202 is almost ready to fix it), so as soon as that's fixed as well I'll make a new release.

[cirosantilli]

+1 On launchpad: https://bugs.launchpad.net/ubuntu/+source/torbrowser-launcher/+bug/1495986