Refactor tunnel #1188

NielsZeilemaker · 2015-02-01T16:09:25Z

Supercedes #1117, rebased on devel fixes hidden tunnels

tribler-ci · 2015-02-01T16:12:02Z

Merged build triggered.

tribler-ci · 2015-02-01T16:12:02Z

Merged build started.

tribler-ci · 2015-02-01T16:20:44Z

Merged build finished.

tribler-ci · 2015-02-01T16:20:44Z

Test FAILed.
Refer to this link for build results: http://jenkins.tribler.org/job/GH_Tribler_pull-request-tester_devel/3320/

NielsZeilemaker · 2015-02-01T17:44:38Z

@whirm the new tunnel crypto depens on cryptography. It's a package that exposes the gcm mode of openssl.

tribler-ci · 2015-02-01T17:47:02Z

Merged build triggered.

tribler-ci · 2015-02-01T17:47:02Z

Merged build started.

tribler-ci · 2015-02-01T17:52:02Z

Merged build triggered.

tribler-ci · 2015-02-01T17:52:02Z

Merged build started.

tribler-ci · 2015-02-01T17:55:48Z

Merged build finished.

tribler-ci · 2015-02-01T17:55:48Z

Test FAILed.
Refer to this link for build results: http://jenkins.tribler.org/job/GH_Tribler_pull-request-tester_devel/3321/

tribler-ci · 2015-02-01T18:00:47Z

Merged build finished.

tribler-ci · 2015-02-01T18:00:47Z

Test FAILed.
Refer to this link for build results: http://jenkins.tribler.org/job/GH_Tribler_pull-request-tester_devel/3322/

whirm · 2015-02-02T09:47:47Z

README.md

@@ -15,7 +15,7 @@ We make use of submodules, so remember using the --recursive argument when cloni

 ### Debian/Ubuntu/Mint
 ```bash
-sudo apt-get install libav-tools libjs-excanvas libjs-mootools libx11-6 python-apsw python-cherrypy3 python-crypto python-feedparser python-gmpy python-libtorrent python-m2crypto python-netifaces python-pil python-pyasn1 python-requests python-twisted python-wxgtk2.8 python2.7 vlc
+sudo apt-get install libav-tools libjs-excanvas libjs-mootools libx11-6 python-apsw python-cherrypy3 python-feedparser python-gmpy python-libtorrent python-m2crypto python-netifaces python-pil python-requests python-twisted python-wxgtk2.8 python2.7 vlc


@NielsZeilemaker could you add python-cryptography? also in debian/control

I guess it's python-crypto.

@LipuFei, its python-cryptography python-crypto was used before. I'll add the package

tribler-ci · 2015-02-02T09:57:02Z

Merged build triggered.

tribler-ci · 2015-02-02T09:57:02Z

Merged build started.

whirm · 2015-02-02T10:00:00Z

@NielsZeilemaker I've updated the gorillas, let's see how it goes.

tribler-ci · 2015-02-02T10:16:56Z

Merged build finished.

tribler-ci · 2015-02-02T10:16:57Z

Test FAILed.
Refer to this link for build results: http://jenkins.tribler.org/job/GH_Tribler_pull-request-tester_devel/3323/

whirm · 2015-02-02T10:19:09Z

retest this please

whirm · 2015-02-02T10:19:28Z

seems that it got executed while I was installing the package :)

synctext · 2015-02-02T10:20:28Z

Almost :-)

Traceback (most recent call last):
  File "/usr/lib/python2.7/unittest/case.py", line 329, in run
    testMethod()
  File "/usr/lib/python2.7/dist-packages/nose/loader.py", line 411, in loadTestsFromName
    addr.filename, addr.module)
  File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 47, in importFromPath
    return self.importFromDir(dir_path, fqname)
  File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 94, in importFromDir
    mod = load_module(part_fqname, fh, filename, desc)
  File "/home/jenkins/workspace/GH_Tribler_pull-request-tester_devel/tribler/Tribler/Test/test_tunnel_community.py", line 16, in <module>
    from Tribler.community.tunnel.tunnel_community import TunnelSettings
  File "/home/jenkins/workspace/GH_Tribler_pull-request-tester_devel/tribler/Tribler/community/tunnel/tunnel_community.py", line 27, in <module>
    from Tribler.community.tunnel.crypto.tunnelcrypto import TunnelCrypto, CryptoException
  File "/home/jenkins/workspace/GH_Tribler_pull-request-tester_devel/tribler/Tribler/community/tunnel/crypto/tunnelcrypto.py", line 3, in <module>
    from cryptowrapper import crypto_box_beforenm, crypto_auth, crypto_auth_verify, Cipher, algorithms, modes, HKDFExpand, hashes, default_backend
  File "/home/jenkins/workspace/GH_Tribler_pull-request-tester_devel/tribler/Tribler/community/tunnel/crypto/cryptowrapper.py", line 14, in <module>
    from cryptography.hazmat.backends import default_backend
ImportError: No module named cryptography.hazmat.backends

tribler-ci · 2015-02-02T10:22:02Z

Merged build triggered.

tribler-ci · 2015-02-02T10:22:02Z

Merged build started.

tribler-ci · 2015-02-02T10:46:10Z

Merged build finished.

tribler-ci · 2015-02-02T10:46:10Z

Test FAILed.
Refer to this link for build results: http://jenkins.tribler.org/job/GH_Tribler_pull-request-tester_devel/3324/

synctext · 2015-02-02T10:50:17Z

Ai. Assert:
http://jenkins.tribler.org/job/GH_Tribler_pull-request-tester_devel/3324/artifact/output//Screenshot-35.png
Multiple Tracebacks

DEBUG   1422873500.43        dispersy:778   ignore vote for ('95.211.198.142', 49165) from ('95.211.198.142', 44738) (voter is within our LAN)
DEBUG   1422873500.43       community:2640  received introduction to {95.211.198.142:22872} from {95.211.198.142:44738}
ERROR   1422873500.43       community:2184  exception during check_callback for dispersy-puncture
Traceback (most recent call last):
  File "/home/jenkins/workspace/GH_Tribler_pull-request-tester_devel/tribler/Tribler/dispersy/community.py", line 2182, in on_messages
    possibly_messages = list(meta.check_callback(messages))
  File "/home/jenkins/workspace/GH_Tribler_pull-request-tester_devel/tribler/Tribler/dispersy/community.py", line 2865, in check_puncture
    identifiers_seen[message.payload.identifer] = message.candidate
AttributeError: 'Implementation' object has no attribute 'identifer'
DEBUG   1422873500.43       community:2011  got 1 incoming packets

INFO    1422873504.39 tunnel_community:420   could not create circuit, no available hops.
DEBUG   1422873504.39   SocketHandler:503   SocketHandler: Got UDP data ('95.211.198.142', 49165) len 166
DEBUG   1422873504.39        endpoint:72                              cell <-  95.211.198.142:49165  166 bytes
DEBUG   1422873504.39       community:2011  got 1 incoming packets
DEBUG   1422873504.39       community:2213  in... 1 cell messages from {95.211.198.142:49165}
DEBUG   1422873504.39 tunnel_community:751   Got create (4076263781) from ('95.211.198.142', 49165)
ERROR   1422873504.39        dispersy:1723  exception during handle_callback for cell
Traceback (most recent call last):
  File "/home/jenkins/workspace/GH_Tribler_pull-request-tester_devel/tribler/Tribler/dispersy/dispersy.py", line 1718, in _update
    messages[0].handle_callback(messages)
  File "/home/jenkins/workspace/GH_Tribler_pull-request-tester_devel/tribler/Tribler/community/tunnel/tunnel_community.py", line 766, in on_cell
    self.dispersy.on_incoming_packets([(message.candidate, TunnelConversion.convert_from_cell(packet))], False, source=u"circuit_%d" % circuit_id)
TypeError: on_incoming_packets() got an unexpected keyword argument 'source'
DEBUG   1422873504.39       community:2011  got 1 incoming packets
DEBUG   1422873504.39       community:2213  in... 1 cell messages from {95.211.198.142:49165}
DEBUG   1422873504.39 tunnel_community:751   Got create (3294018314) from ('95.211.198.142', 49165)

NielsZeilemaker · 2015-02-02T11:01:49Z

I guess it needs a new dispersy pointer, let me have a look to see if I can push one in.

NielsZeilemaker · 2015-02-02T11:05:05Z

I'll have to do it tonight as I don't have access to my laptop, alternatively @whirm can you send me a pull request with the new dispersy pointer?

synctext · 2015-02-02T11:39:27Z

@whirm please help out Niels with that pull request.
If it passes then master student @rjruigrok can start testing matters full-time.

NielsZeilemaker · 2015-02-02T14:05:12Z

That's the old version of libnacl, update your submodule recursively.

Conflicts: Tribler/dispersy

DH ntor handshake + GCM for encrypting packets. Hidden services are completely broken... Maybe even need of a redesign as the new crypto cannot provide a hybrid_encryption using only the key of the destination.

hidden services. Some small changes to crypto

tribler-ci · 2015-02-02T19:57:04Z

Build triggered.

tribler-ci · 2015-02-02T19:57:05Z

Build started.

Add python-cryptography

tribler-ci · 2015-02-02T20:02:06Z

Merged build triggered.

tribler-ci · 2015-02-02T20:02:06Z

Merged build started.

NielsZeilemaker · 2015-02-02T20:02:49Z

@whirm it's ready

@rjruigrok have a look at the community, I would pay special attention to the announce stuff. I'm not sure what will happen if a introduction_point of a seeder closes. Will the seeder open a new tunnel?, are you sure this will always happen? Maybe add a periodic task which creates a new introduction_point after X seconds (similar to the dht_lookup which is scheduled every 60s).

tribler-ci · 2015-02-02T20:20:49Z

Build finished.

tribler-ci · 2015-02-02T20:20:49Z

Test PASSed.
Refer to this link for build results: http://jenkins.tribler.org/job/GH_Tribler_pull-request-tester_devel/3340/

tribler-ci · 2015-02-02T20:26:36Z

Merged build finished.

tribler-ci · 2015-02-02T20:26:37Z

Test FAILed.
Refer to this link for build results: http://jenkins.tribler.org/job/GH_Tribler_pull-request-tester_devel/3343/

tribler-ci · 2015-02-02T20:42:05Z

Merged build triggered.

tribler-ci · 2015-02-02T20:42:05Z

Merged build started.

tribler-ci · 2015-02-02T21:04:21Z

Merged build finished.

tribler-ci · 2015-02-02T21:04:21Z

Test PASSed.
Refer to this link for build results: http://jenkins.tribler.org/job/GH_Tribler_pull-request-tester_devel/3345/

Refactor tunnel

whirm reviewed Feb 2, 2015
View reviewed changes

NielsZeilemaker added 5 commits February 2, 2015 20:55

Fixup for new crypto changes

379950f

Conflicts: Tribler/dispersy

Refactored crypto in tunnel community, now uses the 25519 curve for the

1b4984e

DH ntor handshake + GCM for encrypting packets. Hidden services are completely broken... Maybe even need of a redesign as the new crypto cannot provide a hybrid_encryption using only the key of the destination.

Refactored HiddenServices, split into two communities. With and without

3eeb2b2

hidden services. Some small changes to crypto

New hidden_community file

753a8e8

Redesign of hidden tunnels, now uses the new crypto.

b037cff

NielsZeilemaker and others added 6 commits February 2, 2015 20:57

No need for py-asn1 and pycrypto

8eef58f

Unused methods

cb0887f

Update README.md

fc14856

Add python-cryptography

update dispersy ref.

30b0170

update dispersy ref.

fc1645e

Fixed remove circuit stuffs

f7ecb06

NielsZeilemaker force-pushed the refactor_tunnel branch from 1337036 to f7ecb06 Compare February 2, 2015 19:58

Fixup, merge bug

c83949e

whirm added a commit that referenced this pull request Feb 3, 2015

Merge pull request #1188 from NielsZeilemaker/refactor_tunnel

006a008

Refactor tunnel

whirm merged commit 006a008 into Tribler:devel Feb 3, 2015

synctext mentioned this pull request Feb 5, 2015

Addressing the various security improvements #1066

Closed

4 tasks

Refactor tunnel #1188

Refactor tunnel #1188

Conversation

NielsZeilemaker commented Feb 1, 2015

tribler-ci commented Feb 1, 2015

tribler-ci commented Feb 1, 2015

tribler-ci commented Feb 1, 2015

tribler-ci commented Feb 1, 2015

NielsZeilemaker commented Feb 1, 2015

tribler-ci commented Feb 1, 2015

tribler-ci commented Feb 1, 2015

tribler-ci commented Feb 1, 2015

tribler-ci commented Feb 1, 2015

tribler-ci commented Feb 1, 2015

tribler-ci commented Feb 1, 2015

tribler-ci commented Feb 1, 2015

tribler-ci commented Feb 1, 2015

whirm Feb 2, 2015

Choose a reason for hiding this comment

LipuFei Feb 2, 2015

Choose a reason for hiding this comment

NielsZeilemaker Feb 2, 2015

Choose a reason for hiding this comment

tribler-ci commented Feb 2, 2015

tribler-ci commented Feb 2, 2015

whirm commented Feb 2, 2015

tribler-ci commented Feb 2, 2015

tribler-ci commented Feb 2, 2015

whirm commented Feb 2, 2015

whirm commented Feb 2, 2015

synctext commented Feb 2, 2015

tribler-ci commented Feb 2, 2015

tribler-ci commented Feb 2, 2015

tribler-ci commented Feb 2, 2015

tribler-ci commented Feb 2, 2015

synctext commented Feb 2, 2015

NielsZeilemaker commented Feb 2, 2015

NielsZeilemaker commented Feb 2, 2015

synctext commented Feb 2, 2015

NielsZeilemaker commented Feb 2, 2015

tribler-ci commented Feb 2, 2015

tribler-ci commented Feb 2, 2015

tribler-ci commented Feb 2, 2015

tribler-ci commented Feb 2, 2015

NielsZeilemaker commented Feb 2, 2015

tribler-ci commented Feb 2, 2015

tribler-ci commented Feb 2, 2015

tribler-ci commented Feb 2, 2015

tribler-ci commented Feb 2, 2015

tribler-ci commented Feb 2, 2015

tribler-ci commented Feb 2, 2015

tribler-ci commented Feb 2, 2015

tribler-ci commented Feb 2, 2015