Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add PeerTube service to NewPipe #79

Closed
ghost opened this issue Apr 9, 2018 · 67 comments
Closed

Add PeerTube service to NewPipe #79

ghost opened this issue Apr 9, 2018 · 67 comments
Labels

Comments

@ghost
Copy link

ghost commented Apr 9, 2018

[bounty 300€]

Hi,
Is it possible to add PeerTube to NewPipe ? ;)

https://github.com/Chocobozzz/PeerTube
Regards,

@theScrabi
Copy link
Member

Yes it is, if you can add almost any service to NewPipe. If you want to give it a shot, take a look at our documentation. However doc is currently on work, so you might not find much yet.

@FlorianSteenbuck
Copy link

Absolute possible. PeerTube, is like DTube, but better, all data come from one api (even the default access key). Their is no preloading. And their is a only one path on their api /videos. The only problem is the optional login feature and the torrent protocol and even this can be skipped because their is also a direct video link (Yeah PeerTube is soo much wow p2p).

@okias
Copy link

okias commented Jun 19, 2018

(supportive post) as a long time user of NewPipe, few weeks ago I found SoundCloud support has been added. As I noticed that Blender channel has been blocked on Youtube and Blender guys started testing PeerTube - so, NewPipe support sounds really great!

@ghost
Copy link

ghost commented Jun 20, 2018

Blender guys started testing PeerTube

For the record:

@ blender_org

Update: we are testing #PeerTube on one of our servers. Go to https://video.blender.org . More info: https://www.blender.org/media-exposure/youtube-blocks-blender-videos-worldwide/ #b3d https://pic.twitter.com/kBRKd6mip3

7:18 AM - 19 Jun 2018
Blender PeerTube

@ghost
Copy link

ghost commented Jun 20, 2018

Also, in latest youtube-dl added some improvements in it's PeerTube extractor with example for https://video.blender.org/*

@dscottboggs
Copy link

Is anyone working on this? I'd like to provide some assistance if any is needed.

@theScrabi
Copy link
Member

@FlorianSteenbuck said he was working on d.tube support, maybe he can give feedback on adding peertube support as well.

@theScrabi
Copy link
Member

however please be aware that I will soon push these changes:
#94

@theScrabi
Copy link
Member

By the way I may need help with #94. The changes on the extractor are more or less done, the only thing that is missing now is that the forntend needs to be made compatible with these.

@FlorianSteenbuck
Copy link

FlorianSteenbuck commented Jul 6, 2018

The best way of implementing this thing would be over the settings that going to be added with the DTube support. Therefore we need a mandatory attribute to a single setting attribute and we need to open up the add ServiceList for add custom sources, because everyone can host a own peertube instance.

Peertube Client

Their is no preloading on the peertube client. So no html parsing is required. Their is like DTube no private api key and no hard coded configs. So no javascript regex search or parsing required for get this working. Everything is working over a public API. Their is no other way then using this API, because the Javscript also using this.

Peertube API Endpoints

{host}/api/v1/config/

here is everything that the web client using. We only need the instance endpoint for the name of the Service and to show the prioritized kiosk. Maybe we could check the Server version for compatibility.

{
  "instance": {
    "name": "FramaTube",
    "shortDescription": "Framasoft instance",
    "defaultClientRoute": "/videos/recently-added",
    "defaultNSFWPolicy": "do_not_list",
    "customizations": {
      "javascript": "",
      "css": ""
    },
   "serverVersion": "1.0.0-beta.9"
   ...
}

Everthing else could be a cool feature, like licensing in the future.

{host}/api/v1/videos/{query-for-all-videos}

Everything for query all videos (kiosk).
Recently Videos - {host}/api/v1/videos/?start=0&count=8&sort=-publishedAt
Trending Videos - {host}/api/v1/videos/?start=0&count=12&sort=-views
Local Videos - {host}/api/v1/videos/?start=0&count=12&sort=-publishedAt&filter=local
Search For Video - {host}/api/v1/videos/search?start=0&count=12&sort=-publishedAt&search=Search+For+Video
find out by yourself why the count is so different and how they calc this this way.
This is the response to requests on this endpoint:

{
  "total": 1349,
  "data": [
    {
      "id": 1468,
      "uuid": "839345ed-26b5-4503-99b1-2cd196356019",
      "name": "03.08 Open source et brevets ",
      "category": {
        "id": 13,
        "label": "Education"
      },
      "licence": {
        "id": 2,
        "label": "Attribution - Share Alike"
      },
      "language": {
        "id": "fr",
        "label": "French"
      },
      "privacy": {
        "id": 1,
        "label": "Public"
      },
      "nsfw": false,
      "description": "Le brevet est un titre de propriété industrielle qui confère à son titulaire un monopole d’exploitation - c’est-à-dire un droit d’interdire toute exploitation par des tiers - temporaire (pendant au maximum 20 ans) - sur le territoire d’un État don...",
      "isLocal": true,
      "duration": 180,
      "views": 85,
      "likes": 1,
      "dislikes": 0,
      "thumbnailPath": "/static/thumbnails/839345ed-26b5-4503-99b1-2cd196356019.jpg",
      "previewPath": "/static/previews/839345ed-26b5-4503-99b1-2cd196356019.jpg",
      "embedPath": "/videos/embed/839345ed-26b5-4503-99b1-2cd196356019",
      "createdAt": "2018-05-24T13:00:32.780Z",
      "updatedAt": "2018-07-04T10:11:39.076Z",
      "publishedAt": "2018-05-24T13:02:20.518Z",
      "account": {
        "id": 3,
        "uuid": "0a635cad-dcdb-443f-b600-6c38ffaffe1f",
        "name": "framasoft",
        "displayName": "framasoft",
        "url": "https://framatube.org/accounts/framasoft",
        "host": "framatube.org",
        "avatar": {
          "path": "/static/avatars/f73876f5-1d45-4f8a-942a-d3d5d5ac5dc1.png",
          "createdAt": "2018-05-24T12:48:06.202Z",
          "updatedAt": "2018-05-24T12:48:06.202Z"
        }
      },
      "channel": {
        "id": 54,
        "uuid": "e4985792-98ca-49be-a1aa-bceecd1c8051",
        "name": "e4985792-98ca-49be-a1aa-bceecd1c8051",
        "displayName": "Mooc Comprendre l'Open Source",
        "url": "https://framatube.org/video-channels/e4985792-98ca-49be-a1aa-bceecd1c8051",
        "host": "framatube.org",
        "avatar": null
      }
    }
    ...
  ]
}

{host}/api/v1/videos/{uuid}

All video infos contains mutiple sources in torrent, magnet link and direct link format. And all other data that is required by NewPipe (no donation links, so searching in description is required). Some data are outsourced to other urls.

to be continued

Their is a lot of more stuff I could explain to the endpoints feel free to ask.

The Mystery of Bencode

Bencode is a combination of the delimiter : and the surrounding that starts with d,l or i and ends with e. If data is surrounded you should not use the : delimiter. This going to end up in that only string and bytes using the delimiter. Like in json there is no way to got a int as key in a dict. Everything is written char on char so its quit heavy to read complex or long bencode by yourself.

i is used for int

i1e

= 1

l is used for list

li1ei2ei3ee

= [1, 2, 3]

string and bytes are handled this way:

7:NewPipe

= "NewPipe"
the 7 is the length of the string. 1 char = 1 byte, so it really do not matter if it is a string or bytearray.
(But do not use encoding on this if you do not know that it is a string, this could end up in a mess)

d is used for dict

d7:NewPipeli1ei2ei3e4:tadae4:thisd2:is7:awesomeee

= {'this': {'is': 'awesome'}, 'NewPipe': [1, 2, 3, 'tada']}

.torrent Files

I can not read .torrent files but this python codes helps a lot it take the torrent file and convert it into json. The encoding is the bencode. yield ouput is handled like a loop iterate so decode_item is going to be call the next mutiple times if needed (for dict and list). Their are mutiple implementation in mutiple languages out their for bencode without and with regex.

import re

def tokenize(text, match=re.compile("([idel])|(\d+):|(-?\d+)").match):
    i = 0
    while i < len(text):
        m = match(text, i)
        s = m.group(m.lastindex)
        i = m.end()
        if m.lastindex == 2:
            yield "s"
            yield text[i:i+int(s)]
            i = i + int(s)
        else:
            yield s

def decode_item(next, token):
    if token == "i":
        # integer: "i" value "e"
        data = int(next())
        if next() != "e":
            raise ValueError
    elif token == "s":
        # string: "s" value (virtual tokens)
        data = next()
    elif token == "l" or token == "d":
        # container: "l" (or "d") values "e"
        data = []
        tok = next()
        while tok != "e":
            data.append(decode_item(next, tok))
            tok = next()
        if token == "d":
            data = dict(zip(data[0::2], data[1::2]))
    else:
        raise ValueError
    return data

def decode(text):
    try:
        src = tokenize(text)
        data = decode_item(src.next, src.next())
        for token in src: # look for more tokens
            raise SyntaxError("trailing junk")
    except (AttributeError, ValueError, StopIteration):
        raise SyntaxError("syntax error")
    return data


data = open("torrent.torrent", "rb").read()

torrent = decode(data)

print torrent

taken from http://effbot.org/zone/bencode.htm

But we got another problem with this code. The enpoint pieces seems to have a own thing to parse:

{
  "info": {
    "length": 7250102,
    "piece length": 16384,
    "name": "The Daily Dweebs 720p.mp4",
    "pieces": "\xb6\xef#\xd7\xee\xf83\x0f\x1a\xb3\xbdY\xf5\x88\x16\x96\xc0\xdfc\xf4\xe9]\xfe\x9a\xf6\x937\xca\xf5\xa4\x80\xde\x90\xfc\xc3\xe7\x8d\xe7\x96I\xbbuG \xeb\xa1\xfdf\x1a\x84\x8f\xf5F0F\x01\xb5\x00DB\xd3No\xf3\xd0\x95U\xecdZ\x1fx\x91ug\xf9\x8e\t\xf6\xfa\xe5\xfb\x03\xa2\xe8\xf7\xb5\x9d\x00\x98^\nH\xfc\x8eD\xb32<\x96\x89D\xc7\x0f\xe0a$V\xa2\x87\xb9\x0e=f\xe5\xd6\\\x15h;-\x10\x8aQC\xcf\xec\x03\xe4\xa7s\x98n\xe1\xea\xadI0]\xfeL$;\xf0 \xddE>\xcd\x12\xb0\xbd\xf5Yg\x1c\xc8&E\x86\xe4\x89\x9d\x82\xef\x01\xfb|\xac\x9c$\x8b\xd0s\\+/x!\xa1S\x82V.\xb3\x89Q\xf5\'\xfb\xa4\xa5\xe8\tY3\x1ax\xfe\xac\xe3\xb3b)\xef\xc0\x8b\xe6\x00\x91\xc9\x16\xf7\x1fc\x88+\xbf\xcb\xf4\x88\xc0jIv\x93U\x89\xa0\x80\xd5]\xf8GOM\x8c\x18>\xe4\xf2\x81\xa5\'\x8a\xaf\xa8\x14\xe3P\xe0\x85-J\x0e7\xf2\xd2\xda\x0bj\xda\xd2\xed:\xf4\xb9\x801X\x0cvGr\x11\xc0\x18\xbagS\xad\xcb\xbf&\xf2\x8a\xb8\xf9\xb3\xb8\xe3*7jg\x83\xcc\xa0>>\xe9\xef\x92\x06\x1f\x00?g\xab\xf4{\xb6\x83\x88\t\x0c\x84\x85\xc3][O\xee\xc60:\xbd\xbd\xe3i#\xd8\xb1s\x89I\x98\xc6<\x82\xce\t\xcd=qT[}\xc02d\xba\xb8\xd3W\x89s<\x12p\xda\xfepJo\x85\xcfD\x86v\x13\x0e\x1e\xa85&b\xd0\xde\xe5\xcc_G\xba\xd9\xb0\xf5j\x1a\xa6\x86\xdb\x83\x94\xadxj\xb1\xa7\xdc\x98\xf7\xb8\xd2\xc1I\xfd\x99\xde\xb7\x81r\x12)\xa0\x00d\x83\x0f\x14\x15\x0b?^\xc0\x97\xf9(\x03Sx\tc\xe7R|\xccy\xe8\xba/\x03\xe2\xfe\xf6(\x0by\xc9&\xf7\x9e\x86\xb0`-\x8do\xb9\xd1\x90\xa9E\xb2\n\x1e\x88\xd1D2\x0f\xd2\xec"\x81\xf2]TqW\xfd\x88\x94\x13\x98\xf2\xd1\xdc\xb8,Y\x0c\xea}y\xe4A"\xd0\xda`\x06s\xb6\x80\xd3\xdb\x89D\xa5\x9c\x05\x87\xca\x82L+\xa9\x9bA\xbe\x92\x89\x07\xbdA]\x85\x1b\x99F\x93\xa2\xbdA\xfd\xaf\x10\xc4\xa8\xa7\xa2_\xfc7\xa7\xa5%m\xc2\xb7#u\x8f\xb58s\x81\xaaT\xd0+\xb8[~\xa9\xb9`\xc2\x8f\xfd\xe9\xbbp\x0cs\xe9\xb52\x9a\xf1\xa0\xa7*N\x13\x9d\xbb_\xa30\xad\xd9\n\xaa\xf8{\xbe\x82qzs\x92\x08r\xd2\xdcf\x8f\xab50\xcb\x88l\xc0\xf8u\xcf4\xfe\x91\xb3\xae\n|\xdfm[a\xfb\x8f\\E\x1d\x83\xee\xabS\x10\xf2\x07\xe9\x19\xfa\x10\xf8m\xc7\x8fqx}V\xe6\xd1\x834\xd6\xef\xd8Ih\x7fN\n\xff\x93m\xb3f`Fa\x98hP\xb0T\xfcST\xff\xa5\xcf\xe8g\xc80\x7f\xb6V\x0c\xe2\xd2\x1e\xc54\r.\x9e\xb6\xc80s\xde4d\xca\x91\xb2hP\xed\xf3\x03\xe19\xd2\x1c\xd0\x1d\xd3\x0f\xe2\xc17\x91\xc3V\xfa\xe6\x8e\xb6\xd6\xfda*ZoGv\xf0i\xfa\xc6x\'\x0b\x0c\x0f\'\xc4vo\x9f\xbd\xdf\x12\xd83MbG\xa8\x04\x87h\xef\x0c\xa4\xa4\xb4G\xdcMjV1\x92\xdd\xca\x89\xa8\x16f|\x19\xfc"\xe5\x91MH\x82\xadb~\x17\xcdA\x19\xf9E\xe29\xad\xfa1\xe4\xa4\xec\xb4\xe3\xd1\xca;\xde\x03K\xa5\x90\xa0\xe8\x03{\x9f\xe4C\xeaU\x8c\xbd\x89`\x8a\xb8Q!\xe7/\xc6\x10\x13\xf6J\xa0\xd2;H\n\x8d0\xbeN\xe52\x1e\xcb\xbb\xd8\xb9(\x98x_\x0f\x99\xf6\x8d\xa4Do\x7f\xb6,\x0e\xd9\xc3\xdc\xe1\x8a\xe1\xf0v\x08\xbb\xb3O\xe8\xef\x99\x8euo\xaa\xd3m\xe3\xb4\xb8\x11\x82\xee\xce}\x14#$\xefM\xe4%\x83\\\xd5\xd01\xc1\xce{\xd0\x80Tu`\xdeVL+\x98g5JtP\x80\x19h{\x12\xa3\xc1L\xabDN\xe1\x04\xbd&\x8b\x9d\xcf_\x00w\xcc\xcbu\x18\xf6\xfd\xd9\xff\n\xbcoS\xf05[\xc8\xec\x00\xf4\x7f\xd1\t\xdbv\ra\xf2\x00\x18:Y\x85\x7f\x97\xc6\xce\xd5S\\\xe2j0\xb3t\x12!\xa9n\xc5\xa6_\x057`&\xc8\xf2w\xa3\x99\'\x83\xd1\x86\xc6\x1b\xf1^S\\\xe8\xad\xb5\xb8\xcb\x05\\\xea^\xf6\xce\'q\xa9\xa6\xf8\x0e\xaf\x15\xabu\x0e\x85\xe7\xc9S\xd5\xd5~\x7f\xc8\x1b\xad\xd6\x1d\x9c\xda\xe7 \xdfkzy\x17\xddXV\x8evKd\xee\xa3\x14\rQ:9m\xfa\x99\x05\xfe\xf0\x87\x17\xaf#)\x1e\x1a\x85\xca\x9bTGC\xe5\xb1\xa9q\xe3\xe2I]\xfb\x0cy\xd7\xd8\x94\xdb{\xaf\xb9\xe6\xb6}\xbc7\x98Z\xae\xe5J\xda&_7"\xe65;9\x1c\x1a\x8d\xd3\xe4;\xc8\x8e\xc11P(\xd8HJ\xa0\xe7\xe9\xbc\xb7\x02\x96\xa6\x91\xdb\n\xd3i\xc1\xca\xf3\xc7\xef\x85\xdd\xd5*\x9bw\xfa\xce\xc8F\x7f\xae\xe1]y\xf2\r?=\xc2\x83[\x95\xff\x00n\xa5\xb5^\x0b\x1ai\xf1\x82)\x18\x10\xaf\xe2`(R\xb7\xb0\xefT\xe4]t\x8ea\xc58\xd3\xa3\xb4\xae\xe4\xe4\xe6\xc9@\xf7\xe4\x18u\xe2\x11qq,7\xa4g\xf4\x0c\xb8\xad\x11\xab\x8d\xce\xf1\xa0\x02a+\x9d\xb2\xb61\x80\x81/\x7f<oyw\x12ANE\xec\x11\x1c\x8a\xffn\xca\x1c\xc1J\x17q\x8fb\xc1u\xdc\xc5A[\xe9\xaaU\xaf\xdau\x1d\xb29\x98[\xad\xcb\x0e\x1f\xf3\x9cj\xb2\x01\xe7o\x82\xec\x00t\x9d_\xcas\x85\xb3\tT\xd7\xe0\xa0\x80\x11\x96\x10\x03\x8c!\x0f\x8f\x0f\xf4!@@u\xb9vj~\xc2 \xf6\xd2\xdd~\x14\x0eh\xf5a/\xe5>\x9b7\x8c\x9b!\xef\xa1\x8c\xbbpK"\xe6a\r\xc1C\x13\xa6\xac\x8557xS\x13\xea+\x8f\xb3\xf0\xd3.\xbf2\xc2\xac\xe4\xc0\x9f\x9bA\xd4\x15T{\x03Ei\xea\x91a\xaeRU2\x06\x01\x9fa\xf7a\x05\xf1\xe7\xc6GY3\x8e\xa7\xe5\xc4\x9daD\xa2\x9ac\x12\xf3\x83+\xc2\xe31M\x84\xef\x8fJ\xe0\xd1\xc7\x03:\xaei\x9b\xabI\x16;\xc6\x9b\xe0\xed\x1c\xd3@u\xb7\xd5\xc8:\xda\x1d\xc12\xb6\xabxp\xbf\x91\xfa\x9eX\x98\xaa\xe8\x0b\x85>"\xdc\xc8`\xf8"NW\x84\x07##\xf7\xbf|l\xdd\x9d\xd1F\\\xd18\x92\xa0\xeb\x1fJV\x8aE2\xaf\xbb\xcfz@f\xf6\xe1\xc6H\x8ah\xeb\xb0q\x9c\xa8WR:\xc5\xe3\x12\x1a,k\xbe\xdfqe\xc4\x1ad\x16\xbb\x12G\xe8"0!X\xcfa\xf0\xf9\x06\xb9L\xb2)$\xaf\x02\x06\x83\x81\x1f1\xf88\xd4e\x9c\xe7-CL\x91&\x05\xedM\xc6\xdc\xb4V!\xaf\xd6\xcc\xaf\xbc\xa1f\xa0\xf8\xa4E\xb7\xda\xca\xf3Z@C}\xf7\x9cw\x99\x956y\x8d\xf5\xcex\xc6(\x91T\x1e\x1f\xc1z\xc0\r)[\x9188\x0e\xb1\xf9\xf1I\xdd\xe4\x8bSD\x05\x8cUw\xc7\x1c1V4\xa1@\x05q\xe0\xf8\xd0BkG\x9c\x91\x1a\xb1\x19u\xfd\xb7\x8b\xc9\xe8\x08C\xae~\x8d\x16\x83`\x95]F\x80\x92yo!\x86p\xdf\x1d\x1b?F\xfa\xb0\xec9\xa1#\x83\x8e\xe06lE\x9dR\xee\xbe\xff\x12\x90$A\x1d\xd4\x0cW\xea\x826\xaf\xf4\x1cG\xf7%@\xf7\xe1\x14n@?\x9b)\x19\xbb\xd6\xcf\xbc8"\xfc|\x00\x15\xcdL\xaf\n\x01\x13Zh\xf5\x8c\xb42\x07d\xc8\xd6\x97\xa8v\x91#\tz\xd5\x88\x91g\xa5 N9E\x89\x81\xf7\x9c\xa6`\x08\xc8\x96\xb1\x9cg\x8b;\x193!sj\x88\x82\xfav\xa4\x1bE\x1f\xd7-\xce\xb9\xb4rl\x94\xc5$ILl\xb2\x9b^\x03\xaf\xb2\xe9\xea\xd84\xc5\x86\x900\x1e|(\xd2\xf2\xe4OAp\x12V\x141\\\x04=`N\x99\xe2\x96B\xbc`N\xb1!\xcer\t\xc1\xb5C\x07X\x95\xad@\xc6\x83\x81i\x01\xab\xeb\x18\x96Uwh\xadZ-\x11\x96m\xfb6\xb4\xaa\x13\xe2\x97L\xd4\xb0d\xdb\x1a\xde\x18z\x9bhy&q\x81}hSX\x18,\x8bA\x83\xb5\xe8Q,R\xea\xd6{\x12\xb7\x91\x1f\x8d\xe0\xc1\xb1`\xd3Q\xe1\x05V\xc1\x03\x1e\x95\xccR3\xa4[\xa4\x14\xb0\xda\xa2c\xa9\x9c\xaeE=\x1a\xd7T+\x87\xd4"\x18d\xd5\xfaaw\xe8>\xaaC\xd9\xbc\xf2\x18\xc3\xce\x90S\xcf\'Q\x8cL\x7f\x8b\x8f\xff\xbeN\x94\x80\x19\xc3\xef\xf2t\xf3\xeb(\xb7x\n3#\x11K\xa3\x0e\xc3\xcd\xbdHG \x99c\x8b\x8d\xa8\xb0\xf6\xbc\x92\x9d#\xc94\xc2*\x7f]\xe5\xce\x8d;\xc1\xd5\xf9\x7fN\xfa\x8b\xefV\x82\xbbS<(\x0c@\xbc"\xa5\xa6-\x844<\x9c\xe2x\xbd\x82N\xbey\x95Y\x04\xff\xc70\xd4\'\x93%\xd6\x1b\x08\xde\xa9\xf7\xa0m0\xb6\x84\xd7\xee>\x01B"\x1c\x9dqN\xf2\xebE\xed\xc7\xdc8\xdd\xec\x9a\x97S=\xa8\xd8YB\x91\xec\xd3\x11\xda\xa6p\x86\xee\rq\xa2i\x17!\x8c\xb4\x7f\xe1S\xf7\xc4q_\xf8\x8eQ\x8b\xa8\xb3a\xc2\x9f\x8a\xf3u\xac\x8a]%\xe8\x9e*Y\xd1\xa6\xbe1$\xa9\xc8\x1c\xc4?\xda\xc2\xef\xbb\xc2\x96\xf7Kw,2\x87\xb0o\x98\xb9f\xf0~<\xa3\x8fe\xca\xab\xb7$s\xa7\xdd\x82*G\xbc\x04\xc0\xecg\xa1\xf6\xac\xbe\x81E1\xa9\xb7o\xfa\xdcx\x0c:\x9a"\xb4\xcanc\xb9\x02\xcb\xca\n*\x81A\x13/\xd7EFXv~@\xfb\x94\x89I\x07\xfcx&\x07w\xb0i\xe7o\x858\xc3z\xb3e\xee\xcc\xd7@\x92\x96\x16\x90\xf7\xc7\xf7\xd7\xdbO>\xd5\xd0`\xf4Nk\xc6\xe6\xa4E\xcf\xeaQ\xf4\xd4\x11\xae\x929o7\xf9*u\x8dKU\x0c\x88n\x7f\xe2K\x86\xb6\xb9B\xbex(\x1c\x06*\x08\n\xe5\xb3(\x08\x81&\xa8\x14\xc9Q{V\x0cV\xee\xd8\x07\x90\x91\xb6\xe3\xa5t\xed7\x96\t^\x8c,\x8b\xd0\xc7or\xbc\x0cJ\xd8\xdf\xc5c\xa5\xd0=\x88\xf2J\xf4\xc2\x868\xfe)\xeb`\xc2\xa2\x95\xb5\xd9\xde\xd0\x1f\x13\xccyq\xf5\xaaV={2\xef\x1aj\x90\xa6\x0e\xfc\xe6\xf7\x80F\xbc\xf0H\x12\xed3\xe9\xa3H\x01N\xa8q\xe4S\xb5y\x01\x9b\x8dRK\x13-\xa7\xc3v \xe9\xd5"\xea>\xbe\x1b\xcf\x90\x1e\xff\xac\x05\xa8\x05X$r\xad%:\x00>\xc4\xd0b%\x97g\x9cb=>\x02\xb7t\x01\x8c\xf1\'4\xbf%i\xbc6\x83u\\RNe\xbf\x80[\x97\xad%\xf6\x15\xbf\xf0\x81\xba\xe9\xef\t,\xd4^\xf5P34*\xc9\x17\xbe\x00y\x05\x14\x7f\xab\xf9\xc9\xa3c\xf7\x06\xe6\xe5/\xb5RF\x99\xde\xb8O\xd0B\x07\xd7\xfdet\x80\x99\x93\x1b\xc4\xd0\xd1\xd3\xfe\xaa*~\xa3\x18Z\x1b\x06\x8f\xc3i\xe5\xef\x8b\x00i\xd0\xab\xd6L\xa0N\xbcl\xa7\xbb\x04s(\xa9l\xdc\xdb\xd7\x91y=\x92\x14\xe7.\xe6\x15%\x80~\xb9\xd5&\xf5$8\x17\xaeFX\x80K\xd9\x18&\x05\xe5\xfevD\xe3?,\xcb\tC\x12)L\xa7@\x8b\xa9 *\xfa\xce\xc1*#\xd6-\'iN\x8f,\xb7\xa4\x16\xfa\xeb\'on\x01\xd3f\xd4"\x0f\xdc\x1a\x9e\x80\xc8\x8a~h\xc4N[e\xb7\xb0\xd8\xa5\xe5\xbe\xdd\xa0\xda\xb2#\xfe\x8a\xb1-\x14\xf3\xbc\x13N\xd0K~\xcc\xda\x06\x90G\'\xd2\xb3\xc5.\x14\xda$\x01|~\xce\xdc!\xa7\xb3\x87\xed\xbf\x9c\xcc\x04\xbe\xbf\xb0\xe6gk\xf3F\xe8\x14\x81G\xf1\xdeD\xe6V-\x002\n\x97\xfc\xfb:\x9e:\xcc\xb7\x19\x1bV\x8fp\x9b~\xab\x07\x0b\x04\x03\x18\xc2\xd2$f[\x96 \xaa\xe0T\xe6\x95\xd1\xe6\xdf\xcd\xc8!\xa7\xe3\x12y\xdf\x9c\x8b\xad\x86\x9b\xfd\xe8\x9f\x18\x03Z?\x1eU\xe5\x8a#n:!t\x16Q\xa0B\xc8~\xe6\x91\xa0\xa8\\Jq\x80*B\xed1\xa2\x82j\x90a1\x0fX1O\x87d\xa4\x1e-\x8eX\xd6\xecG\x96\x9e\re\xe6\x9fBZ\x18Qk\xdb%X\xb1`\xf561GDsMG\'O\xca\x19\xde\xdd\tZx\xdb\x07e\xab\xd7\xb1\xde\xe8\xcd\x08\xcd!\x0bS\x1f\xa0\xe2?yg\xdcT\xda\x012\xa2f\xd1\xd8\xe0\xdc\x0f\x9d\xce\xe7\xb4\xe0\x08\xe0\xd8\xe8\xc6\xe7E|\x93k\xbaO\x82X\xa8\x80\xf3\x86\x0e\xfd)\xab\xc5\x10\xa3\xe1\x0b\xe7\xd4\r\xb94\xb7\x01\x16\x95dcS\x81z`e\x89y\xeek\x8e>\xf2I\x9b9\x80\xbf>\xa1\x93\xf1\xdb\x91V-@]\xf8]\x0ck\xe4[\xc1-\x1b\n\xd2|k\x9b\x0e`^\xa5\xa7(\x13*C\xfdq\x89ma/\rB\xb5QA\xfd1\x08\xd7\xfc\x1e\xed\x19\xf4~C\xb1W\x1c\xd1\x16^\xf1\xcc"\xb9\xf3\xf4\xcb\x9b\x93v\xb0\x94\xbd\x7f\x9e\x86\x00\x85\xa1p\xaa\x04f\x06o\xf9\xfa\x1a+J\xaetEp(pY\x17\xd5\x11\xe8\xb1\xae\xccC\x12\xaf\xc3P\xd6\x87\xb4!@/\xc0\x8a\x8aM\xef\x10$\x8d4%\xea&\x80c\xaa\xeb\xc38\xee\xb3\xeaI\x90_\x88t\x0bQ=\x8c[\x94\xae\xd7Q`W\xc6$\xf5@\x9d\xb2K\xfe%Q\x91k!W\xe8\xdd\xc0\xd9f\xef_\x14\xb2\xdd\xc1((]\x87m\xbf\xb7\xd3\xb5y\xf7\xc4gm\xf8\x10.\xe1\t&[\xda\xb07\xbb\xf5p\'\x19^\xb4\x99\xe7T\xb9\xf0>j\x05f\xf2\xa1vg\xfc\x8c\xfaD\xca\xf4\x7fq\'\x86\x8d\x13\xa6\xfa\x13-\x0c@\xc1_\x07\x8c\xb3l\xb6Td\xffU\x80\x9dZd\xc9[\xfa\rl\xd1F\xdb\x87k\xb4\xb1@n\xbe\xe32R\xf8#\x1e\x98\x9d\xa3\x03\xa6\xf2\xa6W\x08aH \'\xb1\xc0\x88\xae-l\xf0\xd5\xf2\xea\xb6\xb7n\xa1*\xb4\xd4\xc5\x8cqS\xbeju[p\x19\xd9q\x9f\xf8|\xbd\xe4\x13f\x8f\xee\xe0^\xaf\x9diR\xf7\t\x1c\xdd\x93\\6\t\x0e\xd5\x90YK\'\xfaY\x1aV\x9d\x14\xaeTJ\x16\xa1]\xba\x89Y\xfd\x8d|tFf\xac\xbf\x9fj\xf6\xc7\xed\x86\xadHGn-1\xe3u\xfc!b\t<8\x9f4\xc4\xc7\xed\xe7\xb1\x81\xee\x11\xb1]\xc0`\xf2\x11\xc3@\xd7%d\xd8=)B\xb6\xedS\x1a!\x05\x97\xefs\xcd`\xddF\xc5\xa9\x8f#\xe8J!A\xb8\xb4,"<(\xec\xd3\x1e<\xbc\xfaQ\x1d\x9a+\x81[R\xee\xd2G\xd5\xcf8\xb2\x8c\x92\x83\x92\x002\xb6\xedh\xfb?\xe9\xace\xd0uA%.\xab+G\xedB\xe3~\xcd\xd4\x8c3\x96\x8f:py\x15\xe3\xe2[\xea\xc9\x18\xbf\x88^\x19]`4\x91o\x02\xef\xda\xa0x\nVy\x95\x0f(\x9d\x1aA\x9eo\x87^3\xb5L\x91b\xc7\xe8K\x95D\x19\xc1\xda\xd8X\x07\x02\x93\'\xf5.\x01?&\xd7R`\xb8\x10\xa2\xbb\x12)\xb3\x18\xa4FI\xf2\xf0\x8b\x07pj\xf4\\<G\x02\xb4\x1c  6l z:\x9a\x94@B\xb5\x06\xa3qY\x96S+S\x93\x92\xfb\xddC[o\x15\x8e\xc2\xbd\xf57ie=L\xf5\x83\xea0\xd1\x9d?\x9e\xcb\xc7\xce3\xe4^\xdd\xe5SlS\xdd]F\x1a4\x83C?\x1b\x1b\xb8\x0eD\xbb \xdess\xb5S\xfeu\xbe\x15\x8dom\x02e\xe6H\xaaG\xe9S\xd9\x9a_\xdd\x97\xca\xa3\xdff\xb8\xb6\xf5\xf4\xc0\xc9d\x9d\xbc\xb2\xdbH\xfb\x81\x82\xa2\x10\xff%\xcb\xffFz\xcf\xdc_\xf0\x89\xcb2K\xa768;\xf7\r\xa8\xfba\xb0#}\xb6r\xe5\x18p\xe5\xae\xd6J\xee\xcd\xf9\x98\x8e\x9a\x07?r\xab\xac\x1cm\x1b\xea\xb7s\xe3v\x8e\x04\x83\xcca8v\xda\xce\xa4\x94t0\\c\x9eo\xb4\xfc \xadr\xfe\x94F\x92[\x8d\xa4\x06\xd5\x9b>\x85Hp\x89U\xec\x80\x1dcsO\x1dX\x81\xf5\xcdQ\xdc\xfd$\xe2\x9b\xdb\xbc\xad\xcb\xac?\x18D\xfaDDf5h\x99\x90.d\x08\x91\x14P7\x00\xbe\xc4\xacu\xde\x06%\x9e9eD\xaf\x1c\x03\xcc\x0c\x8c8Q\xe2^\x9f\xb6\x9c\xed\x90\x8d\xc9v\xa7\x97qz\xdcriw\x01\x01\x83\x14\xe9\x81Z\xa8\xa3\xb1\x02\xabyx\x1d\xf6\x8e\x10\x94\xf4\xe6\x8fbq\xf4v\xbc\xf90n5\xff@\x0f\xa7L&\xbc\x1f\\\xbbC\xd5\x04NY\xbfz\xc6\x12\x08\x04D\xbd\xa9\x83\x8f\xdf\x1d\xae\xa2\xcf{\xfd5.\t1\x94)t\xc5\xad\x9e3+ \x056(\xcd\x81|\xe7i$V\xba7\xfa\x9f\x1f\xf5\x08\xbf\xcb\xf7g \xa1\xf5_S\x1e\x96H\xab\xe8\x9a;C\x8e\x1dy\xe8o\xf1 \xc9\xbd&q\xb82\x1eJ\xef\xf2j Z=\xe0\xdc\xa1*T\xb3\xa2z\xaf\\Y6l\xc2b\x19\xa4\x90^=\x1cO<|-\x08\xc8\x8f\x91%\xedd \xfa\xf9\x95\xe9q\x05\xfd\xd9\x04;\xdd\xf2x\x00\xa0\x99\xd2r\x1f"\xb9\x8e\xe5\x84\xf3\x0f\xe6L!Jl\x1ax\xb3t\r\xba^U\xa5H\xd1\x80:c\xd8{\xb6\xbbN\xd3z\x85\x06\x07\tC\x16cI\xb0\xfbPK\r\xc0F\t:\x89\\\xb6\x07dg\xe9\xff\xb08\xc2A\x86C\xb8B\\\xad$\x13\xd3\x9dM\x95w8\x10i\xb6\x93\x8am\xe2A\xcb\x9a\xa2\x13\x1d\x96\xfa\x86\x9ac\xe1\\\xda-\xdeG\xdf\x80\x0b\x9aP\xc9V\x11K\x97\x06X\xf1\x04\xdb\x98<\xc4X;a\x82\x0e\xa9\x84K\xe2LqP\x86\xf8W\x8d\xa8:\xe8lq\xd9.\xa0\x0e\x9f\xd7\xe0X\x9a\x91+\x97\x9c\x0f\x95g\xcc\x00e[\xbb\xa1\xd2\xae:\xc2\xf6\x846A\xa5G\x99\xd8:\xaet\xf0\xf6\x0168\xe1\rqQ{\x1e\x81J\xda\x1d\x8b\xbb\xe4c_\x00\xf5\xcc\x1c\xad^\x18\xc4\x14E;\x8e\xf4J\x9e\xed\xe6\x8a\x97\x02\xa6\x05mu\xb5\x8d\x08\x10h\x81\xb1\x07\x04\x02>\x14BD\x1c\xe4x\xdb{\x00)\x1b\xee\x0b\xfa\xd5\xe0_\x9e\xb7\xd9\xb3\xfa\xc3\xe0\xc7o6\x11\xac\x89\x95_d\xcb9\xeb%\x87hU\xfb\xa2^U\xa0\x9cv\xd6$\xad\xc20Y0\x18\x1b\xb9\xd6*\x18hh\xe4\x08\xfd\x8c\xf6\x99N\xbb\xe7S\x82\x9c\xf1\xec\xf9\xfb\xec\xe0\x01\xa3\x12f\xd9x\xcci7)\xe3\xe1\xb4\x82:\x08\xb4G\x17\x14\x919\xe3\xf4:i\xff\'RbO\xbf\xa2\x9c\xbb\x98\xdc\x89\x8dh\x0c\x06sC>\xa0\x01\x93\xa0\xcfQ\x0c\xa8m\xc9\xcc\xc0\x03\x0e\xa9\xf5\x02#;\x95\xb1\xf7\xa7Mv\xbe;\xda@\xe6*UhJ\x0cM\x05$\xa6z\xe5\x06\x0f\xfe[\r\xce\x8e\xec\xe3\xb3\xd1\xb1\xf5\x19A\xb4\xda\x9c\x12\xcaWBS\xf7\xddlY\xf9\x16\xeaZj\xe3F\x8e\xf3\x83\x9d9\xd6\xaa\xf7\x88HIn\x88\x0c\xacX\xa7+9q\xc1q!\xa8\xa8\xabj\xa2"\xf2nC\x94\xe4Z2\xd2\xb9t@IBY0\xfa"\x04\xf3\xd4\xe3R\x12:\x9c\xc7\xd1\x14\xb0\xb0\xafK\xd3\x16\xbc<p\xd9\xe4\x9c\xb4tlZ\xda\xed\xaf\xd5\xa2]\x8b\xaf7\x8f\xec\xa7\xb9(\xa2\xda$\tJ\xd9\xb5\x9f\xf8\xd6+\xc6\x8a5\xdf_\x9eYj\x1f\xfard^u\x95\x0b\t2\x88]TQI<x\xb8_<#\nX\n\x02\xe0w\x03PC\x1e\x90_Uy\x81\xb1\xe7+\xec(\xfde\xdc8\xb5\xbe\xf89\xbbU\xe7\xe1\x1e\x93\xef\x9a\xe8\x94\x05\x8a\xd8y\x90\x99\xe8\xa5\xf87\x06\xc0\xd3\\d<_\x7f\xcb\x0b\x14\x17\xa0z\xc6kq\n\xba\x92^\xa9\xba8\x9a\xfdi\x0e\x10\xef\x92\xa6\x18tj\x06K\x1c\xa6f\xea Q\xe7qx\x0cp\xd19:.O\xe0\xcb\xea\xe0\xe0\xce1\xab\xac\x9b\xf5\x9e{m\xcaq\x7f\x06m6\xf1\xf27\x86\x1f=\x15\x83\xba*\xbf\x18)\xf9\xe7\x1e\x1ce\x89M}\x87"\xf0R((\xc0\x12\xb8\x8fO\xa8X\xab\x82\x82\xe1(\xfb\xd2\xe1_\xd98\xb5\x91\xf7\x96\xa6\x10\x87\xb8\x9d\xa6\xadP\x1c\xd9f\n\xc1i\x16\xb1\xd1%\x00T\x92t\xc31\xf3\xednz{\xb9=\x1aR\xde\xe8\xba\x82\xbd\x89\xba\xbd\x98akGPNZs\xa4\x03<\xc2Z>\x8b`\x9f\xd0=\xbe=/=\xc1\xe17\xa6<]\xd4P\xd4I\xda\xe0\x9e`\x1ey\x88\xa3\xb6{\x08\xfb$\x93\x8b\xfe\xbc"\xdbV\xfa\xeds\xe7cc\x93R\xe1p\x0b\x8b\xc4\x8d\x18\xd2e7\xcf\xc0W\x86\xe1\x88\xc0(\xf0\x14\x98?\\\xb31\x86\xebD?\x86!\x06\xaa:\x9b\xe8OI\xa2\xa9\x96\x83V\xaebFw.[\x7f\x82\xb1\xedc5\xbd\xc5\xc9\xb6Ww+\xc0\x94&\x92\xdeA\x03_3>X\x03\xa9D!\x06\xc0\xdct*\x97\x9a\x92C\xa5\xef\x9e\x9b\x97\x0e\x1b!\x84u\xf9]\x1b\xcbG\xa0d\x07\xdfBQ\xb5\xf3\x14Y\x04\x96\xa8\x11\xea\xfe\xd6\x13P8u!6\xe5z\x90\xb9\x90\xb0\xd7;\xc9\xce\x9f\xa1k\xb9\x153\x14)\xe5)\x14}\xb7\x11\xfc \xb0:7\xb8e\xd5\xed\xc7RG\x15*Q\x83\x8e\xa1\x00\xb8\x1e-\x0c\x92"\xa4\xc5\x8d\x12\xed\x13.|\xac\x15\x1e\x1eO\xa4n\xef"BA\x07\xea\x9f\x83\xdc\x03\xc7#\xb1\xaaV\xc3\xae\xe2yj\x91N\xf3\xd4{y:\xc3\x91Z\xb2\x91~\xdf\x01\x0e31\xc3\xb8_\xc9\xe9\x9d\x0f\xb6y\x81\xa2S\xd3|o\x8f\x8ew\x91~$\xe4\xead\x92\x03\x18\xd4?,\xac\x97\xf1 \xc7\x93j\xff\x1b\xea0 i\xdf\xb0\x9b]\x83\xb7\xc2\xd3\x12:\xce\n\xc0\xa1j\xeb:\t\xfd\x19\xed,\xfd:\xa9\x1b\x85$U\xfe+\xb9\xc1\x1d1\xfb\xf9\xa9\x1c\x9a\xf6r\xb8\xadz\xa1\x87\x02E\x90N\x85\xb5\x98jt\xa8\xf1\xb8rh\xf179%\xa2\xa3U\xa7_R\x85\xd9.?\xdb[l\x14^mO\xbc=\x81T\x05-\x16\x08\x8a[\xc2\xa1+_!.\xa5\x02\x02\xeb)bgn\xe4\xb1\x06d\x10P\x9b7\x05+\x0c\x9a\xd1q\x0el-\xbc\xfb_\xe1\x88S\x0fXB?\x16\xc1\xdd\xe9\xe6\xd0\x1b\x9b:\xe9\x19\x07\xfb\x1f\xc2TWs\xe6\xf1SU&\x86\xf0\xb0Q]A\x01T\x08\xae\xb1<\'\xe6\x95\x1c\xae\xb9\xc5\x14\x86f\x82\xd9\xba\xf1b\xb3\x1aj:]dV\xe8\x00\x050\xf9R\x14\x9aO\xa6\xdc\x84\xef\\\xff\xd6i\x91\xf7n\nz,H\x8a;p\x15\xf1\x02VZ\xf9\xa3f\xc4\xa2\x87\x80<wv\xfb\xf5\x98X\x183\xd8Q=\xb7\xdcGe\xe8\xcf\xc7\x03\xb8\x8c*ycnf\x94]\x10\x93.\xd6\x8d\xb1q~\xaf\x8bE<\xc3;\x07\xb7l\xb9)\xab\x99|\xa4\xb0\xf2B\xfb\x8e\x13\x9d\x8b\xcfp\xb2;[[\x0f\xe0\x8d\x99\xdb\x9e\thu=\xa3~\xe6!\xd9"\xc5\xcaW\xb9\xb4@d\xe4q\xea\x96\x07\x13\n\xb3\xb2\xe1\x1b\xb1hw\xdc\x13H\xa5\xf1\xfc}D\x86\xb0\xb8\\gQ\xbf\x85\xb8\xed\x00\x8d\x7f&N\xed\x8d\x9b\x8e\xa4\x80"B\xf1G\xdd\x9a/\xd5;\xd3\x7f\x15\xf9\xb1\x16\xb7\xe6U\x10\xb6\x89\x07\xb2\xd0\x16\xa3\xf3\x9c5tq\xd3\x94\xc5\xac\xb8[D\xea\xe3Ugw\x94\xe5\xddX;B\x1c\x85\xd5\xaf\x03\x1e\xe4\x86\x18\x8e!\xec\xcc\x95\x00aXD\xa2\x0f~\xd7\x9c\x9b\x13+\x90\x83<\x9c\x88\x1dE&\xf4\xe4W\xda8.\x91DB\xb4\xf0e\xe3c\x90{2\xd3\xaf\x9dD\xef\x9bhv4\xa3\x01\xe3\xfa\x9cu\x0eF\x9de\x1e\xaa8\xa6\xbb\xd5\xe7\xb5&\xee\xde6\x1dV]\xe9\x02\x87k\x1a\xdf\xcb"\x02\xb4#4\x96=i\xa7\x17\x05\xf6\x19l\xcbn\xb5\xa8\x7f\xb5e\x96\xda\x08\xfb\xeb\x8f\xe6F,W&V\xaa\xec\xcd\xc1\xe3\xf0\xe4\x90\xff\xc1\\\x9f\xecO\x0bCh\x07\xa1Kpo/J\xbecW\xd9u\x90\x84\xa2)\x82\x8f5_(-\xd6\x92r\x06\xf9?*r\xcdO\xe5\x10*;u\x10\xecv\xe9\x1f_\xb4%\xf9;\xd6a\x80\xe6:\x98\xfc\xad\x0f\x07\xbb=\x81\xd8\xa8\xc7\x84\xb6W!"\xd6\xb6\x9f@"\xcb\x93\xcb\xdc\xc3\x0b\xb3\xbdY\x1eye\xa3\x9e\xdb\x91K_\xcb\xcb\xe1\xd9}\xc26\x93\xe4 \x84\x12\xf6\x1f{W\x83\x19]\x95U\xc9\xdf\xbbk\xdb\x92\xec\xc3\x07\xe7\xc6/\xa7\x11\x8d~\xe3@\xdf\x903|\xe0k:\x06\xf9\xb4\x88\x80!1D\x0c\xc4\xe4~6I\xc5Rj1\xb3\xf3\x98U7\xbe\xd6Qeg\t\xa4K\xc0\x93\xccP\x19E\x93\xea\xa7\xf5\xbe\'\x97\x0b, \xf0,\xc4\\\xf1\x1a\xe6\x8b\xbe\xc3\'hO\xa7qj\xbf\xbb`\x12Ac\xc6^\xf4\xb3\xdf\xca\x96kR\xc7"\xc2\xe3\xe9o\xbe\xd2$\xc2\x1d\x14\x17 \xad#$\xb2\x16_\x1f\xb8\x97\x9a\x8f\x93x\xdb\xf8\xf61Hxq\x10B\xe3\x0b\xc7Wm~\xf7\xf3:\x00&\xc0\x0e\x8c\x98J\x1fbPA\xads\x81VY\rx,\x0b(\xf9\xf1.\xb4b\xdc\x98\x8dr:W\x82\xf5\xf8\xea@\t\x85\x14\x8epN\xe1\xde\x9d\'G\x0c2]\xf0\x9dZ)h\x1d{T\xddR\xf9&\x05d\xc6\xac\xcc\xf0C0\xd0\x90\x14\x97\xa1\x15oF}\xdb\xaa\x9c\xb2\xb9\xe40\xfd}\x94\xb3P\x9f\xd8\xc0\x9cz\x1f\x0b\x9f\xbab`\xe3\x1a\x0b7r\'\xbc\x1f\x1c\x83\xef\xa5\x81\x9c\xe4\x95\xffh#\x13L\xac\x1d\x92\xe3\x17Ml7\xea\x7f\xfb\xaf\x00p\x0c\xa2>\xab1\xa0\x98\x90;\xc8[K\xd5i00P\xb6S\xf8Y\x9c\x01G\xe1\xb5\xdd\xe8b\r\x9d\xf0\x81Co\x8bZ\xdb\x7f\x0c\xa1\xf0\xc0?d\xe9\xef\xb3N\x97\x07\xc2F\'\x85Y\xff\xbd>+>\xc8\xfbr\xc2"\xde\x8d:\x9b\x1e\x99"\x11+\x99\xfc\x92\x97\x05s\x03\x89&\xfd,X\xf4\xba\x8a\xc7B\x10\xbd\xbf{\x96\xb1\xf5\x03\xc7z\x07\xe1>\xc7\xf1V\xcd\xab\xf3\xf9\xfe;1\xec<ux\xda\x97\xd4\x89Q\x12\x12\xf1\xc7$l\xb7c<1\xfe\xcf\x8d\xb8\xb7\x8f\t\xa9!\xd2\xb8_R\t\x17=\x87\x86\x86\xf7\xf4\x05\xe4\xa6V\xbbS\xa5a;\x8b\xee \xb5\xaeq\xff\xf1\xbf\xb7\xd4\xbc\xcf\x10\xc9M<\x8c\xbcw\x96O\x14\xe8!\x0f\xa0\xb9C\x9d\x06;p\xb6\xf9\x13U_\x1flfri;\xc3\xc1\x1e\xda\xafa\x0b\x1ewaDX\xba\xdd\xedHp\n\xd3\xfbv\xd4r\xbc\xca"`q\xca\xe3\xb3\xe3U\x86\x94u\xf8\xfbj\xb2n\x14c\x02d\xa9J&W\xe6z\xf3\xefE^P\xdd\x96\x91\xd0\x8c\xce\xb3@\x81\x17\xe6R\xfd\xbb\xa1\x9a3kA\x1a\xf2\x13\xfb\xc4\xd1\x7f>l\xdd@\xcf\xf7\x17\x1a-\xecH\xd2\x8f\x88\xb0\xcc\x8a7\x90(i\x96B\xed\x14F\xb8\x00\xbf4K\x9b\xd1,W\x83\xfd\xef\xd0\xa2\x92\x06\x8b\xbeq\x7f5\xd0\xfa\xdf,dDOQo\x11\xe7\x7f\x07S\xabr\x92,crOn\x98U\xc4\xa7\xf2FQLp9X\x96\xe8Z\xd7&\x84\x04\x14\xec\x8d\xb6\xc9x\xf4?\x7f*\x8d#\\G\t\xdbS\xf3&\xb2D\t4\xd0\xc4\x96.a\x14\xd8\x99ra\x13"\x021#\xf1\x8e\xaf\xbdF\x83\xed\xe8?w\xc1D\xa9\x9d\x8a\x9d\xd2\x00b\xda\x9e\xf7\xe5\xc0\x89\x1aqb_:\xc8v\xc5r1E\x0b\xb5\x7fI\xcf\x10\xc1>\x93\xb0e\x891WR\xbbJK\xeaI\xb6\xcd)+.\x94\xf5\x18\xa5\xfe\x84#\xd6_:\n\x03\x15\x8a\xc0\x85\t\xa2N\x06bJn\x0c2\xed\x9d\xa3\xca\xa4\x90\xbd\xe0\x14\xf7\xb7\xc4\xeat\xb9\x11z\xa6v5\xaarSS\xfbh\x1e\x11S\xc3\x11\xda&\x0f\xd4\xa4*\x14\xcc\x1f\x01\xe4\xde\xf1K\xc1T\x98l_\x05\xf5\x9c\xd1\xeb\xf1\x9f)\xe8\x88\xbc\xa7\x84Z\xcd\xe2\xf6*\xfa\xa2I\xdd\xac\xa6y\xef\xdb\xbac\xb2\x0c\'[\x0fF\xacd\x9a^\x00\x9b\xfb\x99\xf0\xdc\xea\xe8d13^M\xf28S\x10-np\xad4\xe3\x92&\xee\xba\xad\xe4h\xb4\x12\xa6\xd2\xac~F\xee\xd3YV\x16\x1b\x1a\x9e\x0f\x7f\xde\x87$"\xf1\x8b\xd7\xe5\x12\xaf.t^\xe90\xbe\xb3A\x10\xcd\xf9\x06\x7f\x04\xbf3\x8e\xe4\x12\xad\xcc\xe0\xdf\xc6\xd4"\xf7\x80\xb4\xb0!]i&i\xa7(w\xf1\x97 \xc7\xb9W\xac\xf5fI\'\xb3\x8a\x83lx+\xf1\\\x06\x83\xe7)\xf2\x8d\x1dI"\xd1\x01\xce/\xcc\xf9\x8a|\xbf\x90q\xfa\x1c\xfb\x95\x81_\\\x9f\xb9\xcf=\x18~\xc9;\x15\xfd`\x04BL\x14\x0e\xf1\x8cQ\xf9\x87\xbar\xb1T\xf7\xd0\x1a\x9a\x997\xbf08\xd4\x02\x13\xe1\x03\xfd\x0fe\xbcR\xdc6v\xd4\x08:m\xd5\xa3J}j\xee\xf75\x91\xdf\x15\x02\xfc\xd3\n\xcf\xee\x98\xa9&\t:\x11c\xe8\xad\xfd|\x7fzI\x03\xdc\xfaxH\xe4+N\x12e@:#\x14\xa0\x9d\x0b\x00\xdfq\xfb\xdf\xf9\xcb\xa8\xe7\xb9\xf7w\xd4\x1a~P\x7f;\x05%\xa43\x08\x13(HJ4\xd6U\x13\xb1\x89\xd9\xe7\xde\x07\x02\x9b\xffk\xd1\xc5+\xb2\xd1N/\xe4\xcc6y\x7f\xf2\x1aW\xaf\xdbe\xdek\xf1fybr\x8es\xff\x02\xb0\xe91^\xc7\x08C\x89\\\xe8\x15\x89Us\x89\xbf\r\xee\x9a\xa3S\xc1\x02A\xe8sf\xe5\x01=\xc2\xed\x84b\x90\xbd\x93c\xddm\xec8\xeeZ\xaak\xee\x97r\xddfpds\xbcR\xbbv0\t\xd4\xc2n\x10\x87\xaa\xad\xbf\x1a\x01\x1bc\xa1l\xff-\xa5m\x87P\xad\x84;_5\x84\x14-@\xb0T\xf3\xc9\x1a\xa2\xd6\x19i\xcaC\x03O\xda\x0e\x95<\xe5T\xa7\xd6\x1a-\x80\xb0*\x0f-u\xc00\x8aBt,\xf0\xcb"\x03\xf0Z\xe8_q{\x17\xba\xbe\xa5\xc8\'\\)L|\x8b\x07\xe9|\x8b1\xd7\xd0\xe0\xdaLr\xb9(\x0f\xaa(\x8f\xf6\x8d\xd6O\xf0\x88\x8a\xffyq\nJ\xb7\xb8\x12H#\xfe\x11T\rz\xe9Z43\x8c\n\xcc\x96\x06\xacx\x9d3\xea\xc1U\xf9z7\x06\xd4\x94A\x8f#\x8a\xa2\xbf\x0cl3_\xf5\x07\xf6f#\xcai(a\xf6\x9f\xe3:\xac\xe2\xa7Q\xdf\xb2\x84x=\xb8\xc8\xd3\x99@=9\xa6\x1c\xb1W\x8f\xef\x9c]\xe4\xef\xbb\xfd\xb0\xe6\xcc\x1c\xfb\xa2\x9e\x07\x19\xa0=\xf1.\x9b\x1d\x02\xdb\xb2=\x83\xdb}\xbb\xcc\xdfu^\xe0\xb1R\xf8P\x152\xb6sJ\xfc<F\xe2\xfcL\xb0\xe9\x0b\x8dr\x82#<wn\xee0\xe1\xd5\x7f\xa5\x8c"\x9c\x7f\x9e\x10\xe2\xb1_\xf1\xf4$_\x8b\xba(5 ?\xd69R\x01@Q\xee"\xa7\x1ej\xa3\xdbIc"\xedt\xf0\x88@\x1cHA\x16\xe3j\x81\x91\x1bVU~\xa0\xa8\x81N\xb2\x11\x014^-\xf8\xb9\x81\xa8u!\xe2p-\x15\x97\xb3\xb8\xf0(\x8a\xbc{\x9c!\x07\xb9[Lw\x87H.k\xdb\xad\xa8\xfb=\xd4\x08\xec\xa3S\xfe\xef*\xc1\xb2Q\n\xe8\x1e\x8b\xad\x12\x911\x13U\xf1\x95p\xbbF\x1f\x08\xfa\xc2\x8d\xf9{\x9f\xfe\x1f^\x19]\x89\x0e\x12\xf1\xa5\xef\x88\xdb\xab\x90\xa5\xc4g\x83\xf2\xc0u\xa4\x9aSsZ\x97\x15\x17\x86c\xc5\xd9\x88\x00[\xcb[\xf6\x04\xca\x87C\x93\xaehgP\xfa\x9a\xe2\xcbg\x7f\xdf\x1a\x88\x1a\xc9\x1e\xca\x92\x1e\xf5\xf5\xb6\rv \xc6,/\xb4x\xe6`\r\xce\xbbM\x99\xb0\xe9K6w\xdc\xbc\x12\xf8\xd3\xc5\xc2\xa2P\xe3h{G\x817\x01\x0b\xea0\x14\xca\xd12%\xdb\xd3\xdf\xde\x8b\x128\x1e.\n(cn\xcd\x11\x84a~5\x8f2Z\xfb\x957\x1aM\x06\xfd\xcc\xcc\xcd\xce~\xf1\xfa.Z\x16EH;sLo\xa0\xd4f\xb3\xa9n\xa6U\x1b\'\x04\x8c\xf5\xa1\xea\xceX\xb3d\xba;FaR\xfb$\xaf\xeb\xbd\xdb\x95\x96\xd8V\xd5f;\x9d\x8b\x17\xea\xf9^W\x92\x88\x05\x02\\\xdfkP\x13.\xbb\x02.\xee\xf6p4s4\xb7\xf1\xc9\xcb\x04\xbb\xdd\x12\xfe\x9ai\xab\xa2~\xa7\x9f=\xea\x91\xb1\xf16\x87\x05\x84\x87.5J\x92\xa8\xfa\xbb\xd7\x95`\x0f\x15\xcaOys\xc5\x83\xa9\xca\xa2\xf0G3`\x85z\xc62\xb8.\xe3QLM\xeaL\xb5\xf1\xf0q\x9c\xa1\xed\xbd\xe1\xa3 \xf3\x10\xb1\xae\x96\xb1o\xe2\xef \xf4(\xfb\x01\xfc\x02\xdd\xd6\x1e\x91m\xb1\xba\xe1\x89\xd0\x12\x03\xc7\x0e\x02\xab\xb8^\xf6\tI\xf1\xac\xa5\x1c+\x1d\x80\x90hQ\x92]\xc3\xb55t\xfe\xf2\xbd\xc3y\xe0\xee\xabF[\x82\xa9Lh%\xaer\xc5{\xb5BO\xcc\x12\x19\xaf+\'G\xcam\x1e\xe3\xaa\xc9\xe4^\x87}\x93\xc7\xef\xca\xd1\x9d4\xaecXc 8\xbe\xf8\x9bM\x14\xa1\xb4\n\x90\xc4\xd2\xf4\xe1\x1d<\xd9\xa4\xd3\x97\x97\xaa\x91/:\xc7\x99\x82_\x94\x0c\xea\xa3=\'4"#\n\x03\x9fa\xe3\x9eW\xdb\x01Q\x80G\xa9\xf7`\xf0o\xc4"q\x800Y\xb30v\x9c\x0c\xfd\xe2\x1a\rSz\xa1\x90u\x98\x13\xa2\xe2H\x05\x86q\x93\xd6B\xa3\xaf-\xa7\x94\xdf.I\xa6S\xc2ik\xb8N\xae\x18\x81\xe3\xf2i1\x0bz\xeb\xf0\xcb\x84Wd\xb5Z\xc6\x0c\xf0\x03\xceO\xb4\x85\xd4\xdc\xd9\xebA\x00\x84_6$c}qg\xe2_6x\xb0a\x144V\x16\xc1\x01\xadm/\xec\xdfSho\x82\xa3\xf0\x9b:w3\xebQ\xa4Du?\x07q49\xbf\xbdg;\xcdF\xddo\xff\x8f\xfc\xb3\xd7\xae\x0c\x91T\x12\x97 c\xe7\xef"\xff\x1e\x99\xe8~\x1d\xac@%-_r\xba\xc0\xf0\xd2\xda\xfd\x0e6%`U\x81\xfe\xa2\xbf\xaf\xea\x817Tc\xb8\x1f\x8a\x14d9\xa6\xde0\x7f?p\x95NHQ\xb0\xc0\xc3-\xfb\xc8\x0b\x87+\xd9<\xda-(\x18z\xd8\x1d\xa9wi6\x10^\xad\x15P\x8b\xfb\x93\x13\x87[ P\xbe\x0e\x0b\xca\xd6j\xd7n\xfb\xd0\xcdfe&c\xd8\xbf\xac24@\x9b\xebn+\x83\xaf\xc0\xc8\xbf\xa8\xe3s\x1ck\xf2\xf1\xc9\x068X\xf0\xde\xe5\t\xf9\xdc\xcb\x90u]`\x15\x16\xcd\x99\xf5\xfb\x925\xc6\xfb\xe8\xcfX\xaepB\xb2\xe4l\x94y\x15r)\xda?`\xa3c\x9f\x1b\xfb\xab\xf4a\t\n\xe9x<\xe84p\t\xf4m\x18\xd5\x95\xac\xed\x135\xfc\x8d\x94\xbc\xe4v\xe3$\x9b1\xb1\xf9\x8fo\x14\xfdO\xdb\xca\xa9\x90;\x80F\xde\x1e\x88 \xd0K\xcc\x88(\xce\xa8\x91\x04\xfb\x8a\x07\xc1\x13:`\x1c\x13B\'\xaf6D\x80\xf9\x02\xe0\xc7\x1a\xbf7\xab\xac\x8a/\xa6\xe8\x8a\x896\xa3TIh\xad\xea\x0c0\xf3\xd5-\x1dx\xd8\xa5/7Y\tp\x90\x85\r\x0cl\xf4\xf6\xc5>\x9e\x08\x95\x83P\x8c\x89\x84\x82\xc1&\xde\xe3SaG\xb2\xa4W\xb7{\x85\xfcB\x9e26\xef\xa8\x83\xb3\xe2""Y(\xde\xd2W\x9d^D\xa4d\xa3,\xa4\xd3\t\x8a\xe7,\rQd\x9d\x9f\xf7\x13h\xb0o\xcf+\x08\x02\xc2\xb7\x80\xf2\x7f\n\xd1\x00yKcrEMec\xc4\xe7\x13\xf7\xabC\xf2\xeb\x94\xad6&Sk\x82#\xa0\xda\xa8\xd9\xa0\xc1\xf1\x15}(\xd0\xe7\xe9ZQ\x89$9\xf2$\x19\xe8\xdfYC\x96\xf7(\xa5Sq\xd4v{\x05\xbf!\x9c\xf3v\x17*c\xc9\x99\x0e\xdfp\'\x15\x9af\x83\\\xdc\x0b\xc0\xc5*z=\xb9\xb1IPp\xbc\x0b,dGQ:\xd1\xd1\nK\xaa\x86\xfe\xf4\xdc\xf3\x88~\x82U\xd9Y"\xe6^\xcb\xaa\xdbby\xd4\xcfpy\x13\xd9\xe3\xa3|\xaa\x8a{\x85\xb3,zJl\xcf\xe4\x01\xf7j\x8b)\xb7\x8e$\x93\xf6\x99y\x8eL\xbfc\x86\xd3\x19\x08@\xc4\xfdG\t\x1a\x88\'\x89\x9d\xc5\x10\xd4Y\xad\xdd\xaeRo\xe3\nR^\x9f\n<v*\xca\xca,f\xd8\xe3R9F\xfa\x08p\x8d\x12WJl\x8cf\x1dWe\xaav\x1aj\xdc\xcbi"o\x8bh\xb6\x00\x05G\xcf\x05\xda\x127\xa2R\xadA}\x88\x89\x1e\x8b\xafR\xb7\xd0_\xfdE7\x04\xca\xfcPW\xc6&\xd9!\x8c\x80\x93-k\xd0\xa2@z?\xef0\x81\xc7\xec\xb5y(*N\x97i*\x86I*I\xd3\xe7\x00\x8a\xa2pT\xffD7\x91u{k\x9b\xd4\xf6\x8c\xee\xcbf\xee\xcf\x04|\xbe\xb7gm\xd9\xe9\xa0\xed\xf2\x04\xc8H\x00#\x97\x0b\xaf\x06\x05\x96\xf1\xab\xc2\x00*T\xb4t\x00\xfd\xber|\x185#aG\xef\x9f\xffl\x95Ds\x917m\x8bj<o\x16\n\xc8\x10\x94\xe6\'\x8cL=7\x9e\x980 \x91)\x93\xb0|\xee\x82\x81\x00\xc4\x99-:\x06ly\x12\xa23\x90^x~Pi\xd6\n\xd1\xc7\x95\x17\xc4\x9c\xdd\xb8^\x8f\x14\x01@\x17\xc6\xa7\xdd\xb6\xf1`\x84\xb9\x87+\xbd\xc4\xd9m\xd0\r\xbe\xa5\x91*7\xf3\x85`\xc7\xc1\xaa\xa1 \xa3"
  },
  "encoding": "UTF-8",
  "creation date": 1529449174,
  "announce-list": [
    [
      "wss://video.blender.org:443/tracker/socket"
    ],
    [
      "https://video.blender.org/tracker/announce"
    ]
  ],
  "created by": "PeerTube",
  "announce": "wss://video.blender.org:443/tracker/socket",
  "url-list": [
    "https://video.blender.org/static/webseed/b37a5b9f-e6b5-415c-b700-04a5cd6ec205-720.mp4"
  ]
}

After some research I got a solution:
Pieces are simply bytes of mutiple sha1 hashes. sha1 output = 160bits = 20 bytes

So after some changing of the code you can understand now how the torrent file works.

import re
import json
import binascii

def tokenize(text, match=re.compile("([idel])|(\d+):|(-?\d+)").match):
    i = 0
    while i < len(text):
        m = match(text, i)
        s = m.group(m.lastindex)
        i = m.end()
        if m.lastindex == 2:
            yield "s"
            yield text[i:i+int(s)]
            i = i + int(s)
        else:
            yield s

def decode_item(next, token):
    if token == "i":
        # integer: "i" value "e"
        data = int(next())
        if next() != "e":
            raise ValueError
    elif token == "s":
        # string: "s" value (virtual tokens)
        data = next()
    elif token == "l" or token == "d":
        # container: "l" (or "d") values "e"
        data = []
        tok = next()
        while tok != "e":
            data.append(decode_item(next, tok))
            tok = next()
        if token == "d":
            data = dict(zip(data[0::2], data[1::2]))
    else:
        raise ValueError
    return data

def decode(text):
    try:
        src = tokenize(text)
        data = decode_item(src.next, src.next())
        for token in src: # look for more tokens
            raise SyntaxError("trailing junk")
    except (AttributeError, ValueError, StopIteration):
        raise SyntaxError("syntax error")
    return data


data = open("torrent.torrent", "rb").read()

torrent = decode(data)

pieces = torrent['info']['pieces']
piece_len = 20
pieces_count = len(pieces)/piece_len

list_pieces = []
for x in range(pieces_count):
    current = (x*piece_len)
    list_pieces.append(binascii.hexlify(bytearray(pieces[current:current+piece_len])))
    

torrent['info']['pieces'] = list_pieces
print torrent

And here a readable torrent file.

{
  "info": {
    "length": 7250102,
    "piece length": 16384,
    "name": "The Daily Dweebs 720p.mp4",
    "pieces": [
      "b6ef23d7eef8330f1ab3bd59f5881696c0df63f4",
      "0596f1abc2002a54b47400fdbe727c1835236147",
      "ef9fff6c95447391376d8b6a3c6f160ac81094e6",
      ...
      "278c4c3d379e983020912993b07cee828100c499",
      "2d3a066c7912a233905e787e5069d60ad1c79517",
      "c49cddb85e8f14014017c6a7ddb6f16084b9872b",
      "bdc4d96dd00dbea5912a37f38560c7c1aaa120a3"
    ]
  },
  "encoding": "UTF-8",
  "creation date": 1529449174,
  "announce-list": [
    [
      "wss://video.blender.org:443/tracker/socket"
    ],
    [
      "https://video.blender.org/tracker/announce"
    ]
  ],
  "created by": "PeerTube",
  "announce": "wss://video.blender.org:443/tracker/socket",
  "url-list": [
    "https://video.blender.org/static/webseed/b37a5b9f-e6b5-415c-b700-04a5cd6ec205-720.mp4"
  ]
}

Everything else is perfectly explained here:
https://en.wikipedia.org/wiki/Torrent_file

Magnet Links

More About Magnet Links:
https://en.wikipedia.org/wiki/Magnet_URI_scheme

Trackers

A tracker reference to nodes that give you a piece of the file. Also keep in mind the checksums are binary and must be encoded as binary not as representative hex string. For URLs simply using the "%nn" encoding.

Definition of General Requests

Announcement Request

This request is announce your current download status to the tracker.

Scrape Request

This request get you a overview of the peers that are currently connected to the tracker.

HTTP/HTTPS

This trackers are the simplest trackers. For HTTP Connections the The Safe Habor Problem is relevant too.

UDP/TCP

UDP is used for the server here and the client actually using TCP to transfer the data to you. For both of these connection The Safe Habor Problem is relevant.

UDP Requests
Connect Request

This request is for obtaining a id that you can use later in other requests.

Announcement Request
Scrape Request
TCP Requests

Webtorrent

WebRTC

WebRTC is not protocol but a Collection of Interfaces that can be used for Peer2Peer or Peer2Gateway2Peer. The Interfaces are based on NAT Protocols and using in end the RTP or RTCP.

NAT
TURN
STUN
P2P
RTP
RTCP

Websocket

Websocket connection are keeped like a big mystery, because the protocol is ugly and no web dev wants to known what their are using, they just want that the thing work. For the websocket protocol their is currently only one webtorrent javascript interface that every web app seems to using. Their is no protocol description at all. For unencrypted websocket only connection The Safe Habor Problem is relevant.

Websocket only works in combination with WebRTC because you can not open a connection directly to another peer in the browser.

DNS Protocols

The Safe Habor Problem

If their is no encryption so you need to have a safe internet connection or a safe habor vpn (My Recommendation is ipredator.se). This Problem strongly depends on you ISP (Internet Provider) and Internet Connection Providers. Their are ISP and Internet Connection Providers that do evil stuff and thats somehow is one hand of the problem. My experiences with this kind providers are that their is no save habor. So choosing a VPN is only solution that moving this problem to another provider and also add another provider to the stack of providers that are used to create the internet routing/connection, the VPN provider. Also the connection to the VPN depends on the same structure, so on quantity aspect of the providers that are used a VPN is not a safe habor.

Your ICP Struct + Your ISP Struct + Your Side Internet Struct
+ VPN Provider Structure + VPN ICP Struct + VPN ISP Struct + VPN Side Internet Struct
= not a safe habor

You are not using a VPN warning (?)

Before we init a connection to these construct, I think it is important that we warn users that not using a vpn. So we need to detect if the user is using vpn to warn the users that not using a vpn.

We can do this by using the ip address check sites, which is a heavy task because their are mutiple out their. If we want to start with this the simplest site is https://check.ipredator.se/ (ip detection working without js).

The other way is simple but not safe enough I think. Android got the possibility of create a VPN Interface. This is used by VPN Providers but also by Adblocking Apps that not really care about your Internet Connection their care mostly about ads (thats somehow their jobs). As long as I know its currently not possible to detect the app that init the network. We can detect with the ConnectivityManager if their is a VPN Connection with the methods getAllNetworks (for lolipop and upper) and getAllNetworkInfo (api v1-lolipop). For lolipop we should use hasTransport with TRANSPORT_VPN and for all other versions we should use getType with equal check for ConnectivityManager.TYPE_VPN (maybe we need a existing check for this constant)

Other Problems Disclaimer

Of Course their are other problems but these is one step we need to do or at least warn the user about the security issue that appears if the user is not using a vpn.

Proxy Solution

A alternative would be to host tracker proxy, this is somehow the same solution and get also the general problems that I mention above. On the user side this is the simplest method. This needs to be setup with a RSA Encryption Protocol. And normaly their is only a little RSA Encryption Protocol for exchanging a shared secret and then using this shared secret for encrypting with AES. So we need for this a proper UDP and TCP Protocol proxy that first handle a handshake over RSA and share a secret for AES encryption later, if their is any protocol that is a solution for this, let me know. Depended on my current knowledge we could also use RSA, but shouldn't cause their can be multiple attacks to a RSA Protocol.

Solution: "Simply" use a libsodium bindings for this. :-)
(PGP emails are here a different because its a encryption extension for the content of the mail, MIME Headers and Pseudo HTTP Features should be the Email Content)

Need more Content and Explainations !!!1!

yep, I am working on it

thx to https://wiki.theory.org/index.php/BitTorrentSpecification this helps a lot but is somehow is also confusing

@Nutomic
Copy link

Nutomic commented Aug 1, 2018

I am starting a bounty on this issue. I will pay 250€ to whoever implements Peertube integration in Newpipe. Required features to receive the bounty are:

  • Display trending videos on the main screen of Newpipe
  • Search for videos/channels
  • Display videos/channels
  • Download videos (Will work automatically)
  • Make NewPipe automaticity choose an instance from joinpeertube.com

You should also commit to maintaining these features in the future, after the bounty is completed.

@theScrabi will decide when the bounty is completed. If multiple people implement these features, he will decide how the bounty gets split up.

I will pay out the money in the cryptocurrency of your choice (it has to be available on Shapeshift).

Edit: Just in case this wasn't clear, anyone is eligible for the bounty, including Newpipe team members. All that matters is that you (help to) implement the Peertube support.

@theScrabi
Copy link
Member

theScrabi commented Aug 2, 2018

  • Preference to select a Peertube instance (I suggest you select an instance as default to make it easier for the user, eg my instance https://peertube.social)

Maybe we can add a list of instances if the ping is to high, or (if possible to detect) if the response takes to long. @TheAssassin maybe we will be able to host our own instance one day.

@FlorianSteenbuck
Copy link

We can use https://instances.joinpeertube.org and the ajax endpoint they are using:
https://instances.joinpeertube.org/api/v1/instances?start=0&count=500

@Chocobozzz
Copy link

Hi, I'm the developer of peertube.

@FlorianSteenbuck yes you can. Sorry, it's not documented yet.
If you need help or some additional parameters regarding this endpoint or anything else, do not hesitate.

@rugk
Copy link

rugk commented Aug 3, 2018

BTW as someone mentioned in this the first comment: I would not consider NewPipe to have 100% PeerTube support unless it actually (also) uses p2p. Unless that is done, it's just straining the server with the API and not making advantage of the unique thing about PeerTube, so that is not completly supporting PeerTube then.

@theScrabi
Copy link
Member

BTW as someone mentioned in this the first comment: I would not consider NewPipe to have 100% PeerTube support unless it actually (also) uses p2p. Unless that is done, it's just straining the server with the API and not making advantage of the unique thing about PeerTube, so that is not completly supporting PeerTube then.

Thats true, but I guess on a phone you don't want to do that always, especially if you are in a cellular network (in germany). If we implement p2p we should add some switch that only enables this if the network connection is wifi, and good enough.

@rugk
Copy link

rugk commented Aug 3, 2018

Of course you may add a switch, and maybe also a switch, which can enable it for wifi, but disable it for mobile connections, sure. But it should be on by default (at least wifi), as that's the way PeerTube works.

@theScrabi
Copy link
Member

So yes on mobile make it opt out and on wifi make it opt in sounds good already :).

@rugk
Copy link

rugk commented Aug 3, 2018

@theScrabi I'd suggest the reverse, but yeah… 😄

@rugk
Copy link

rugk commented Aug 3, 2018

What you could also try, of course, is that you only download on mobile connections from p2p connections, but not seed/upload.

@theScrabi
Copy link
Member

What you could also try, of course, is that you only download on mobile connections from p2p connections, but not seed/upload.

Yes this whay a specific server does not have to suffer.

However what i ment with switch was not some UI ellement, i would much rather prefere some algorithm that decides that. This way the frontend does not have to be changrd and the user does not have to be bothered.

@rugk
Copy link

rugk commented Aug 3, 2018

What kind of algorithm? I think only the user can decide whether they have a big enough mobile data plan, so they can upload/share data.

@TheAssassin
Copy link
Member

TheAssassin commented Aug 3, 2018

@rugk that won't work well. Who would take into account all that data just for using PeerTube on mobile data? When in a WiFi, it's perfectly fine to upload data, otherwise it should be disabled. Otherwise, NewPipe will be blamed for its high traffic consumption.

@rugk
Copy link

rugk commented Aug 3, 2018

That's exactly what I meant:

  • disable p2p upload on mobile
  • enable p2p upload on wifi
  • but just let the user decide, maybe they also want to use it on mobile, or don't want to use it on wifi

@theScrabi
Copy link
Member

Thinking about all once more what @rugk says is not bad.
I mean p2p via cellular ... well I would not enable it, and how much sense it would make is something else, but making it opt out sounds ok for me.

On the other hand making p2p via wifi opt in, but letting the user decide weather he wants to disable it (maybe because of a public hotspot, or limited data access) is also a good idea, and in the end we may not have to let NewPipe automatically decide whether we should upload or not.

For now I'd say first we need support for the PeerTube Service, once we have that we can continue discussing about the data plan :)

@rugk
Copy link

rugk commented Aug 4, 2018

I mean p2p via cellular ... well I would not enable it […] making it opt out sounds ok for me.

That's opt-in: P2P is disabled by default, but you can enable (opt-in to) it.

making p2p via wifi opt in, […] letting the user decide weather […] want[…] to disable it

And that's opt-out: Enabled by default, but user can disable (opt-out) of it.

You just seem to confuse opt-in/out or have another definition or it (do you consider "not p2p" the "true" state?).

@spaetz
Copy link

spaetz commented Aug 4, 2018

Let me top up the bounty with another 50€ :-).
P.S. As for the uploading, there are Android APIs on whether you are on a metered network or not (see https://stackoverflow.com/questions/23877476/android-why-connectivitymanager-isactivenetworkmetered-always-returning-true)

@yausername
Copy link
Contributor

Yes I am aware of that but I thought of trying it for a start. I guess we need to use https://webrtc.org/native-code/android/ for webrtc support as suggested Chocobozzz/PeerTube#250

@FlorianSteenbuck
Copy link

as I mention before TURN or STUN Server DNS Discovery.

Okay, never heard of this discovery method. Anyway, I think once you have the domain name of a PeerTube instance (and you need to have this), they very likely have some API/way to query the TURN server details and such stuff or you just use one you trust. Actually, I don't know if PeerTube even uses TURN or what exactly they do there. (as obviously the originating server has to "broadcast" the video/torrent somehow, when no one else is streaming it currently)

STUN DNS Discovery is normed by DNS RFC which is used for get the first STUN Server this are fixed domain names, this variables are completely unchangeable in the browser (please proof me wrong, this seem to be a bug in my eyes). Then they connect to the Tracker and get JSON Format that support binary content in json strings and get over this mutiple Offers over the SDP Norm for RTCP, but it seems to be used for the next STUN server and then we got a P2P DTLS Connection, that I am unable to decrypt.

Optional Namecoin Support is not only important for security it is also Important for include PeerTube Instances that got a .bit domain.

Possibly, yes, however do you really think the effort of implementing a blockchain and stuff is worth it? Especially, as there are not even any PeerTube servers I currently know that use it. And if there were, you could argue in the same way you now need to implement .onion support just because a few PeerTube servers use it. (or, as that is relatively easy and may already work, totally different blockchain-based TLD) Or things like OpenNIC etc. You can continue here…

You are right its a huge effort to implementing as I call them In-Network Protocols.

But it turns out to be not that much effort for only namecoin and onion as they are currently two libarys out their orchid namecoinj. Of course I would to suggest to understand the network structures, protocols and the implementation to rewrite a lighter version of these, because their are depending on too much google code.

But anyways the current recommend usage of namecoin is over the namecoin rpc interface which can be simply used with one dependency or even a one class implementation.

Getting back to "TURN or STUN Server DNS Discovery" I see it is 1. "optional" 2. it is just STUN.
Remember what STUN is (note) it is really just for returning your own IP addresses so you can connect.

The only thing an attacker could do is block the connection and you may then not be able to stream the video.

This is the main feature of PeerTube, the thing is you going to be able to stream the video, because PeerTube do not force the user to use WebRTC, they got a fallback to HTTP Range Requests.

PeerTube is crap, it just using a framework that using a new technology that some guys from google created, with old technology that is only used for SIP and NAT, to realize in the end a p2p video stream in a browser and a old technology that is used for p2p file sharing.

A attacker could not only block your connections, they can trace your through your connections, which shouldn't be possible at all.

So, all this DNS stuff is totally complicated and unlikely to be worth implementing – at least considering Peertube.

DNS is one of the biggest mistake that was made in the history of the Internet. I only suggest to implement a encrypted version of this. Namecoin was only a Idea and is Possible in light way as I mentioned above.

Maybe I am little bit radical, but Privacy is equal important as Security.

A issue split is still ok for me @theScrabi

@FlorianSteenbuck
Copy link

@yausername

@FlorianSteenbuck are you working on this? If not I am thinking of writing the extractor for peertube (using APIs) and video playback using https://github.com/butterproject/TorrentStream-Android.

Currently I am not working on this caused by to many problems in private life.

Yes I am aware of that but I thought of trying it for a start. I guess we need to use https://webrtc.org/native-code/android/ for webrtc support as suggested Chocobozzz/PeerTube#250

As you already know me I not so cool with using google dependency so maybe you can implement it yourself the source code seems to readable: https://webrtc.googlesource.com/src/+/master/examples/androidapp/src/org/appspot/apprtc

@rugk
Copy link

rugk commented Oct 14, 2018

@FlorianSteenbuck

PeerTube is crap

Then why are you still commenting in this issue? If you don't like/want it here, go somewhere else and complain about it. This way, it does not contribute to this issue here.
I only miss your comment "NewPipe is crap, because it depends on Google services, i.e. YouTube.".

@FlorianSteenbuck
Copy link

@rugk PeerTube is crap, but is still a challenge for a dev to implement. That is the reason why I still comment on this issue. Most of the Mozilla devs can not implement WebRTC. It is extremly hard to implement this thing. And it is important to implement it to understand what are problematic s are and understand how it actually works. Learning by Doing

NewPipe is great, because it providing a structure for State of the Art Media Platforms in a logic way

I complaining more about Android dependencies than Google dependencies
If you only look at the Source Code of WebRTC you can see that they clearly using Android Classes for the implementation.

Their are multiple things that I mention without my opinion
Sad to see you only want to bash my opinion and take this out of context and even mix it with something that I would never say.

But if you want go to this level I also could:
(Don't take this to serious I just try to show you how wrong you can be with a judging someone to quickly)
@rugk do you got a life ? It seems to be the case that you a got in the abstract understanding of threema, but it seem to be the case that you now after creating a threema library only talking on issues instead of solving issues. That would be absolutely ok, if you know what you are talking about, but it seem to be the case that you not reading the rfc and not reading the norms. You are just one of these web devs that are fascinated by new fancy stuff that is not available in older versions of the languages. You are just some of these jquery devs that do not known that even their fancy loop like function ($.each) can be implemented simply and can be even made more useful by own implementation than their would thought. You not want to solve problems, you want to use fancy stuff to made a whole shit of money of it.

If you want detailed explanation, why this are my first thought about you and your profile, let me know.

@supermamie
Copy link

Just 2 cent about an Android app for PeerTube : https://github.com/sschueller/peertube-android

It seems it is not using WebRTC yet

@TheAssassin TheAssassin changed the title Request add PeerTube Add PeerTube service to NewPipe Dec 4, 2018
@TheAssassin
Copy link
Member

It seems it is not using WebRTC yet

It was suggested already to add PeerTube support w/o the webtorrent stuff to have a first version in NewPIpe. A "proper" implementation could then be written eventually.

@yausername
Copy link
Contributor

It seems it is not using WebRTC yet

It was suggested already to add PeerTube support w/o the webtorrent stuff to have a first version in NewPIpe. A "proper" implementation could then be written eventually.

I already added peertube service in newpipe extractor https://github.com/yausername/NewPipeExtractor/tree/peertube and it works well with NP app. But I am struggling with WebRTC data channels.

@Chocobozzz
Copy link

Honestly, it's not very important if the application does not use WebRTC P2P. If it supports classic TCP/UDP it's enough already.

@yausername
Copy link
Contributor

PeerTube on NewPipe (video playback is via http).
https://peertube.mastodon.host/videos/watch/d94202b0-70fe-4ba6-87b0-e8e206eb4674

@theScrabi
Copy link
Member

O.o oaaah yasuserdame, the code magician.

@yausername
Copy link
Contributor

O.o oaaah yasuserdame, the code magician.

lol. you are the real saviour man. good luck for your finals.

@theScrabi
Copy link
Member

So I think for the beginning we can add peertube support without peer to peer support. However it would be nice if the extractor could select its server dynamicly using the instance api.

@strypey
Copy link

strypey commented Mar 3, 2019

Great work @yausername I look forward to testing it!

@Rik44444
Copy link

great idea, i would love peertube support!!

@TobiGr
Copy link
Member

TobiGr commented Dec 3, 2019

I am excited to mention, that @yausername's extractor PR was merged today. Thank you again!
(P2P has not been implemented yet).

@TobiGr TobiGr added the peertube service label Dec 3, 2019
@Rik44444
Copy link

Rik44444 commented Dec 3, 2019

that's great news!! am gonny try immediately

@Rik44444
Copy link

Rik44444 commented Dec 3, 2019

i don't see a new version on f-droid yet, is that correct?

@Stypox
Copy link
Member

Stypox commented Dec 4, 2019

The PR was merged on the extractor. The app PR is still pending. Also, the release journey takes time.

@seniorm0ment
Copy link

Glad to see this happening!

@theScrabi
Copy link
Member

This is the pr of the frontend that made this possible: TeamNewPipe/NewPipe#2201

@wb9688
Copy link
Contributor

wb9688 commented Feb 25, 2020

Could this issue be closed? Or should we leave it open as we don't have actual P2P support yet?

@strypey
Copy link

strypey commented Feb 26, 2020

@wb9688 the issue description asks if support for playing video from PeerTube instances can be added to NewPipe. If it's now possible to do that, I'd say the issue can be closed. P2P support was only mentioned as a possible way of achieving that, and I imagine there is issue(s) for that.

@Stypox
Copy link
Member

Stypox commented Feb 26, 2020

@Stypox Stypox closed this as completed Feb 26, 2020
@Rik44444
Copy link

i agree, for me this issue can be closed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests