Provide GitHub as Onion Service (The Onion Routing)

[Kreyren]

At GitHub, our priority is supporting open source and the developer community. -- Github November 16, 2020 (https://github.blog/2020-11-16-standing-up-for-developers-youtube-dl-is-back/)

Privacy is one of the most important factors when it comes to developing free (as in "freedom") open-source software to ensure that the internet traffic sent by the developers cannot be abused to gain privileged access to our repositories or otherwise used agains us in an unwanted and/or illegal manner such as racial profiling[5], doxxing[4], blackmail[3], etc..

As such the onion services give the site owner control over both authentication and network address lookup in a way that can't be usurped like TLS certificates or DNS can while maintaining the end-user's control over their data to mitigate or solve following issue as the World-Wide-Web (WWW) provides next to no privacy even through the use of proxy and Virtual Private Network (VPN).[13]

1. issue: Browser fingerprinting

Practice used to track end-users over the internet without their consent by using the informations exposed about them by the web browsers such as used version of graphical drivers through the WebGL[2][6], Screen size[6], Device Memory[6], Use of an AdBlocker[6], language[6] and many more..

As of 07/12/2020 the only known solution to this is issue is for everyone to use the same browser with same extensions and configuration such as Tor Browser Bundle that allows the end-users to blend in a crowd of users making them less likely to be fingerprinted[7].

2. issue: Commercial mass surveillance

Practice used to create a database entry of the online users (often without and/or uninformed consent) of gathered (and often sensitive) informations for the purpose of targeted advertisement[9], paid redistibution[10] and interfiering in state election[11].

3. issue: Internet Service Provider (ISP) logging, man-in-the-middle (MITM) attacks, certificate authorities and DNS servers

Some end-users are in situation where in their area there is only one ISP which policy is to resell the user data[12] gathered from the end-user's internet traffic or the ISP might be using a compromised relay that could be used to monitor the traffic and to perform MITM attacks.

Tor allows these users to gain control over their data by making their traffic harder to interpret.

4. issue: Certificate authorities (CA)

On WWW the certificate authorities (or anyone who tricks CA) have full control over the sensitive data transmitted to the github.com which is not a concern on an onion service as onion services are also self-authenticating (the key is built into the address)[13]

5. issue: Domain Name System (DNS)

The WWW depends on DNS servers to interpret the domain names into an IP address which is understandable by the web browsers that depending on the used encryption and policy set by the DNS provider will expose the traffic to the DNS providers and possible bad actors monitoring the traffic.

The DNS providers and parties that has control over these servers have the ability to redirect the domains on an IP address of their choice.

DNS resorvers are also vulnerable to "DNS cache poisoning" which is a practice used to trick the DNS resolver into a caching of a false information.[17]

Tor does not use DNS to resolve the onion service thus eliminating these issues.[16][18]

Proof-of-concept

Implementation on dotya.ml

I was helping with implementation of a onion service on https://git.dotya.ml that provides it on http://2crftbzxbcoqolvzreaaeyrod5qwycayef55gxgzgfcpqlaxrnh3kkqd.onion

Experiment by @alecmuffett using EOTK to provide github as onion service

More in #2843 (comment)

How does tor works

When a user visits github through Tor, they go through a 3 hops circuit. Specifically an entry node, a middle node, and finally, an exit node. Then they will reach their desired website.
If github offered an onion the communication would stay within the Tor network. This mean the user wouldn't have to go through an exit to reach the onion site [15], resulting in a better experience, privacy and security for the Tor user.

Further explained on:
On youtube: https://www.youtube.com/watch?v=JWII85UlzKw
On Invidious: https://invidious.snopyta.org/watch?v=JWII85UlzKw

Use of tor in the wild

New York Time https://www.nytimes3xbfgragh.onion/tips

Facebook http://facebookcorewwwi.onion

DuckDuckGo https://3g2upl4pq6kufc4m.onion

BBC https://www.bbcnewsv2vjtpsuy.onion (EOTK)

Deutsche Welle https://www.dwnewsvdyyiamwnp.onion (EOTK)

Brave Browser https://brave5t5rjjg3s6k.onion (EOTK)

The Free Software Foundation Europe http://fsfeorg3hsfyuhmdylxrqdvgsmjeoxuuug5a4dv3c3grkxzsl33d3xyd.onion

Tor has been also adapted by FLOSS projects such as:

Snopyta.org http://cct5wy6mzgmft24xzw6zeaf55aaqmo6324gjlsghdhbiw5gdaaf4pkad.onion/

Websites marked with 'EOTK' are set up using https://github.com/alecmuffett/eotk, see #2843 (comment)

References

1. EFF's campain "Cover your tracks" dedicated to reduce the fingerprint of your browser https://coveryourtracks.eff.org/learn
2. Wikipedia's article on Device fingerprinting related to Browser fingerprint through Canvas and WebGL https://en.wikipedia.org/wiki/Device_fingerprint#Canvas_and_WebGL
3. CNN report on an example of online blackmail https://edition.cnn.com/2020/11/25/asia/korea-telegram-sex-crime-verdict-intl-hnk/index.html
4. Wikipedia article providing examples to doxxing https://en.wikipedia.org/wiki/Doxing#Examples
5. Associate Press report on online social profiling in relation to France law https://apnews.com/article/race-and-ethnicity-emmanuel-macron-racial-profiling-france-media-9075d5e1933a429bea99c88c5b3a7c19
6. EFF's Cover Your tracks tool used as a fingerprinting test for the public https://coveryourtracks.eff.org/
7. Torproject's docs on handling browser fingerprinting https://blog.torproject.org/browser-fingerprinting-introduction-and-challenges-ahead
8. Wikipedia's article on Commercial Mass surveillance https://en.wikipedia.org/wiki/Mass_surveillance#Commercial_mass_surveillance
9. Facebook rejecting 2.2m targeted ads indenteded to interfiere in US election https://www.theguardian.com/technology/2020/oct/18/facebook-says-it-rejected-22m-ads-seeking-to-obstruct-voting-in-us-election
10. Electronic Frontier Foundation's article on Google sharing user-data for alleged profit https://www.eff.org/deeplinks/2020/03/google-says-it-doesnt-sell-your-data-heres-how-company-shares-monetizes-and
11. Wikipedia article on political usage of profiled users https://en.wikipedia.org/wiki/Facebook%E2%80%93Cambridge_Analytica_data_scandal#Potential_usage
12. Comcast's Xfinity terms of service to process personal information https://www.xfinity.com/Corporate/Customers/Policies/CAVoiceDisclosure
13. Feedback from a tor affiliate Provide GitHub as Onion Service (The Onion Routing) #2843 (comment)
14. The "Peel of Onion" docs https://www.acsac.org/2011/program/keynotes/syverson.pdf
15. Overview of the onion services https://community.torproject.org/onion-services/overview/
16. Clarification to how Tor handles the DNS servers Provide GitHub as Onion Service (The Onion Routing) #2843 (reply in thread)
17. Cloudflare reference to DNS cache poisoning https://www.cloudflare.com/learning/dns/dns-cache-poisoning/
18. Internet Engineering Task Force's reference to onion's RFC https://tools.ietf.org/rfc/rfc7686.txt

---

Feel free to provide resources and review the provided information as i will be happy to update the original post.

[ghost]

Thank you very much for this great proposal, well researched, very timely, and one that we should all consider necessary. Github is one of the most used platforms by developers, and surely the most used for code hosting. This implies a responsibility, and a lot of effort has to be made!

[syverson]

Generally, I am enthusiastically in favor of this proposal. A few suggestions

1. "Onion Service" not "Hidden Service".
   Hidden service was the name for version 2 onion services (or earlier). These are now deprecated in favor of v3 onion services and will be completely disabled in the tor client in less than a year. https://blog.torproject.org/v2-deprecation-timeline
2. This is not just a verbal preference. When we originally called these "hidden services" (actually "location hidden services" originally) we were focusing exclusively on the kinds of issues emphasized in your description. Those are important, but ignores other important features. You touched on it when you said that users gain greater control over their data. And, I would add meta-data. Autonomy isn't just about hiding site information such as network location except for those to which you want to give it. It's also about who gets to say what counts as legitimate information about that site. In particular,
   onion services are also self-authenticating (the key is built into the address). And, the lookup of an onion address is authenticated by the same private key. It's not only commercial surveillance and ISP logging that is a problem. Certificate authorities (or anyone who tricks a CA) have control over what counts as authenticating a connection to github.com. And the DNS system can be manipulated to even say how to get to github.com. Onion services give the site owner control over both authentication and network address lookup in a way that can't be usurped like TLS certificates or DNS can. You might add to/revise your description to indicate that hiding is only one aspect of the general goal of greater autonomy for site owners and site visitors.
3. More of a quibble. It's 'The Onion Routing', not "The Onion Router", though that mistake has happened enough it is pretty much a change in the language. See p. 129 of https://www.acsac.org/2011/program/keynotes/syverson.pdf if you want to know more.

[Kreyren]

Also please elaborate on metadata and DNS as i currently don't know how to interpret it in the OP.

[amcgregor]

@Kreyren DNS is typically requested over UDP, not TCP, which avoids the use of most proxies which would route such queries over Tor onion routing. These requests would "leak" to your ISP or other configured DNS provider (8.8.8.8 being Google, for example,) and those providers determine how your browser directs requests for a given domain name. Those DNS servers can both monitor your activity (in terms of which domain names you are accessing) and misdirect you towards servers other than those actually associated with the domain.

There are edge cases, where the request or response are larger than the limited space available in a UDP packet where TCP will be utilized, but this is currently an edge case and not the default. There is a push for DNS-over-TCP-by-default and DNS-over-HTTPS to try to address this privacy weakness in the current way we resolve names.

[sporksmith]

To clarify - when using the Tor Browser, the user's DNS activity isn't freely observable at the point of the user. The Tor Browser does route DNS over tor (as a special case, since as you point out tor doesn't generally support UDP).

Using an onion service does prevent some other attacks via DNS though, such as observing the clear DNS requests at the exit and correlating with traffic at the user, or manipulating or blocking DNS results at the exit.

[syverson]

To clarify further, sporksmith is right if what is being looked up is the IP address of GitHub.com. If, however, the address Tor Browser sees is [onion-address-of-github].onion then DNS is never touched. (In fact RFC 7686 essentially does not permit lookup of onion addresses by DNS resolvers https://tools.ietf.org/rfc/rfc7686.txt ) So, onion address lookup never exits the Tor network and does not use DNS at all.

Lookup happens via 3-hop Tor connections into a distributed hash table comprised of thousands of Tor relays. And for any onion address that a relay hosting a descriptor (address lookup information) does not know in advance, the relay does not even know what onion services they are hosting descriptors for. And you need to know the private onion key to be able to place the current descriptor for an onion address into the DHT.

[Kreyren]

Updated OP, please review ^-^

[alecmuffett]

EOTK is the solution in use by the NYTimes, BBC, Deutsche Welle and Brave Browser onion sites, to set up an Onion service with minimal effort and maximal compatibility, and working around the security issues in handling an Onion site. Disclosure: I am the author, and would be delighted to help.
https://github.com/alecmuffett/eotk

[Kreyren]

Updated OP with information provided ^-^

[hiromipaw]

This is a great proposal.

For users under surveillance that use Tor to access github, an onion service will allow them not to have to leave the tor network to reach you.
To understand why this is important, let me provide a bit of background.
When a user visits github through Tor, they go through a 3 hops circuit. Specifically an entry node, a middle node, and finally, an exit node. Then they will reach their desired website.
If github offered an onion the communication would stay within the Tor network. This mean the user wouldn't have to go through an exit to reach the onion site [1], resulting in a better experience for the Tor user.

Please do let us known if Github is actually interested in running an onion. I am a sysadmin The Tor Project and we would love to help.

[1] https://community.torproject.org/onion-services/overview/

[sporksmith]

+1

If github offered an onion the communication would stay within the Tor network. This mean the user wouldn't have to go through an exit to reach the onion site [1], resulting in a better experience for the Tor user.

In particular, this removes the Tor exit nodes and those exit node's ISP's as a potential bandwidth bottleneck, and potential attack surface for traffic censorship, de-anonymization (via timing correlation by an attacker who can also observe the user's traffic), or modification (via https downgrade attacks, etc).

[Kreyren]

Added in OP thanks for info provided ^-^

[hiromipaw]

Yes and it is also important to note that if any third-party service that github is using is blocking tor-exit-nodes, the experience for github users accessing the service through tor is degraded.

[Kreyren]

@hiromipaw Is there such 3rd party service? Or how do you propose it to be interpreted in the OP?

[hiromipaw]

Third party services are like CDNs that serve some content for github website. If these block the current exit node that the user is going through to access github via Tor some assets might not load, hence the experience is degraded. I have checked the requests on github.com and it seems these are all going through github.com and their other subdomains. So third-parties blocking tor exit might not be an issue in this case except if one github service is implementing different rules regarding tor traffic.

[FormerlyChucks]

It would be nice to be able to utulize GH via Tor. I only use Chrome for the sites that block Tor. This is great!

[deufrai]

+1
Yes guys, that would be a huge step

[FormerlyChucks]

@deufrai
You can add a reaction to show support. Useless comments like +1 sends an email to everyone who has added input.

[Kreyren]

I think that people expressing support for this integration only helps the proposal.

[asn-d6]

Hello all. I work on Tor as a developer and more specifically I work on onion services. I would be glad to help you deploy onion services on your site. If you need any sort of support or help or directions, please let us know :)

[alecmuffett]

Hey All!

As an experiment, I just set up an Onion site for Github using EOTK, to see how hard it would be. I learned something that I will share below, that will impact deployment of a .onion site irrespective of how it's done.

The configuration which I created, is this github.tconf file:

# don't rewrite user@github.com email addresses into an onion
set preserve_csv tld-email,github\\.com,i,github.com

# deployment; use of "mkcert" requires you to install it and set it up beforehand
# and to adopt your mkcert CA certificate into your TorBrowser.
set nginx_resolver 8.8.8.8 1.1.1.1 ipv6=off
set log_separate 1
set ssl_mkcert 1
set nginx_cache_seconds 60
set nginx_cache_size 256m
set nginx_tmpfile_size 64m

# needed to develop this for github, see: https://github.com/alecmuffett/eotk/pull/85
set kludge_disable_sri 1

set project github
hardmap %NEW_V3_ONION% github.com
hardmap %NEW_V3_ONION% githubapp.com
hardmap %NEW_V3_ONION% githubassets.com
hardmap %NEW_V3_ONION% githubstatus.com
hardmap %NEW_V3_ONION% githubusercontent.com
hardmap %NEW_V3_ONION% octocaptcha.com

...and I then did eotk config github.tconf and eotk start github, and it worked, kinda.

With a tiny bit of EOTK-hacking I got it working fully:

...and I have now taken this onion site down to ensure that nobody else uses it untowardly.

In the process I learned that Github makes significant use of subresource integrity (SRI) which is a very good thing for them to be doing, but which complicates greatly any rewriter-based approach, other than for EOTK to stomp upon any integrity tags as the new kludge_disable_sri option lets you do... but that's not really a solution fit for production.

This observation suggests that the fastest route towards getting onion-assist for people who use Github over Tor, might be to use content-neutral Alt-Svc headers, which do not require content rewriting; or maybe a more extensive discussion with the people of Github towards addressing SRI-based integrity wants.

[*] edits for typos

[Kreyren]

mentioned in the OP, thanks for working on this!

[loannaflip]

[loannaflip]

Not little. A lot slower.

[PrivateGER]

Then your info is severely outdated. Tor can reach a solid 5MB bandwidth nowadays, with latency being the only slow thing involved.

[alecmuffett]

I have to admit that I have not yet done a sustained Gigabit over Tor yet, but that's a good problem to have.

[loannaflip]

I've 100mbps dedicated line. And I'm getting around 0.24kbps on Tor without VPN.

[PrivateGER]

@Uniminin Try to refresh your guard, this should not happen.
Here's a speedtest I just did with Tor while connected to a VPN: https://www.speedtest.net/result/10554132278

[Kreyren]

CC @becca, official response from GitHub would be appreciated.

[nopara73]

Oh wow, this'd be great! 🚀 We were considering to move our release downloads out of GitHub because of the lack of Tor support, but if users could download our software over Tor without needing to touch a Tor exit node, that'd fix this privacy leak!

[becca]

👋 hello there! I saw I was pinged in #2843 (comment) and wanted to respond here officially. This repository is currently targeted only for feedback related to the Discussions and GitHub for Mobile features (#1). This request might belong in https://github.community/.

[Kreyren]

@becca Could you add a new category for it and process it here then? Seems quite off-topic for https://github.community as that's Support Community where this is proposing integration in the GitHub as a service (irrelevant to the support by definition i would think).

[Kreyren]

CC @becca did you miss the reply? ^

[gman999]

@becca, what is the best route to this getting addressed?

Is there any official word on the content of Kreyren's OP?

Appreciated.

[PowerPress]

Please implement this.

[gman999]

I think there is a very strong case for this, Kreyren, and really glad you submitted.

I hope GitHub takes this feedback seriously and addresses.

There are a number of reasons why GitHub should setup a .onion site, and one of the strongest is really about providing fuller accessibility. If GitHub wants to provide access to censored users, whether at work or on the nation-state level, then an .onion site is a critical tool.

In a world where the internet is transforming into separate internets, .onion sites are vital to the global internet community.

[FormerlyChucks]

10$ says they'll never implement this - it took them 7 years to simply implement a dark theme. They can't hide behind "it's too hard" either, we're on a site where millions of programmers come each day.

[Kreyren]

CC @ashtom With twitter taking steps to circumvent russian censorship in the conflict[https://www.engadget.com/twitter-tor-onion-service-evade-censorship-210549633.html] - why don't you do the same?

Alec Muffett (@alecmuffett) who helped with the deployment for twitter already prepared the EOTK for github (#2843 (comment))