ping torproject.org over plainnet?

[nyxnor]

can't acquire new bridges or list their status just because the domain is blocked.
Why not make the request over Tor? Possible responses for that:

questions to think of possible outcomes

1 - domain blocked

• But if Tor is not running?
  • Because connection to the network is blocked? Well, reaching torproject.org over plainnet would the worst in this case, they already block that or even store the user query.
  • Wrong config? If Tor is not running not because of this, some config might be wrong.
• But wouldn't it be a hassle to modify?
  • nah, just use: session.proxies['http']='socks5h://localhost:9050' basically, of course there is more involved. Example (1) (2)

2 - tor connection blocked

• Should try the connection any way? Well, possibly the attacker also blocks the domain, so this method shouldn't be recommended, but instructed to add manually if got no connection to the database. Is it possible to now if it is blocked without trying? If user already knows, should a parameter be set to not trying tor connections without bridges?

mapping

domain blocked

• domain blocked > cant connect to the database > instruct to add manually
• domain blocked > try to connect to the database over tor > connection established > bridges added

tor connection blocked

• tor connection blocked > cant connect to the database (domain probably blocked) > instruct to add manually
• tor connection blocked > try to connect to the database over tor > user already had bridges online and connection is established > bridges added

also, there is already preset bridges in the torrc, so shouldn't have to worry about using the defaults if needed at first

can't connect for whatever reason

• instruct to add manually
• give info about what was the cause if possible.

[nyxnor]

Why set config at the begginning?

• If user knows domain is blocked, why even ping?
• If user knows connection is blocked and he wants bridges, why even start without them?

Tails changelog

https://tails.boum.org/news/test_4.20-rc1/
Over the years, we have identified many issues in how to connect to the Tor network from Tails, thanks to your feedback. With this release candidate, we are trying to solve the most important and pressing of these issues:

• The lack of feeback while connecting to the Tor network.
• The option to use Tor bridges has to be turned on in the Welcome Screen and cannot be changed afterwards.
• The lack of default Tor bridges.
• The previous configuration screen for Tor bridges wasn't accessible to people who use the screen reader.
• The lack of guidance when Tails is not connected to a local network yet.

My comment

So, before Tor even starts, mask the service, ask to configure bridges or change other configs, but bridges is what matters

[radio24]

Thank you, nyxnor. You raise good points, but I have to look into it more in detail.
The ping, for example, has not only the function to see if the domain is blocked - but the service could also be offline. Actually, that was the initial reason for this kind of check.

A very good suggestion is to use session.proxies['http']='socks5h://localhost:9050'. However, that has to be checked in detail on side effects. We have to avoid a situation in which someone needs tor to fix the tor installation. Also, we have to keep in mind that TorBox also has to work with Captive Portals. This is why the local DNS resolution (not the ones from the clients) cannot be routed through Tor (see more details here).

In other words: very good suggestions, which we should look into, but this will not be done in a few weeks.

For whom it concerns
Currently, I'm on travel, and for that reason, I don't have time to make basic changes. My focus is on a check & fix system, which detects a connection loss with the Tor network and which automatically initiates countermeasures (for example, the change of an entry guard or the restart of tor). If you are interested in the development, check out the "automatisation-test" branch.

[nyxnor]

Tor Browser starts with DisableNetwork 1, so the first prompt is to configure bridges if needed or set other parameters such as proxy, security settings.
https://manpages.debian.org/buster/tor/torrc.5.en.html

DisableNetwork 0|1
When this option is set, we don’t listen for or accept any connections other than controller connections, and we close (and don’t reattempt) any outbound connections. Controllers sometimes use this option to avoid using the network until Tor is fully configured. Tor will make still certain network-related calls (like DNS lookups) as a part of its configuration process, even if DisableNetwork is set. (Default: 0)

But tor (daemon, service) does not come with this by default, so it automatically connects to the network.
Either place torrc in /etc/tor/torrc with DisableNetwork 1 before instaling the package (could already be done and I haven't noticed, as it is already in the path), or mask before installing with sudo systemctl mask tor@default.service.

[radio24]

I guess the DisableNetwork 0|1 is a different topic, and I don't see the benefit it would have for TorBox.

Either place torrc in /etc/tor/torrc with DisableNetwork 1 before instaling the package (could already be done and I haven't noticed, as it is already in the path), or mask before installing with sudo systemctl mask tor@default.service.

What do you mean with "before installing the package"?

[nyxnor]

Just start tor service after configuring bridges, making it similar to Tails or Tor Browser way #67 (comment)
By disabling network, could achieve this, considering the topics expressed above.

[nyxnor]

Another good thing is to acquire bridges with moat https://gitlab.torproject.org/tpo/applications/tor-browser/-/blob/tor-browser-78.11.0esr-10.5-1/browser/modules/BridgeDB.jsm
Well, if this could be translated to python would be good.

[radio24]

Another good thing is to acquire bridges with moat https://gitlab.torproject.org/tpo/applications/tor-browser/-/blob/tor-browser-78.11.0esr-10.5-1/browser/modules/BridgeDB.jsm
Well, if this could be translated to python would be good.

This is already integrated. Entry 3 in the Countermeasure sub-menu gives the possibility to automatically add a OBFS4 bridge every 24h.

[nyxnor]

With moat? It is with HTTPS no?

TorBox/bridges_get.py

Line 32 in c27028f

tor_get_bridges_url = 'https://bridges.torproject.org/bridges?transport=obfs4' # url where we get the bridges

If domain is blocked, this won't work.

I don't know at the current time how to implement moat .jsm to python to be called via command line.

[radio24]

With moat? It is with HTTPS no?

TorBox/bridges_get.py

Line 32 in c27028f

tor_get_bridges_url = 'https://bridges.torproject.org/bridges?transport=obfs4' # url where we get the bridges

If domain is blocked, this won't work.
I don't know at the current time how to implement moat .jsm to python to be called via command line.

You are absolutely right... it is https. I will ask my Python guru, if he has any idea how to integrate that.

[radio24]

Smart torification

Old code: use clearnet
OCHECK=$(curl -m 6 -s $BRIDGE_DB_URL)

New code: first try to use tor, and if this is not working, use clearnet

#-m 6 must not be lower, otherwise it looks like there is no connection!
OCHECK=$(curl --socks5 192.168.42.1:9050 --socks5-hostname 192.168.42.1:9050 -m 6 -s $BRIDGE_DB_URL)
if [ $? == 0 ]; then OCHECK="0"; else OCHECK="1"; fi
if [ $OCHECK == 1 ]; then OCHECK=$(curl -m 6 -s $BRIDGE_DB_URL); fi
if [ $? == 0 ]; then OCHECK="0"; else OCHECK="1"; fi
if [ $OCHECK == 0 ]; then

[nyxnor]

Thank you for this: get bridges through tor in case of dns attack and Try to fetch bridges over tor

[radio24]

With the latest commit from Carlos Martinez, we turned it around: first, we try to fetch the bridge over tor and only if that fails then we use clearnet. In the script executing bridges_get.py, the nameservers were switched to public ones before. This means that if the script is using clearnet, the public nameservers will not be affected by cheap DNS related censorship.

We will implement the same for bridges_check.py

[nyxnor]

In the build script, you first acquire the keys from the torproject.org domain., then build from github, then add snowflake from the torproject.org domain.
Acquiring the keys will fail if the domain is blocked, as well as the snowflake.

What about ordering the build script this way:

1. Mask tor
2. Install with apt from the debian repo
3. Ask if want to configure bridges
4. Unmask tor
5. Acquire TPO keys with torsocks
6. Install from any git (git.torproject.org, gitlab.torproject.org, github.com/torproject.tor) with torsocks git clone.

You can see this build example here 1, 2, 3, 4, 5 - All from here

If using the torproject.org plainnet domain, it will fail when the domain is blocked, see related issue here (it was using cloudflare 1.1.1.1 dns resolver and failing. What should be used, or at least an option to choose is to APT over Tor, my example here. That user is from Turkey, he has been using my build script and working for him, see PR for him here. Not merged mainstream on the other project due to the build script being programmed to change a lot.

Change the git repos of Snowflake to https://github.com/keroserene/snowflake.git, avoiding pinging torproject.org or acquire it with torsocks.

See how you can change sources with this whiptail I made, sources here, the equivalent file of torbox.lib.

Can I do these changes?

[radio24]

In principle, I fully agree with you. Some of the points you raise are on my personal "smart torification" list, which I like to integrate for the upcoming version. However, there are also some differences and open questions between your approach and my current idea:

• I agree with you that the URL "torproject.org" should probably be avoided if used through clearnet.
• I also already studies your whiptail menu, although I wouldn't give the user too much possibility to interfere. My idea is that the script does the safest way, and if that fails, it tries an alternative method.
• The approach to use torsocks seems interesting. However, I don't think it is necessary to clone the entire repository. Also, I guess this is only a viable solution for an update and not for an installation when tor is not installed yet.
  Regarding Cloudflare public nameserver, I didn't find the linked discussion comment that 1.1.1.1 would not work to circumvent a cheap censorship mechanism. I had the same situation in the country I was staying, and it worked without any problem with Cloudflare and Google public name servers. However, I agree that more sophisticated censorship will be a problem, and for that reason, torrifying is a good approach, where possible.
• I guess one significant difference could be that I use two variants of tor installation so far. One is fetched from the torproject.org source repo and the other from GitHub. With the GitHub approach, the user can select which version of tor he wishes to install. Although the goal is to torify the TorBox, I will not change the possibility of choosing between different versions.
• What are the reason that you mask tor. I guess this would be only in case of an update. However, for me, it is not clear the benefit it brings with it. Instead of linking the service to /dev/null, it would be better to try using tor as long as possible to download the needed files for the installation. Of course, this is only possible with updates, not with the initial installation.

Although I may not accept all of your changes, it would be very interesting, helpful and time-saving for me if you could create such a pull request.

[nyxnor]

I also already studies your whiptail menu, although I wouldn't give the user too much possibility to interfere. My idea is that the script does the safest way, and if that fails, it tries an alternative method.

I cringe at my whiptail menus ( I tried to keep it super simple). But about user messing up the configuration, well, this is just giving them choices. Maybe when you build TorBox, you just set the security you want with it just like Tor Browser:

• Do you want all trafffic router through Tor?
• Do you want to configure bridges with pluggable transport before starting Tor?
• Do you want to APT over Tor?

This can all be solved with one simple option, ./install/run_install.sh [normal|torified].

I didn't find the linked discussion comment that 1.1.1.1 would not work to circumvent a cheap censorship mechanism

It is inserted here. Although it was not discussed on that issue, I was chatting with that user privately to help him and he stated he was using.

I guess one significant difference could be that I use two variants of tor installation so far. One is fetched from the torproject.org source repo and the other from GitHub. With the GitHub approach, the user can select which version of tor he wishes to install. Although the goal is to torify the TorBox, I will not change the possibility of choosing between different versions.

Building from git is much better. I just mentioned to install from package manager cause from the Blitz project, it is to be faster, but for a Tor router, yes should focus on fetching from github.

What are the reason that you mask tor.

Yes, I did not explain here, but the reason to mask tor is to avoid making any connections to the Tor network before configuring bridges. If the user already has bridges, he can just uncomment that from the torrc and then let tor start with all traffic routing with bridges with pluggable transport, so no tries before a safe configuration. If the user does not have bridges, even though there are already 36 in the torrc sample, tor needs to be unmasked and acquired from the database with onion routing.

Although I may not accept all of your changes, it would be very interesting, helpful and time-saving for me if you could create such a pull request.

Let's discuss thoroughly what is needed so when there is enough information and agreement, I will start.

[radio24]

I'm thankful for all the inspiration you give to the project. Nevertheless, currently, it is a little bit hard for me to explain in detail what I want. Usually, I start with an idea and then it develops it further when I see the code in front of me. So, I began to torify the update process. For me, the best way is that first, the script tries to fetch all the necessary packages through tor. If that fails, it will switch to clearnet (it asks for permission to do so, first). The name server was changed to public ones before.

You can find parts of the new code here, here, here, and here

Summary
The idea is not to route the local TorBox data through tor because that will break the captive portal capabilities and has other side effects if tor is not working properly on the TorBox anymore. However, I will torify - if possible - all data traffic for the maintenance, bridge fetching, bridge checking and so on. I guess this is the main difference between the Raspiblitz and TorBox.

Problem

• This approach will not work with an installation script running in a censored network - for the installation script, there has to be another way —> nyxnor‘s approach.
• To switch from tor to clearnet could be problematic in a very autocratic state —> introducing a „paranoia switch“, which forbids to use clearnet.

[radio24]

I just finished looking at the install / build script for the Raspiblitz. Very well done and very impressive work. Also, I understand now the ordering of your build script. If you have time and motivation, I would be thankful, If you could work on run_install.sh for TorBox.

[nyxnor]

I can do that. update script looking good!
Will add in the install script apt-transport-tor, good to add here after apt-transport-https.
I recommend having a review on that page, it is missing python packages too such as pillow and requests (requirements)

[radio24]

I recommend having a review on that page, it is missing python packages too such as pillow and requests (requirements)

I will update the page after the release of v.0.4.2. This is also the branch we working on.

[nyxnor]

Thoughts on implementation:
Test if tor is working, if it is, try with torsocks and hidden service for all its benefits, if not working, either:

• ping torproject.org plainnet.
• exit script warning it avoided to ping the domain? I don't know if you want this or not with paranoia switch.

[radio24]

Thoughts on implementation:
Test if tor is working, if it is, try with torsocks and hidden service for all its benefits, if not working, either:

• ping torproject.org plainnet.
• exit script warning it avoided to ping the domain? I don't know if you want this or not with paranoia switch.

I guess you are speaking about the installation script. If possible, it should be the same process as we have now in the update script. Test if we can reach the URL (.onion) through tor. If no, ask the user if he wants to connect through clearnet or not. If no -> exit.

At the end of the installation and on the image file, tor can still be masked. Therefore log in (SSH or web) is mandatory to start the initialisation function in the main menu script. There the user can be asked if he wants to activate bridges and to forbid clearnet access on tor related URLs. After he answered this question and the settings were set accordingly, tor can be unmasked.

[nyxnor]

Slight plan:

run_install.sh

# SYNTAX
# ./run_install.sh <--select-tor> <--circumvent>
#
# The <--select-tor> option allows the user to select a specific tor version.
# Without this option, the installation script installs the latest stable version.
#
# The <--circumvent> option will:
# * default to APT over Tor for the Tor Project packages
# * calls menu-bridges to configure bridges and bypass censorship
# Without this option, will ping https://deb.torproject.org for sources and no bridges by default.

if [ "${CIRCUmVENT}"== "--circumvent" ]; then
  CIRCUMVENTb=1
  SOURCES_TOR_UPDATE_CURL="--socks5 192.168.42.1:9050 --socks5-hostname 192.168.42.1:9050 http://apow7mjfryruh65chtdydfmqfpj5btws7nbocgtaovhvezgccyjazpqd.onion"
else
  CIRCUMVENTb=0
  SOURCES_TOR_UPDATE_CURL="https://deb.torproject.org"
fi

sudo sed -i "s/^CIRCUMVENTION=.*/CIRCUMVENTION=${CIRCUMVENTb}/" ${RUNFILE}`
sudo curl ${SOURCES_TOR_UPDATE_ONION_CURL}/torproject.org/A3C4F0F979CAA22CDBA8F512EE8CBC9E886DDD89.asc | sudo apt-key add -

menu

check_fresh_install
{
  CIRCUMVENT=$(grep "^CIRCUMVENT=.*" ${RUNFILE})
  if [ $CIRCUMVENT = "CIRCUMVENT=1" ]; then
    clear
    bash menu-bridges
    online_check_update
    set_sources_list
  fi
}

[radio24]

In the update script, I turned it around: first, try over the tor socket, if that is not working, ask if using clearnet is ok. It may be better, we use the torsocket for all tor related URLs as default and not the other way around.

[radio24]

Implemented with TorBox version 0.4.2!