Run on Whonix #2
Comments
Thank you for your interest of running unMessage in Whonix! Much appreciated! Related:
|
To run on Whonix, I used the What is not clear to me is the need of using a specific SOCKS port because unMessage always connects to another hidden service. From the link above: "Different Tor Hidden Services are automatically stream isolated". Can you clarify?
That's it! I did follow those instructions, but unMessage actually only listens on localhost. Do you think we should forward connections from the gateway's IP to localhost, or directly listen on the external interface? The former might require some action from the user, while the latter might be easier because unMessage can automatically do that when starting the server. I think we just need a way to find out the address to use for both Whonix and Qubes-Whonix.
Thank you very much Patrick! |
|
Felipe Dau:
> (easy) Using SocksPort 127.0.0.1 9050 should work in Whonix. (Gets
> automatically redirected to the gateway by
> anon-ws-disable-stacked-tor.) (a bit harder) Point directly to
> Whonix-Gateway. (As per
> https://www.whonix.org/wiki/Stream_Isolation. We can do that
> later.)
To run on Whonix, I used the `--no-tor-proxy` flag to **not** use any
SOCKS port. It then creates a regular Twisted `TCP4ClientEndpoint`
and connects to the onion address. This outbound traffic (initiating
the connection to a hidden service) is working, just like `telnet
domain.onion port` "magically" works, for example.
I understood that. I was suggesting to use socks anyhow even in Whonix.
Because...
What is not clear to me is the need of using a specific SOCKS port
because unMessage always connects to another hidden service. From the
link above: "Different Tor Hidden Services are automatically stream
isolated". Can you clarify?
Connection to a.onion will use a different circuit than a connection to
b.onion.
Now, let's say unMessage would connect to a.onion and another
application such as Tor Browser or sdwdate would also connect to
a.onion. A while ago these would have been contaminated, i.e. used the
same circuit. There however won't be such a risk if different SocksPorts
are used. Then connection to a.onion through SocksPort X and connection
to a.onion through SocksPort Y would also be stream isolated.
The risk may be low, since Tor Browser now also uses per-tab isolation
as well as a random string after browser restarts for its socks user
name and sdwdate a dedicated separate Tor SocksPort.
Not using socks but regular system TCP has the disadvantage, that it
goes through Tor's TransPort. We at Whonix try to keep that fallback
port by default unused. Reserved for traffic generated by user's custom
installed applications only.
Another reason might be simplicity. Using the same code in Whonix as
elsewhere. Using TCP4ClientEndpoint outside of Whonix would be bad, I
mean if that code unwanted kicks in.
Even though leaks are very unlikely in Whonix, using socks (that does
not leak outside of Whonix) is still safer than using TCP4ClientEndpoint.
I am not sure what you mean by "unMessage always connects to another
hidden service". Do you mean it is inherently unlikely that some other
application accidentally connects to the same onion a.onion because
unMessage always uses new random onion domains?
> Are you sure unMessage's server is not only listening on localhost?
> For the gateway being able to reach it, it needs to listen on it's
> external network interface (eth0`s IP). If that does not help, do
> Whonix's Tor hidden web server example instructions work for you?
> Would be good to have that running first before trying other Tor
> hidden server software like unMessage.
That's it! I did follow those instructions, but unMessage actually
only listens on localhost. Do you think we should forward connections
from the gateway's IP to localhost, or directly listen on the
external interface? The former might require some action from the
user, while the latter might be easier because unMessage can
automatically do that when starting the server. I think we just need
a way to find out the address to use for both Whonix and
Qubes-Whonix.
Good question. I think listening on the external interface would be better.
This is currently being discussed with @ricochet-im also.
ricochet-im/ricochet#512 (comment)
In summary I was suggesting "listen on 127.0.0.1 by default, but allow
us to ship some file in some Whonix package that allows us to change the
default listener to an IP of our choice".
It's an important question which way we go. For uncooperative
applications or applications that get this feature too late, we are
using a uwt wrapper that then uses bindp to force the listen IP behind
the applications back without application support required. These
wrappers can be very problematic. (conflicts with apparmor, other
wrappers, uwt bugs, and m
Still wondering about the ideal way to do this. I think config files
that get configured by the distribution is the canonical way to do.
Cleaner. Not specific to Whonix. And standard conform - because other
server applications (such as web servers etc.) also support their
listener IP being configured.
|
|
On Fri, Feb 03, 2017 at 05:11:23AM -0800, Patrick Schleizer wrote:
I understood that. I was suggesting to use socks anyhow even in Whonix.
Because...
> What is not clear to me is the need of using a specific SOCKS port
> because unMessage always connects to another hidden service. From the
> link above: "Different Tor Hidden Services are automatically stream
> isolated". Can you clarify?
Connection to a.onion will use a different circuit than a connection to
b.onion.
Now, let's say unMessage would connect to a.onion and another
application such as Tor Browser or sdwdate would also connect to
a.onion. A while ago these would have been contaminated, i.e. used the
same circuit. There however won't be such a risk if different SocksPorts
are used. Then connection to a.onion through SocksPort X and connection
to a.onion through SocksPort Y would also be stream isolated.
The risk may be low, since Tor Browser now also uses per-tab isolation
as well as a random string after browser restarts for its socks user
name and sdwdate a dedicated separate Tor SocksPort.
Not using socks but regular system TCP has the disadvantage, that it
goes through Tor's TransPort. We at Whonix try to keep that fallback
port by default unused. Reserved for traffic generated by user's custom
installed applications only.
Ohh, right. Now I get it. I thought that using regular TCP to connect
to a hidden service would not go through Tor's TransPort.
Another reason might be simplicity. Using the same code in Whonix as
elsewhere. Using TCP4ClientEndpoint outside of Whonix would be bad, I
mean if that code unwanted kicks in.
Even though leaks are very unlikely in Whonix, using socks (that does
not leak outside of Whonix) is still safer than using TCP4ClientEndpoint.
You are completely right. That will indeed make the code simpler and
more secure.
I am not sure what you mean by "unMessage always connects to another
hidden service". Do you mean it is inherently unlikely that some other
application accidentally connects to the same onion a.onion because
unMessage always uses new random onion domains?
Oh no, those domains are fixed. I said that because I thought you did
not understand unMessage only uses hidden services and also because I
did not understand both regular and hidden service traffic go through
TransPort. You clarified that above. Sorry!
We need each user to have a fixed domain so that anyone can find them
to send conversation requests and reconnections (for existing
conversations).
Good question. I think listening on the external interface would be better.
This is currently being discussed with @ricochet-im also.
ricochet-im/ricochet#512 (comment)
In summary I was suggesting "listen on 127.0.0.1 by default, but allow
us to ship some file in some Whonix package that allows us to change the
default listener to an IP of our choice".
It's an important question which way we go. For uncooperative
applications or applications that get this feature too late, we are
using a uwt wrapper that then uses bindp to force the listen IP behind
the applications back without application support required. These
wrappers can be very problematic. (conflicts with apparmor, other
wrappers, uwt bugs, and m
Still wondering about the ideal way to do this. I think config files
that get configured by the distribution is the canonical way to do.
Cleaner. Not specific to Whonix. And standard conform - because other
server applications (such as web servers etc.) also support their
listener IP being configured.
Very interesting! I agree, I believe it will be fine to do so. Do you
think we can do that automatically, by checking if unMessage is
running on Whonix and use that file to set things up? Do you have an
application that already does that?
Thanks @adrelanos!
|
|
Glad we sorted that out! :)
Felipe Dau:
> Good question. I think listening on the external interface would be better.
>
> This is currently being discussed with @ricochet-im also.
>
> ricochet-im/ricochet#512 (comment)
>
> In summary I was suggesting "listen on 127.0.0.1 by default, but allow
> us to ship some file in some Whonix package that allows us to change the
> default listener to an IP of our choice".
>
> It's an important question which way we go. For uncooperative
> applications or applications that get this feature too late, we are
> using a uwt wrapper that then uses bindp to force the listen IP behind
> the applications back without application support required. These
> wrappers can be very problematic. (conflicts with apparmor, other
> wrappers, uwt bugs, and m
>
> Still wondering about the ideal way to do this. I think config files
> that get configured by the distribution is the canonical way to do.
> Cleaner. Not specific to Whonix. And standard conform - because other
> server applications (such as web servers etc.) also support their
> listener IP being configured.
Very interesting! I agree, I believe it will be fine to do so. Do you
think we can do that automatically, by checking if unMessage is
running on Whonix and use that file to set things up? Do you have an
application that already does that?
Maybe not great specifically check for Whonix - because well, it's
Whonix specific.
Always look for that config file - and when it exists - use it, that's
what I would suggest for a clean implementation.
- If you ship the config file by default - that is an issue for Whonix.
Then it is not easy for another package to take it over.
- If you do not ship the config file by default - that is not great, but
works.
- If you ship a config.d folder, that's ideal.
|
|
On Fri, Feb 03, 2017 at 11:14:21AM -0800, Patrick Schleizer wrote:
Maybe not great specifically check for Whonix - because well, it's
Whonix specific.
Always look for that config file - and when it exists - use it, that's
what I would suggest for a clean implementation.
- If you ship the config file by default - that is an issue for Whonix.
Then it is not easy for another package to take it over.
- If you do not ship the config file by default - that is not great, but
works.
- If you ship a config.d folder, that's ideal.
Hmm got it. I think we might be able to do that once we package it to
be installed by the package manager, but not by pip. I remember a
while ago we had issues to install files outside of the package dir
with pip. They did that because that breaks the isolation of
virtualenvs and there's nothing we can do about it (there are
certainly ways to hack the `setup.py` script, but that would be
wrong).
Here is what I think we can do:
- Provide `config.d` only with the system package, because then we can
save files anywhere.
- Use settings from `config.d` if found (that means it was installed
with the package manager or it was manually created). Use default
settings otherwise (which means it was installed with pip,
setuptools or regular python setup).
To implement the first option, I think we inevitably have to add some
logic to pick a config for Whonix and non-Whonix systems, the
difference is that it will not be in unMessage's main code itself but
in the package/setup code. Otherwise the configs must be provided by
the system and we implement just the second option instead of both.
We just need to decide that either us (unMessage devs) or you (Whonix
devs) will provide such settings. I do not like the idea of asking you
to do so because I do not think it would be feasible if every app
asked the OS devs to provide config files for them and also because
they would be provided to every single user by default, which means
that every workstation will have such files regardless if unMessage is
installed or not (I'm not sure and might be wrong about this). I think
that's is the app's devs responsibility to make sure it runs on the
systems they want to run it.
What do you think? Is there something I'm missing?
|
|
Felipe Dau:
On Fri, Feb 03, 2017 at 11:14:21AM -0800, Patrick Schleizer wrote:
> Maybe not great specifically check for Whonix - because well, it's
> Whonix specific.
>
> Always look for that config file - and when it exists - use it, that's
> what I would suggest for a clean implementation.
>
> - If you ship the config file by default - that is an issue for Whonix.
> Then it is not easy for another package to take it over.
> - If you do not ship the config file by default - that is not great, but
> works.
> - If you ship a config.d folder, that's ideal.
Hmm got it. I think we might be able to do that once we package it to
be installed by the package manager, but not by pip. I remember a
while ago we had issues to install files outside of the package dir
with pip. They did that because that breaks the isolation of
virtualenvs and there's nothing we can do about it (there are
certainly ways to hack the `setup.py` script, but that would be
wrong).
You don't need to ship a config.d folder. Just add to the main script
that it parses the /etc/unmessage.d folder. Pip should still work as
usual since it does not analyze the unMessage script or sandbox it.
Here is what I think we can do:
- Provide `config.d` only with the system package, because then we can
save files anywhere.
Sounds great.
- Use settings from `config.d` if found (that means it was installed
with the package manager or it was manually created). Use default
settings otherwise (which means it was installed with pip,
setuptools or regular python setup).
Also sounds great.
To implement the first option, I think we inevitably have to add some
logic to pick a config for Whonix and non-Whonix systems, the
difference is that it will not be in unMessage's main code itself but
in the package/setup code. Otherwise the configs must be provided by
the system and we implement just the second option instead of both.
We just need to decide that either us (unMessage devs) or you (Whonix
devs) will provide such settings. I do not like the idea of asking you
to do so because I do not think it would be feasible if every app
asked the OS devs to provide config files for them and also because
they would be provided to every single user by default, which means
that every workstation will have such files regardless if unMessage is
installed or not (I'm not sure and might be wrong about this). I think
that's is the app's devs responsibility to make sure it runs on the
systems they want to run it.
What do you think? Is there something I'm missing?
Thank you very much for raising these points. This could lead to
something very useful!
I guess the path isn't super clear and doesn't immediately find
consensus since we are inventing something new here? Both, server
applications reachable over onions such as unMessage, ricochet and
onionsahre as well as anonymity distributions such as Tails, Whonix are
relatively new developments in the *nix world.
Well, you are essentially providing a server application. Indeed a very
special server application that you do not have like being reachable
over clearnet. That's why you by default for most (non-Whonix) users
listen on localhost only.
Now, it's somewhat up to upstream (you) of server applications and linux
distributions such as Debian where they let their services listen by
default. As I observed, some server applications by default listen on
localhost only. Others such as web servers listen on all interfaces by
default.
You already opted for listening on localhost only.
Now, the configuration of a server that by default listens only on
localhost and should be reachable on other network interfaces or the
other way around is usually the task for the sysadmin.
We at Whonix opted to be somewhat a sysadmin ourselves. To oversimplify
it, Whonix is a giant sysadmin project. Or Debian derivative project.
Therefore I am not surprised we have to lightly preconfigure any server
application we install by default.
You have a very valid point. The brings me to an idea. Indeed, from the
perspective of the Whonix derivative it would be great to have a global
standard that all server applications are adhering. Such as a
configuration folder /etc/server-config.d.
Debian could ship a config file /etc/server-config.d/20_debian.conf.
```
listen_ip=127.0.0.1
```
Whonix could ship a config file /etc/server-config.d/30_whonix.conf.
```
listen_ip=
listen_interface=eth0
```
Then /etc/server-config.d/30_whonix.conf would win. And users would
still have plenty of room to override that setting.
And then it would be up to unMessage, ricochet and onionshare, to...:
- If /etc/server-config.d does not exist - ignore it - use your defaults
- listen on 127.0.0.1 only.
- If /etc/server-config.d exists - adhere it and parse it in lexical orrder.
If there was such an convention, this could make some thing simpler.
|
|
On Fri, Feb 03, 2017 at 12:57:10PM -0800, Patrick Schleizer wrote:
Thank you very much for raising these points. This could lead to
something very useful!
I guess the path isn't super clear and doesn't immediately find
consensus since we are inventing something new here? Both, server
applications reachable over onions such as unMessage, ricochet and
onionsahre as well as anonymity distributions such as Tails, Whonix are
relatively new developments in the *nix world.
Well, you are essentially providing a server application. Indeed a very
special server application that you do not have like being reachable
over clearnet. That's why you by default for most (non-Whonix) users
listen on localhost only.
Now, it's somewhat up to upstream (you) of server applications and linux
distributions such as Debian where they let their services listen by
default. As I observed, some server applications by default listen on
localhost only. Others such as web servers listen on all interfaces by
default.
You already opted for listening on localhost only.
Now, the configuration of a server that by default listens only on
localhost and should be reachable on other network interfaces or the
other way around is usually the task for the sysadmin.
We at Whonix opted to be somewhat a sysadmin ourselves. To oversimplify
it, Whonix is a giant sysadmin project. Or Debian derivative project.
Therefore I am not surprised we have to lightly preconfigure any server
application we install by default.
You have a very valid point. The brings me to an idea. Indeed, from the
perspective of the Whonix derivative it would be great to have a global
standard that all server applications are adhering. Such as a
configuration folder /etc/server-config.d.
Debian could ship a config file /etc/server-config.d/20_debian.conf.
```
listen_ip=127.0.0.1
```
Whonix could ship a config file /etc/server-config.d/30_whonix.conf.
```
listen_ip=
listen_interface=eth0
```
Then /etc/server-config.d/30_whonix.conf would win. And users would
still have plenty of room to override that setting.
And then it would be up to unMessage, ricochet and onionshare, to...:
- If /etc/server-config.d does not exist - ignore it - use your defaults
- listen on 127.0.0.1 only.
- If /etc/server-config.d exists - adhere it and parse it in lexical orrder.
If there was such an convention, this could make some thing simpler.
Wow you just made me realize Whonix is really a "giant sysadmin
project"!
That is a great idea and IMO that would be very useful. Unfortunately
I do not have enough experience to discuss about conventions of a
global standard :/ I can certainly see it working though. I'd like to
know what @rxcomm thinks about this.
Until we get to a consensus on this. It will be probably easier to do
what you suggested:
You don't need to ship a config.d folder. Just add to the main script
that it parses the /etc/unmessage.d folder. Pip should still work as
usual since it does not analyze the unMessage script or sandbox it.
Whonix would then provide the `config.d` dir or we could ask users to
manually do that until the standard is discussed/approved.
Thanks @adrelanos, I'm learning a lot with these posts!
|
This was referenced Feb 4, 2017
|
Felipe Dau:
That is a great idea and IMO that would be very useful. Unfortunately
I do not have enough experience to discuss about conventions of a
global standard :/ I can certainly see it working though.
I wouldn't worry too much about this. If Open Source enthusiasts that
are affected like we don't propose such conventions, I am pretty sure,
no one will. There also might be such a standard already, and we're not
aware of it.
Just now pointed the developers of ricochet and onionshare to this
discussion.
Looking forward to further comments, then I can write a draft for the
debian-devel mailing list, listen for more comments and then boldly post
it there.
|
|
On Sat, Feb 04, 2017 at 07:21:57AM -0800, Patrick Schleizer wrote:
I wouldn't worry too much about this. If Open Source enthusiasts that
are affected like we don't propose such conventions, I am pretty sure,
no one will. There also might be such a standard already, and we're not
aware of it.
You're right! And even if we don't get it right at first, I'm sure
we'll have help to improve it later :)
Just now pointed the developers of ricochet and onionshare to this
discussion.
Looking forward to further comments, then I can write a draft for the
debian-devel mailing list, listen for more comments and then boldly post
it there.
Thanks @adrelanos, me too!
|
|
It is sad to see that after two weeks, no commented :/ I would say that you should go ahead and post to the debian-devel mailing list, where someone may be able to contribute. What do you think? Thanks! |
|
Yes. Draft writing in progress.
I'll notify you once semi-finished. Suggestions on any step on the way here or on Whonix phabricator appreciated. |
|
On Sun, Feb 19, 2017 at 10:30:07AM -0800, Patrick Schleizer wrote:
Yes. Draft writing in progress.
`write draft for local listener standard on debian-devel`
https://phabricator.whonix.org/T635
I'll notify you once semi-finished. Suggestions on any step on the way here or on Whonix phabricator appreciated.
Thanks @adrelanos, I think it is a good idea to center the discussion
there!
|
|
Initial draft is done. Please have a look https://phabricator.whonix.org/T635 when you have a chance. |
|
Remaining tasks to make unMessage run on Whonix:
For now, launching will require:
Is there anything I am missing? |
|
Looks great! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
The
unmessage-clisupports the following options:--no-tor-socksprevents unMessage's own Tor process to open a SOCKS port--no-onion-serviceprevents unMessage's own Tor process to open an onion service portWhen these two options are provided, unMessage will not even start its own Tor process. It is assumed there is a Tor SOCKS port open (9054 by default, overridden with
-t), as well as there is a system's onion service mapping to the local server port (50000 by default, overriden with-l) and~/.config/unMessage/<peer>/tor/onion-service/hostnamecontains its domain name, to properly compose the peer's identity (name@domain.onion:port) in order to send requests.The approach currently works for a regular OS, with a Tor process using those configs. To use on Whonix, another option must also be used:
--no-tor-proxyprevents unMessage to use the Tor SOCKS port (or the one provided with-t) and connects to the address with a regular Twisted EndpointWhonix handles the onion service address and properly connects to the peer (I am not sure how stream-isolation works when connecting to an onion service). What is not currently working is its own server. Following the Hidden Services wiki page did not work and all connections to that workstation's server port result in
ConnectionRefusederrors (tested on Qubes-Whonix).It looks like the gateway is receiving the connection and the issue seems to be the mapping to the workstation that is not allowed.The text was updated successfully, but these errors were encountered: