Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"Stop sharing after first download" doesn't work with v3 onions #812

Closed
micahflee opened this issue Oct 23, 2018 · 35 comments · Fixed by #831 or #873
Closed

"Stop sharing after first download" doesn't work with v3 onions #812

micahflee opened this issue Oct 23, 2018 · 35 comments · Fixed by #831 or #873
Milestone

Comments

@micahflee
Copy link
Collaborator

Just while using OnionShare (built from the develop branch) to send a file to myself, I'm running into a weird issue. When I try sharing a 899kb jpg to myself, the server stops and the client thinks it's complete, but it only downloaded 250kb. I was able to reproduce this.

@micahflee micahflee added the bug label Oct 23, 2018
@micahflee micahflee added this to the 2.0 milestone Oct 23, 2018
@mig5
Copy link
Collaborator

mig5 commented Oct 23, 2018

I just reproduced this, but only when 'Close after first download' is checked.

When unchecked ('stay open' as we call it), the file downloads fully.

So somehow the shutdown is occurring too soon in the default mode - which is even more curious.

@mig5
Copy link
Collaborator

mig5 commented Oct 26, 2018

I haven't been able to solve this bug.

But here's what I've been able to find out, in case it helps you:

  1. I was able to confirm that Flask definitely sends all bytes of the file (until the chunk is b'') before closing. In theory, that should mean the entire file has made its way into the Tor network.
127.0.0.1 - - [26/Oct/2018 11:08:13] "GET /wool-paralysis/download HTTP/1.1" 200 -
100.0 KiB, 59.18%          [Oct 26 2018 11:08:13 AM] ShareModeWeb.b'\x83\xfb\x19\xe4\x00\x01\xb9L\x81\x9c\xefe\x9erE\x9e7x\xb1\xbf\x03\xe7;\xd0\x03\xc0\xff\x02\x9f\x80S\x1a\x90J\x07(\x00\x00\x00\x00IEND\xaeB`\x82PK\x01\x02\x14\x03\x14\x00\x00\x00\x08\x00b\x81JM\x00\x15\x12a\x98\xd5\x00\x00O\xe1\x00\x00\x12\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xa4\x81\x00\x00\x00\x00appdata-client.pngPK\x01\x02\x14\x03\x14\x00\x00\x00\x08\x00b\x81JMab\x9bL^\xcd\x01\x00j\xd2\x01\x00\x12\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xa4\x81\xc8\xd5\x00\x00appdata-server.pngPK\x05\x06\x00\x00\x00\x00\x02\x00\x02\x00\x80\x00\x00\x00V\xa3\x02\x00\x00\x00'
[snip]
169.0 KiB, 100.00%          [Oct 26 2018 11:08:13 AM] ShareModeWeb.b''

Stopped because download finished

(note the ShareModeWeb.b'' - I am printing to common.log(), the value of chunk in the ShareModeWeb class, as it reads out the file)

  1. I was able to confirm that if the share is forced to stay open (either in 'stay open' mode, or when I temporarily turned off the auto-close logic either way in the code), the file downloads fully.

  2. I confirm it occurs whether one file is shared or multiple (e.g zip or not) - so it's not specific to the Gzip compression work.

  3. I confirm that it occurs with CLI mode and GUI mode.

  4. It is not limited to Tor Browser (a torify curl -o download.zip of a share exhibited a corrupt .zip file too)

Somehow the .onion is closed too soon, despite all bytes having been 'sent' from Flask, and it seems to cut off the download.

None of the logic re: when to stop the share has changed (we still only consider a download 'finished' in the UI when the REQUEST_PROGRESS' bytes sent == the file size. Furthermore, the same issue occurs in CLI mode)

:/

@mig5
Copy link
Collaborator

mig5 commented Oct 26, 2018

OK, this is wild.

I reproduce it on my Onionshare dev VM even when I've checked out the master branch (v 1.3.1)

However, I don't reproduce it when using my 1.3.1 MacOS instance (even though in both cases I'm still downloading the file via a Whonix DispVM).

This suggests to me it's either:

  1. a bug affecting Linux users even on 1.3.1, or

  2. a bug in a dependency (since on my Linux machine, even though I checked out v1.3.1, I was still using the same dependencies/versions as I was when I had develop branch checked out).

I've experimented with downgrading Tor and Flask on the Linux machine, to no avail.

I'll next try and install 1.3.1 on a separate Linux VM (maybe the upstream Fedora package) to see if I reproduce it 'natively' (ie. with no poisoning of dependencies from my dev machine).

@mig5
Copy link
Collaborator

mig5 commented Oct 26, 2018

Hmm, still confused. I don't reproduce it on Fedora 28 with dnf install onionshare.

I reproduce it on Mac against develop branch (assumed it was my locally installed pip stuff though). But then I checked out master branch on Mac and ran it, and I don't reproduce it (works fine), so this is inconsistent with my Linux experience..

Also it's not v2/v3 onion specific.

@micahflee
Copy link
Collaborator Author

Wow thanks for all of the troubleshooting. I had no idea this would be such a weird and tricky bug. I'm starting to take a look too.

@micahflee
Copy link
Collaborator Author

I'm reproducing it when using Tor, but not reproducing it when using --local-only.

@mig5
Copy link
Collaborator

mig5 commented Oct 26, 2018

You know, I did wonder if the 'difference' in my tests was the Tor version... my test on MacOS against 1.3.1 branch (where I didn't reproduce it) was via dev_scripts, which meant I was using newer Tor via Tor Browser in the background... are you able to experiment with an older Tor and bundled or socket mode? Nevermind, this makes no sense (if I didn't reproduce it and was using newer Tor, then it's not a new Tor bug).

@micahflee
Copy link
Collaborator Author

Hmm I'll give it some more time soon but I'm actually about to board a flight where I won't have internet. Good luck for tonight though :)

@mig5
Copy link
Collaborator

mig5 commented Oct 26, 2018

I find that if I add a time.sleep(5) right at the start of the finally catch in onionshare/__init__.py, it's enough to fetch the whole file when using CLI mode (in my case, a 169 kb zip containing the two appdata images from the screenshots/ dir). Likewise, if I add a (blocking) time.sleep(5) in onionshare_gui/mode/share_mode/__init__.py just before it calls self.server_status.stop_server() after the download is complete.

In other words, we know that the web module sends the whole file and it's even OK for it to shutdown Flask immediately afterwards.

The core of the problem is that the onion service stops too suddenly and this causes the incomplete file client side.

Since nothing has changed in the way we start/stop the onion service, as far as I can tell.. this somehow points to some sort of buffering change in how onion services are served....? :s

I am not advocating the time.sleep() hack, since I assume it depends on the size of the file being shared as to when the onion service can safely shut down. Problematically, I don't know how else to measure this than how we were already doing it (basing on 'bytes sent' as reported by Flask)

I'm surely wrong about all this... but I get consistent results if I keep the onion service running just a bit longer (which also tallies with the fact that we knew 'stay open' mode also works). It all comes down to how long the onion service is available.

@mig5
Copy link
Collaborator

mig5 commented Oct 26, 2018

I'm now convinced it's a Tor version or Tor Browser version thing.

I just connected an old Tor Browser (7.5.5 - the version we shipped the tor binary from in our 1.3.1 release) as my client (for downloading a share from OnionShare).

I reproduced the issue when using Tor Browser 7.5.5 as a client, but with OnionShare itself using a newer ('bundled') Tor on Linux (0.3.4.8).

So, then I changed OnionShare itself to use the Tor Browser 7.5.5 running in the background as its tor provider ('automatic') mode, instead of bundled mode. Still using the same Tor Browser 7.5.5 as my client, I got a complete share! I couldn't reproduce the incomplete download issue.

Of course, I also got a v2 onion when using 7.5.5 as my connection method, whereas the previous attempt in bundled mode, due to newer Tor, was a v3 onion. But it's not v2 specific. I know this because I then switched OnionShare back to using bundled mode, but with legacy v2 onion mode turned on, and using this new v2 onion address, I reproduced the incomplete file issue again.

This is all against develop branch.

So what this tells me, is that in between Tor 0.3.3.7 (the version that was shipped in Tor Browser 7.5.5), and Tor 0.3.4.8 (my version on Debian Buster that I use in 'bundled' mode), the way data is sent to/from onion services to the end client (regardless of the client's Tor Browser version) has changed. It suggests that closing an ephemeral onion service (still via the same Stem library version, remember!) changes whether any remaining data flows through to the client. In newer Tor, it immediately is shut off...

I'll try and find upstream evidence that corroborates this change, but it will be hard to figure out what to look for..

EDIT argh, no, I cannot be right, because if I downgrade to Tor 0.9.8 on Buster (via jessie-backports repo I had lying around) I still reproduce it.. it would not make sense to reproduce in 0.2.9.8 and 0.3.4.8 but not 0.3.3.7... however, I definitely don't reproduce it once I switch back to 'automatic' mode using TBB 7.5.5 in the background... going slightly mad now, might have to give up. Hope I haven't caused too many rabbit holes for you to dig into.

One thing I know for sure, is that adding a sleep before stopping the .onion in either CLI or GUI mode, avoids the issue..

@micahflee
Copy link
Collaborator Author

I've been distracted for a few weeks, but I'm finally looking at this one again. Ugh, this is a really annoying, but critical, issue.

So basically, it appears that with some versions of Tor, as soon as the OnionShare side closes the ephemeral onion service, somehow the Tor network closes it on the Tor Browser as well, even if the download isn't complete.

The hack where we time.sleep(5) before shutting down the onion does indeed work, but only if we sleep long enough. I confirmed that, for sharing a 8mb folder (7.2mb zip file), if I sleep for 2 seconds, it downloads 167kb before quitting. If I sleep for 5 seconds, it downloads 1.1mb before quitting.

And the issue with relying on that hack is that we have no way of learning how long we should sleep. It completely depends on the speed of the data across the Tor network. Like, we could try to make an educated guess maybe based on how long it took to send 100% of the bytes, but if our guess is wrong (like, maybe one of the Tor nodes gets extra congested after OnionShare has sent the last byte) then we might not wait like enough, and the onion connection will shut down early.

@mig5
Copy link
Collaborator

mig5 commented Nov 18, 2018

Yep I completely agree. I think fixing it in OnionShare would be unreliable.

I tried trawling Tor core release notes and commits to try and pinpoint the cause but didn't succeed.

Is there anyone there you might be able to speak to who might know more or recognise this behaviour as being attributed to a change in core? If it's a regression and gets fixed, at least we could document a range of Tor versions that might end up being known to be a problem for OnionShare..

@micahflee
Copy link
Collaborator Author

So I think the best way forward is to either write an email to the tor-dev listserve, or ask on IRC, and explain the situation as best we can. In preparation of writing that email, I decided to try to make a simple PoC of the bug, and now it's not acting like a bug...

Check out this simple python script: https://gist.github.com/micahflee/bade960e96d35007bc5c182a0ca61b56

You need Tor Browser in the background for stem to connect to the tor controller, and you need a system tor so you can run torify. It basically creates a web server on port 8080, and make it an ephemeral onion service. If the server receives any GET request, it responds with 5mb worth of A characters, then shuts down the web server and the onion service.

So for example, in one terminal, I start the server and wait for the onion service to start:

$ ./onion-bug.py 
Starting http server in separate thread, http://127.0.0.1:8080/
Starting onion service...
Onion service at http://hy46l5a4a234d376.onion/

In another terminal, I make a GET request to http://127.0.0.1:8080/ (not the onion service) to make sure it works locally:

$ curl http://127.0.0.1:8080/ > file1
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 5120k    0 5120k    0     0   384M      0 --:--:-- --:--:-- --:--:--  384M

Over in the first terminal, I see the request and it shuts down:

127.0.0.1 - - [18/Nov/2018 16:36:57] "GET / HTTP/1.1" 200 -
Shutting down http server

It downloaded the full 5mb and shut down the server.

Now here's trying again, but this time downloading over the onion service. I would expect it to hit this bug, but it doesn't seem too...

Back in the first terminal, I run the script again and wait for the onion service:

$ ./onion-bug.py 
Starting http server in separate thread, http://127.0.0.1:8080/
Starting onion service...
Onion service at http://td676gqieanp2qtr.onion/

In the second terminal, I download it over the onion service:

$ torify curl http://td676gqieanp2qtr.onion/ > file2
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 5120k    0 5120k    0     0   202k      0 --:--:--  0:00:25 --:--:--  239k

While it was still downloading, in the first terminal, the server already shut down, and in fact the whole program ended, so the ephemeral onion service must have stopped:

127.0.0.1 - - [18/Nov/2018 16:37:49] "GET / HTTP/1.1" 200 -
Shutting down http server

Yet in the second terminal, it kept downloading until the full 5mb was received.

So, I didn't reproduce it...

I'm using Tor Browser 8.0.3, which comes with tor 0.3.4.8.

I just changed my develop branch OnionShare settings to connect automatically to Tor Browser's tor. Now, I'm sharing my whole onionshare source code directory which is about 8mb.

$ ./dev_scripts/onionshare --debug ../onionshare
OnionShare 2.0.dev | https://onionshare.org/
[Nov 18 2018 16:54:38] Web.__init__: is_gui=False, mode=share
[Nov 18 2018 16:54:38] ShareModeWeb.__init__
[Nov 18 2018 16:54:38] Onion.__init__
[Nov 18 2018 16:54:38] Onion.connect
[Nov 18 2018 16:54:39] OnionShare.__init__
[Nov 18 2018 16:54:39] OnionShare.set_stealth: stealth=False
[Nov 18 2018 16:54:39] OnionShare.start_onion_service
[Nov 18 2018 16:54:39] Onion.start_onion_service
Setting up onion service on port 17636.
[Nov 18 2018 16:54:39] Onion.start_onion_service: key_type=ED25519-V3
[Nov 18 2018 16:54:43] Settings.save
[Nov 18 2018 16:54:43] Settings.save: Settings saved in /home/user/.config/onionshare/onionshare.json
Compressing files.
[Nov 18 2018 16:54:43] ShareModeWeb.set_file_info
[Nov 18 2018 16:54:43] Web.start: port=17636, stay_open=False, public_mode=False, persistent_slug=
[Nov 18 2018 16:54:43] Web.generate_slug: persistent_slug=
[Nov 18 2018 16:54:43] Web.generate_slug: built random slug: "hypnotic-perfected"
 * Running on http://127.0.0.1:17636/ (Press CTRL+C to quit)

Give this address to the recipient:
http://ogg67eun6lu3jn2qacenazo2loomuazwxvfpynkxswhcwthh7uwpwxad.onion/hypnotic-perfected

Press Ctrl+C to stop the server

When I load it in Tor Browser, it says the size is 8.1mb, and the compressed zip file size is 7.3mb. Then I download it, and shortly after the OnionShare process stops, the download in Tor Browser stops at 5.5mb, corrupted. Here's the rest of the terminal output:

[Nov 18 2018 16:55:56] Web.check_slug_candidate: slug_candidate=hypnotic-perfected
127.0.0.1 - - [18/Nov/2018 16:55:56] "GET /hypnotic-perfected HTTP/1.1" 200 -
127.0.0.1 - - [18/Nov/2018 16:55:58] "GET /static/css/style.css HTTP/1.1" 200 -
127.0.0.1 - - [18/Nov/2018 16:55:58] "GET /static/js/send.js HTTP/1.1" 200 -
127.0.0.1 - - [18/Nov/2018 16:55:59] "GET /static/img/logo.png HTTP/1.1" 200 -
127.0.0.1 - - [18/Nov/2018 16:55:59] "GET /static/img/web_folder.png HTTP/1.1" 200 -
127.0.0.1 - - [18/Nov/2018 16:56:00] "GET /static/img/favicon.ico HTTP/1.1" 200 -
[Nov 18 2018 16:56:23] Web.check_slug_candidate: slug_candidate=hypnotic-perfected
127.0.0.1 - - [18/Nov/2018 16:56:23] "GET /hypnotic-perfected/download HTTP/1.1" 200 -
7.3 MiB, 100.00%            
Stopped because download finished
[Nov 18 2018 16:56:43] OnionShare.cleanup
[Nov 18 2018 16:56:43] Onion.cleanup
[Nov 18 2018 16:56:48] Onion.cleanup: trying to remove onion ogg67eun6lu3jn2qacenazo2loomuazwxvfpynkxswhcwthh7uwpwxad

So, how come I can reproduce it OnionShare with tor 0.3.4.8, but I can't reproduce it in this little script with the same Tor version?

OnionShare is using flask, and that script with was http.server. Hmm, the develop branch of OnionShare generates its own key, but the script lets stem generate it...

@mig5
Copy link
Collaborator

mig5 commented Nov 19, 2018

OK weird..

Do you at least see the same as I did if you download the old Tor Browser 7.5.5 and use that as your Tor connection method in OnionShare - do you get a complete download? Because if it's an OnionShare-specific issue then I'm still confused how I succeeded with the older Tor Browser (older Tor version for the connection)

@micahflee
Copy link
Collaborator Author

I just reproduced this bug from the v3-revisited branch, for PR #806. So it's not the onion key.

Let me try with Tor Browser 7.5.5 next.

@micahflee
Copy link
Collaborator Author

So it's kind of hard to find old releases of Tor Browser, but I found them on their github mirror: https://github.com/TheTorProject/gettorbrowser/releases

They don't have Tor Browser 7.5.5, but I did try it with Tor Browser 7.5.3, and you're totally right. I don't reproduce the error in that version of Tor...

So how do we proceed? OnionShare is kind of a beast of a project now. I think the best way forward is to reproduce the bug, using the current version of Tor, in a small script like that gist. If we can reproduce it, then at least we can properly ask for help.

@mig5
Copy link
Collaborator

mig5 commented Nov 19, 2018

Sorry I should've clarified I fetched my old TBB from https://archive.torproject.org/tor-package-archive/torbrowser/

I agree. I will look at your script and try and reproduce (probably not today). It would be interesting if Flask somehow has something to do with it but can't imagine how since it works without Tor.

@mig5
Copy link
Collaborator

mig5 commented Nov 21, 2018

Just a note that I had the same experience as you with that PoC. Could not reproduce the issue. Very frustrating.

I've tried to re-implement it in Flask in case that's the difference, but my Python-fu is just not as strong.

@micahflee
Copy link
Collaborator Author

I made a branch, 812_troubleshooting, where I'm ripping out almost all of the un-needed logic to reproduce this bug. I can still rip out more and make it even simpler, but so far it only supports CLI mode, only share mode, only connecting using Tor Browser, no longer has a slug. And it reproduced the bug. I'll keep trying to make it even simpler, but thought I'd share the progress.

@micahflee
Copy link
Collaborator Author

I've reproduced the bug in this gist!

It turns out, when I start the onion service like this, it works fine:

res_nobug = c.create_ephemeral_hidden_service(
    { 80: 8080 },
    await_publication=True
)

But when I start the onion service like this (which incidentally uses a v3 onion server) it triggers the bug:

res_bug = c.create_ephemeral_hidden_service(
    { 80: 8080 },
    await_publication=True,
    key_type="NEW",
    key_content="ED25519-V3"
)

It's consistent, every time. So, I think the bug is either in stem's create_ephemeral_hidden_service and how it handles the key_type and key_content fields, or it's in tor and how it handles the ADD_ONION command.

@micahflee
Copy link
Collaborator Author

Ok, I've updated the gist. Now it runs create_ephemeral_hidden_service in 3 different ways, each way passing in different values for key_type and key_content:

if sys.argv[1] == "1":
    key_type = "NEW"
    key_content = "RSA1024"
elif sys.argv[1] == "2":
    key_type = "NEW"
    key_content = "ED25519-V3"
elif sys.argv[1] == "3":
    key_type = "ED25519-V3"
    key_content = generate_v3_private_key()[0]

For example, to try the key_type='NEW', key_content='RSA1024' onion service, run python3 onion-bug.py 1:

$ python3 onion-bug.py 1
http service: http://127.0.0.1:8080/

starting onion service with: key_type='NEW', key_content='RSA1024'
http://ek46epzgpqu6uqsk.onion/

Then, in another terminal, I run torify curl http://ek46epzgpqu6uqsk.onion/ > out1, and when it finishes see how big out1 is. It should be exactly 1048576 bytes, or 1mb. Meanwhile, in the first terminal, the http server will have already stopped, and the script finished running:

127.0.0.1 - - [24/Nov/2018 19:59:45] "GET / HTTP/1.1" 200 -
shutting down http server

Using Tor Browser 8.0.3, with Tor 0.3.4.8, I've confirmed that these settings don't reproduce the bug:

key_type = "NEW"
key_content = "RSA1024"

And these ones do:

key_type = "NEW"
key_content = "ED25519-V3"
key_type = "ED25519-V3"
key_content = generate_v3_private_key()[0]

With Tor Browser 7.5.5, with Tor 0.3.2.10, making a new RSA1024 key works, but both of the ED25519-V3 options fail with this exception:

$ ./onion-bug.py 2
http service: http://127.0.0.1:8080/

starting onion service with: key_type='NEW', key_content='ED25519-V3'
Traceback (most recent call last):
  File "./onion-bug.py", line 154, in <module>
    main()
  File "./onion-bug.py", line 138, in main
    key_content=key_content
  File "/usr/lib/python3/dist-packages/stem/control.py", line 2937, in create_ephemeral_hidden_service
    stem.response.convert('ADD_ONION', response)
  File "/usr/lib/python3/dist-packages/stem/response/__init__.py", line 124, in convert
    message._parse_message(**kwargs)
  File "/usr/lib/python3/dist-packages/stem/response/add_onion.py", line 31, in _parse_message
    raise stem.ProtocolError("ADD_ONION response didn't have an OK status: %s" % self)
stem.ProtocolError: ADD_ONION response didn't have an OK status: Invalid key type

Which makes sense, because Tor 0.3.2.10 doesn't support ephemeral onion services.

So, it really seems like this is either a v3 onion service issue, or possibly a stem issue related to v3 onion services.

@micahflee
Copy link
Collaborator Author

Tor Browser 8.0 comes with Tor 0.3.3.9, and it has the same issue. v3 onion services fail.

Tor Browser 7.5.6 comes with Tor 0.3.3.7, it has the same issue.

Tor Browser 7.5.1 comes with Tor 0.3.2.10, and like with Tor Browser 7.5.5, it doesn't support ephemeral v3 onion services.

@micahflee
Copy link
Collaborator Author

I've tested with stem 1.6.0 and stem 1.7.0, but there's no difference.

@micahflee
Copy link
Collaborator Author

I updated the script to reproduce the issue again -- this time you just pass either v2 or v3 in as an arg, and it creates that type of ephemeral onion service.

I also just wrote a detailed email to the tor-dev list that explains the problem, shows the script that reproduces the bug, and asks for help in solving it: https://lists.torproject.org/pipermail/tor-dev/2018-November/013556.html

@mig5
Copy link
Collaborator

mig5 commented Nov 25, 2018

Huh! Ok. I was sure I reproduced this with v2 per #812 (comment) but maybe I was just going cross-eyed by then.

@micahflee
Copy link
Collaborator Author

Yup, and I just tested using Tor 0.3.3.9 (the version that comes in the Ubuntu 18.10 tor package) and also Tor 0.3.4.8, which comes with Tor Browser 8.0.3. With both those versions of Tor, v2 onion services work fine in OnionShare.

So, while we're waiting to see if anyone from the tor-dev list can help, I think we should maybe try to finish and merge #806, to simplify the onion service code.

And then I think we need to make a decision. There's a decent chance that this is an upstream bug, and it needs an upstream patch before it's fixed. If this is the case, should we:

  • Not include v3 onion services in the 2.0 release
  • Include them in v3, but:
    • Default to v2 onion services so we can have "Stop sharing after first download" on by default
    • When v3 onion services are checked, disable "Stop sharing after first download" option

@micahflee micahflee changed the title Share mode isn't completing shares in develop branch "Stop sharing after first download" doesn't work with v3 onions Nov 25, 2018
@asn-d6
Copy link

asn-d6 commented Nov 26, 2018

Hey hey, thanks for pointing this out. This is now on our radar. We will try to look into this ASAP and get back to you. Give us a few days tho.

@dgoulet-tor
Copy link

I've responded on the tor-dev@ email thread! The underlying issue in tor is now here:
https://trac.torproject.org/projects/tor/ticket/28619

@micahflee
Copy link
Collaborator Author

Thanks for your detailed response @dgoulet-tor! And thanks @asn-d6, a few days is definitely fine. I'm so glad we finally understand the underlying cause of this bug.

Since this is a tor issue, and its milestone is tor 0.3.5.x, I wonder how OnionShare should handle it.

If the tor version is less than 0.3.5, OnionShare could disable v3 onion support, and if it's newer, it could enable it. And depending on tor's release schedule, we could maybe wait until there's at least a tor 0.3.5rc release to make the OnionShare 2.0 release, so that at least we can ship tor 0.3.5rc with the Windows and Mac versions of OnionShare. And in the meantime Linux users could install tor from TPO's repo, or run the beta Tor Browser, to get a new enough tor if they want v3 onion services.

@unkaktus
Copy link

@micahflee I guess tor may implement (as part of the ticket) some GETINFO key that would indicate that RP circuits are not being closed on service removal. It should be much more straightforward than dealing with tor versions.

@micahflee
Copy link
Collaborator Author

@nogoegst this is the PR that fixes the upstream tor bug: torproject/tor#546

I believe OnionShare will have to wait until that PR gets merged and makes it into a new tor release (version 0.3.5). Then, OnionShare can create v3 onion services using the tor 0.3.5 binary, and those onion services shouldn't have this problem. So my idea is to only support v3 onion services is it's using tor 0.3.5+.

Am I misunderstanding how this bug fix will work?

@unkaktus
Copy link

unkaktus commented Dec 5, 2018

@micahflee all I am just saying is that in case this bugfix gets backported to some version below 0.3.5 it will be easier to detect if there was a flag. It's seems not to be an issue here, version check should suffice.

@micahflee
Copy link
Collaborator Author

micahflee commented Dec 6, 2018

I just added Tor's nightly-master debian repository and installed tor 0.4.0.0-alpha-dev-20181206T011118Z-1~bionic+1, which includes the upstream bugfix. I can confirm that I can't reproduce this issue when using that version of tor!

(I also tested and I can't reproduce it yet in 0.3.5.5-alpha from the repo yet, but it looks like the commit was just merged upstream today, so they probably just haven't made another experimental release yet.)

So yeah, I think the way to resolve this is to disable v3 onion services unless tor is new enough.

@micahflee
Copy link
Collaborator Author

I'm keeping this issue open until there's a Tor Browser release that includes a version of Tor that fixes the v3 bug. When that happens I need to:

  • Update onionshare/onion.py to hardcode the version in self.supports_v3_onions
  • Make the Windows and Mac releases bundle with that version

@micahflee micahflee reopened this Dec 17, 2018
@micahflee
Copy link
Collaborator Author

Blocked waiting for a Tor Browser release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants