Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Success rate in IRAN #6

Open
taesiri opened this issue Dec 12, 2019 · 6 comments
Open

Success rate in IRAN #6

taesiri opened this issue Dec 12, 2019 · 6 comments

Comments

@taesiri
Copy link

@taesiri taesiri commented Dec 12, 2019

Hi There!

First of all, thank you for your research and your project. I tried Geneva using all the strategies in this file, but I had pretty much zero success up until now. It was very surprising because other simpler tools like GreenTunnel and GoodbyDPI are working here.

Our ISPs are doing extensive DNS tampering and without some tools like dnscrypt-proxy, Geneva does not work at all and most of the time I see "Detected GFW Censorship, strategy failed" in the output.

Only these two strategies seem to work in HTTP mode, not HTTPS.

[TCP:flags:PA]-duplicate(tamper{TCP:dataofs:replace:10}(tamper{IP:ttl:replace:10},),)-|
[TCP:flags:PA]-duplicate(tamper{TCP:load:corrupt}(tamper{IP:ttl:replace:8},),)-|

When I do a curl http://facebook.com I can get some data from facebook's servers but doing curl https://facebook.com will result in curl: (35) OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to youtube.com:443

I also created a Docker to test all strategies at once. You can find my logs here. And here my Dockerfile:

FROM ubuntu:18.04

RUN apt-get update && \
    apt-get install -y \
    zip \
    unzip \
    python \
    tcpdump \
    python-crypto \
    build-essential \
    python-dev \
    libnetfilter-queue-dev \
    libffi-dev \
    libssl-dev \
    iptables \
    python3-pip \
    bridge-utils \
    net-tools \
    iptables \
    curl \
    wget \
    libpcap-dev && apt-get clean

ADD https://codeload.github.com/Kkevsterrr/geneva/zip/master /tmp/master.zip
RUN unzip /tmp/master.zip -d /&& rm /tmp/master.zip

RUN python3 -m pip install -r /geneva-master/requirements.txt

WORKDIR /geneva-master/
COPY check.py /geneva-master/check.py

ENV CONFIG "http://facebook.com 1"

ENTRYPOINT python3 /geneva-master/check.py $CONFIG 2>&1 | tee /tmp/log.txt

And here is my script

import os
import sys

# Add the path to the engine so we can import it
BASEPATH = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
sys.path.append(BASEPATH)

import engine

# Port to run the engine on
port = 80
# Strategy to use
strategy = "[TCP:flags:A]-duplicate(tamper{TCP:flags:replace:R}(tamper{TCP:chksum:corrupt},),)-| \/"

strategy_bank = ["[TCP:flags:PA]-duplicate(tamper{TCP:dataofs:replace:10}(tamper{TCP:chksum:corrupt},),)-|",
                "[TCP:flags:PA]-duplicate(tamper{TCP:dataofs:replace:10}(tamper{IP:ttl:replace:10},),)-|",
                "[TCP:flags:PA]-duplicate(tamper{TCP:dataofs:replace:10}(tamper{TCP:ack:corrupt},),)-|",
                "[TCP:flags:PA]-duplicate(tamper{TCP:options-wscale:corrupt}(tamper{TCP:dataofs:replace:8},),)-|",
                "[TCP:flags:PA]-duplicate(tamper{TCP:load:corrupt}(tamper{TCP:chksum:corrupt},),)-|",
                "[TCP:flags:PA]-duplicate(tamper{TCP:load:corrupt}(tamper{IP:ttl:replace:8},),)-|",
                "[TCP:flags:PA]-duplicate(tamper{TCP:load:corrupt}(tamper{TCP:ack:corrupt},),)-|",
                "[TCP:flags:S]-duplicate(,tamper{TCP:load:corrupt})-|",
                "[TCP:flags:PA]-duplicate(tamper{IP:len:replace:64},)-|",
                "[TCP:flags:A]-duplicate(,tamper{TCP:flags:replace:R}(tamper{TCP:chksum:corrupt},))-|",
                "[TCP:flags:A]-duplicate(,tamper{TCP:flags:replace:R}(tamper{IP:ttl:replace:10},))-|",
                "[TCP:flags:A]-duplicate(,tamper{TCP:options-md5header:corrupt}(tamper{TCP:flags:replace:R},))-|",
                "[TCP:flags:A]-duplicate(,tamper{TCP:flags:replace:RA}(tamper{TCP:chksum:corrupt},))-|",
                "[TCP:flags:A]-duplicate(,tamper{TCP:flags:replace:RA}(tamper{IP:ttl:replace:10},))-|",
                "[TCP:flags:A]-duplicate(,tamper{TCP:options-md5header:corrupt}(tamper{TCP:flags:replace:R},))-|",
                "[TCP:flags:A]-duplicate(,tamper{TCP:flags:replace:FRAPUEN}(tamper{TCP:chksum:corrupt},))-|",
                "[TCP:flags:A]-duplicate(,tamper{TCP:flags:replace:FREACN}(tamper{IP:ttl:replace:10},))-|",
                "[TCP:flags:A]-duplicate(,tamper{TCP:flags:replace:FRAPUN}(tamper{TCP:options-md5header:corrupt},))-|",
                "[TCP:flags:PA]-fragment{tcp:8:False}-|",
                "[TCP:flags:PA]-fragment{tcp:8:True}(,fragment{tcp:4:True})-|",
                "[TCP:flags:PA]-fragment{tcp:-1:True}-|",
                "[TCP:flags:PA]-duplicate(tamper{TCP:flags:replace:F}(tamper{IP:len:replace:78},),)-|",
                "[TCP:flags:S]-duplicate(tamper{TCP:flags:replace:SA},)-|",
                "[TCP:fla3gs:PA]-tamper{TCP:options-uto:corrupt}-|"]


website_to_test = sys.argv[1]
print("Website: " + sys.argv[1] + " - Using Strategy #" + sys.argv[2])

stragetgy_to_use = strategy_bank[int(sys.argv[2])]

# Create the engine in debug mode
with engine.Engine(port, stragetgy_to_use, log_level="debug") as eng:
    os.system("curl --insecure " + website_to_test  + " 2>&1 | tee /tmp/curl.txt")

and I run it like this:

docker run --rm -d --privileged --dns=MY-DNS-SERVER -v PATH-TO-LOG:/tmp/ -e CONFIG="http://facebook.com 5" geneva

With or without docker the results are the same, Not sure what is wrong here.

I probably have to read the paper first, but my question is how can I evolve existing strategies? Or how can I create new ones?

@Kkevsterrr

This comment has been minimized.

Copy link
Owner

@Kkevsterrr Kkevsterrr commented Dec 13, 2019

Hey @taesiri - awesome writeup, and awesome script! Thanks for reaching out, its great to see other community members starting to experiment with Geneva. A couple of thoughts for you.

Quick note on Docker - Geneva's engine relies on raw sockets to send its modified traffic, and we've had issues with docker on certain platforms sending those packets out of the container. As far as I know, Geneva in docker will likely will not work at all in NAT mode, and might have issues in bridged depending on your host. We've reported this to the docker team (see here: docker/for-mac#3275), though they didn't seem to do much with it. For best results, I would strongly encourage you to run Geneva outside of Docker.

HTTP vs HTTPS - In your script, I notice its hardcoded to port 80 - did you change it to port 443 when doing your HTTPS tests? The engine only modifies traffic it explicitly is allowed to access, so if its only capturing port 80, it won't help port 443 (HTTPS) at all. If you did change it, that's a very interesting result - that would suggest Iran's HTTP censorship operates on fundamentally different TCP stacks compared to their HTTPS censorship systems.

Why did so many strategies not work? The number of strategies is not as important as the number of strategy species. A species defines the underlying bug a strategy exploits in a censor - in the original Geneva paper, we released strategies in 4 species. (We have more coming too!) We did not include the same number of strategies for each species - we limited it just to what we thought was notable. One of the most diverse species we rederived was the TCB Teardown species - most of the strategies included fall under this species. However, if the underlying bug exploited by this species (in China's case, premature TCB state removal) is not present in Iran's systems, none of the strategies in that species will work.
Of the ones you tested, only one species works - TCB Desynchronization. It's a pretty powerful one we've found that works pretty decently for a broad number of censors around the world. If you're interested, I can write you additional TCB Desync strategies to experiment with to test the bounds of what works and what does not, or you can experiment with this yourself!

DNS: If you want to test against DNS censorship directly, you can swap DNS to use TCP mode and run strategies specifically for DNS over TCP (remember the port would need be 53!). Setting options use-vc in /etc/resolve.conf should be sufficient to routing DNS over TCP, though DNS over TLS or DNS over HTTPS will also get through the DNS censorship.

How can I create new strategies? We have not yet released the full genetic algorithm for Geneva yet (still cleaning it up, polishing, documenting, and making it easier to use), but we plan to do so soon! Once that is released, you'll be able to run Geneva yourself and evolve new strategies.

(Quick note, there is a typo in the trigger of the very last strategy in your script: TCP:fla3gs:PA instead of TCP:flags:PA. I doubt this strategy will work in Iran (I was pretty shocked that worked anywhere frankly), but at least we can test it.

@taesiri

This comment has been minimized.

Copy link
Author

@taesiri taesiri commented Dec 13, 2019

Freedom is contagious :-) I am sure your project will attract very people very soon.

Oh! Shoot! My mother always told me not to code at night 😅! thank you for pointing out my mistakes.

I fixed my previous mistakes and did all some tests again, and here is the most beautiful screenshot of the day:

Screen Shot 2019-12-13 at 10 30 19 PM

Here is a quick summary:

According to my results, A strategy that works for HTTP doesn't work for HTTPS.

What works with HTTP

Species Sub-Species Variant Genetic Code
TCB Desync Inc. Dataofs Small TTL [TCP:flags:PA]-duplicate(tamper{TCP:dataofs:replace:10}(tamper{IP:ttl:replace:10},),)-|
TCB Desync Inv. Payload Small TTL [TCP:flags:PA]-duplicate(tamper{TCP:load:corrupt}(tamper{IP:ttl:replace:8},),)-|

What works with HTTPS

Species Sub-Species Variant Genetic Code
TCB Desync Simple Payload syn [TCP:flags:S]-duplicate(,tamper{TCP:load:corrupt})-|
Segmentation With ack Offsets [TCP:flags:PA]-fragment{tcp:8:False}-| [TCP:flags:A]-tamper{TCP:seq:corrupt}-|
Segmentation Reassembly Offsets [TCP:flags:PA]-fragment{tcp:8:True}(,fragment{tcp:4:True})-|
Segmentation Simple In-Order [TCP:flags:PA]-fragment{tcp:-1:True}-|

Additional Notes:

  • Docker seems to work fine for me (Both on Windows and Docker on Ubuntu VM). I wrote a bash script on Ubuntu and confirm the consistency of my results (between Docker and Ubuntu VM).
  • Youtube seems not working, but I can't confirm it 100% because I had previous issues with it, even I had ShadowsocksR + DNS Over HTTPS.
  • I think I should gather a list from websites that are blocked from Iran and test against all of them to see what is working and what is not.
  • Updated log files are here

I am more than happy to test additional strategies and also write some automated tests for Geneva.

By the way, Is it okay to run multiple engines working on different ports with different strategies simultaneously?

Again, thank you so much for your awesome paper and project.

@xhdix

This comment has been minimized.

Copy link
Contributor

@xhdix xhdix commented Dec 13, 2019

@taesiri So update your blog post. :P

@taesiri

This comment has been minimized.

Copy link
Author

@taesiri taesiri commented Dec 14, 2019

@xhdix Done! 👍🏻😀

@ecthros

This comment has been minimized.

Copy link
Collaborator

@ecthros ecthros commented Dec 14, 2019

Thanks so much @taesiri !

By the way, if you're interested in trying to train some of your own strategies, please reach out to me at ghughey@umd.edu :) I'm not sure if @Kkevsterrr wants his email public so if you send me an email I'll be sure to add him to the thread.

We have quite a bit of exciting research we're working on beyond the scope of the strategies released here that we've been spending the last 6 months or so working on, and everything will be made public here eventually. However, a lot of cleanup has to happen first and we want to be sure of everything first before we make it all public, so we're keeping everything private and slowly releasing it.

@taesiri

This comment has been minimized.

Copy link
Author

@taesiri taesiri commented Dec 14, 2019

@ecthros Let's do this 😎

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.