Investigation on Blocking of Reality in IRAN - Test Results

[Phoenix-999]

Before delving into the issue concerning the reality protocol, I would like to make a couple of notes.

• As we are aware, the Great Firewall (GFW) employs various techniques to detect and block proxy servers. This discussion specifically addresses recent events involving the blocking of the reality protocol in Iran, primarily by MCI ISP.

• The information provided below is our speculation and arises from several users experiencing similar test results, raising suspicions about GFW's techniques and methods for identifying the reality protocol.

• Additionally, this issue is a continuation of the previous conversation regarding the blocking of reality. If you are new to this topic, I recommend reading the previous issues, which can be found at the following links:

Link 1
Link 2
Link 3
Link 4

Now, let’s delve into the matter.

Over the last five months, most Iranian users have encountered challenging situations, given that the reality protocol was the only one functioning seamlessly in Iran. Most other protocols were easily detected by the Great Firewall (GFW). It's a challenging task but still a viable choice.

Now, in the last 10 to 15 days, we are experiencing new waves of blocking Reality, which seem a bit different and more aggressive compared to the blocking waves of the past couple of months. We recently observed something that has been discussed before and rejected by many. Still, we believe that GFW, operating under MCI command, might have found a way to block reality by analyzing heavy traffic. We are not 100% sure, but our test results strongly suggest that these new waves of blocking are due to traffic monitoring by GFW. This is how we think GFW might find reality.

Test Conducted:

• We created two fresh VPS servers, correctly set up with all security measures in place.
• Both servers had clean IP addresses in the same range, and SSH was done through VPN proxy to provide extra security.
• In both servers, the reality protocol was created using the same port (443) and the same whitelisted SNI.
•The only difference was that VPS number one was shared with only 5 to 7 people with low to medium traffic, while VPS number 2 was shared by more than 200 people, and heavy traffic was directed to it in a short period.

After about 2 hours, VPS number 2 was suddenly disconnected, and the CPU usage spiked to 100%, causing the server to temporarily disconnect. It then returned to normal percentage, but the IP was subsequently blocked by GFW.
The screenshot below illustrates exactly what we experienced while using the reality protocol with heavy traffic. Of course, we considered the possibility of it being accidental, so we tried it multiple times with different users, and the same thing happened consistently.

Although VPS number one survived for about two days, after reaching a certain traffic point, we realized it was also blocked by GFW. Additionally, we had VPS number 03, which we subjected to very low traffic with one to two users. After a week, it was still running and had avoided detection by GFW.
It's essential to mention that we took careful measures to implement security, blocking all domestic IPs and domains on the servers and bypassing them through VPN client applications. We used IP protection, random DNS switching, fragmentation, and many other techniques in our X-ray configuration to see if we could escape detection. However, every time the traffic reached a certain point within a specific period, GFW detected and blocked it.

After IPs are blocked by GFW, we can still conduct the SSH connection, as evidenced in the screenshots below. Additionally, even after MCI blocks reality, Shatel ISP follows suit; however, reality continues to function in some other ISPs. Once again, the above is our speculation, based on the same test results conducted by different users in various regions of the country.

While not certain, we offer some suggestions that might help reality withstand the GFW's new measures. Please forgive us in advance, as we might not be as technically proficient as many of the great people here. Consider utilizing advanced obfuscation techniques such as:

⚪ Decoy Traffic Ratio
⚪ Dummy Traffic Ratio
⚪ Traffic Padding
⚪ Adaptive Behavior
⚪ Dynamic Switching
⚪ Dynamic Switching Interval
⚪ Fingerprint Spoofing

We would be immensely grateful for any help, guidance, and suggestions from any of you.

🇮🇷 To all my fellow Iranians, we would greatly appreciate it if anyone could conduct the same test and share their results with us here for more clarity

@RPRX, we are eagerly and patiently awaiting your thoughts on this matter.

^{"Everything that has a beginning has an end. I see the end coming. I see the darkness spreading. I see death. And you are all that stands in his way."}

[Havard20009]

Nice job
Thank you

[MrVb0]

👍
@RPRX We look forward to your valuable comments. You always provide the best solutions. We appreciate you. ❤️

[irgfw]

Both servers had clean IP addresses...

The problem is that we couldn't know if that's correct!
Iran's GFW has a graylist containing many IP ranges from a year ago!

Did you have that IP address you stated clean from a year ago and did not implement obvious protocols like plain Wireguard, Shadowsocks, OpenVPN, and/or many other things other than Reality-tcp-vision ?

Before we buy a VPS IP address, maybe someone had Shadowsocks set up on the server! That would just put that IP address on the graylist.(or any other ancient and not safe proxies.)

And this is not solely just on Reality. if you set up a simple TLS proxy like vless-tcp-tls with flow=vision, it will be blocked if your IP address is on the graylist.

IRGFW has zoomed on TLS proxies now. all kinds of them. AND have a very large list of IP addresses (Gray List).

For example, we had many IP addresses that didn't have any traffic for 3 months on them but got blocked last week! and by traffic I mean the VPS server had no xray-core or panel or any VPN services...

---

By the way, the CPU usage report: It's interesting and others reported this! they are probably or likely Active-probes as we are investigating it. Did you check what was using the CPU? or strange IP addresses with many requests to the server?

[sambali9]

I think they are targeting the tls in tls features of the proxy flow (according to this paper)
In my experience different vps providers have different levels of censoring. For example IPs from microsoft azure and google cloud have the least censoring and interference whereas IPs from hetzner have a very high censoring and interference.

In the paper mentioned above if the GFW allows for more false positives they could detect more xtls-vision traffic and my theory is that IPs from hetzner are set to have a very high false positives and servers from GCP or azure are set to have very low false positives. So hetzner proxies are more likely to be detected and blocked rather than GCP or azure.

[Fangliding]

In fact, it is easy to easily block all connections of reality. Reality manifests as port forwarding (a relatively common service) to the outside. As long as GFW is willing to block all port forwarding, no reality connection can survive
I have a friend who works at the Chinese Academy of Sciences, and they are currently in this situation
In your country, it may not completely block all port forwarding, but rather judge based on the data traffic
This is no longer related to fingerprints, so unfortunately we have no way to deal with it

[gfw-report]

Hi @Phoenix-999 and @irgfw,

Your report and findings fascinating to us. We'd be happy to help with this concerning situation. We are a group of researchers that have experiences on revealing the censorship mechanisms, such as real-time traffic analysis and active probing by the GFW of China, in the past few years.

If you think it's a good idea, we'd be happy to help investigate this situation, together with Xray developers and other anti-censorship researchers. We'd like to establish a more private communication channel to exchange information, but we couldn't find your email addresses. Can you drop us an email? Our email address can be found on our Github profile page.

We can still use this thread for public communication for transparency and to let more people aware of the situation in Iran and our latest progress.

[gfw-report]

Both servers had clean IP addresses...

The problem is that we couldn't know if that's correct!
Iran's GFW has a graylist containing many IP ranges from a year ago!

Hi @irgfw , we agreed with you that we could not conclude that both servers have "clean" IP addresses without holding the IP addresses for a long long time; and there may be one thing to test:

Our understanding is that while @Phoenix-999's high-traffic volume server has been blocked, the low traffic-volumne server has not. @Phoenix-999 can now assign the low-traffic volume servers to more people to use. And if this server that hasn't been blocked for a while suddenly got blocked after getting high traffic volume, it shows evidence that traffic volume may indeed affect censors' blocking decisions.

We'd be happy to discuss more in detail and even help setup more controlled experiments.

[tobyxdd]

In fact, it is easy to easily block all connections of reality. Reality manifests as port forwarding (a relatively common service) to the outside. As long as GFW is willing to block all port forwarding, no reality connection can survive

But how do you detect port forwarding? One server forwarding (reverse proxying) another would look exactly like the original with no way for a third party to tell, or am I missing something?

[Fangliding]

In fact, it is easy to easily block all connections of reality. Reality manifests as port forwarding (a relatively common service) to the outside. As long as GFW is willing to block all port forwarding, no reality connection can survive

But how do you detect port forwarding? One server forwarding (reverse proxying) another would look exactly like the original with no way for a third party to tell, or am I missing something?

It's very easy
For example, if you are using the Hetzner server but your dest is apple.com, it is obvious that Hetzner does not provide network services for Apple
Other sites are similar, and there is evidence to suggest that Azure servers and Microsoft's CDN can also be distinguished, so don't have a lucky mentality

[sambali9]

overall I think Iran's gfw isn't really advanced , it's sh*t. all they can do is blocking as much IPs as possible from vps providers with high vpn traffic like hetzner. that's all they can do. their whole strategy is blocking most of these servers.

Reality on MCI is getting blocked on a large scale (having tested more than 10 reality servers with different configurations and SNIs) so GFW is advanced enough to block the Reality at least.

honestly I think reality is working fine. the issue is, so many Iranians were using speedtest.net as dest.(because it was the only site that worked fine with reality on some servers) and using this site as dest is very suspicious because when you do a speedtest it connects you to a sever within Iran , not outside Iran . so when the gfw sees the traffic of this site is going to a server for example in Germany, they can easily find out that sever is a vpn server.

www.speedtest.net is a website and anyone wanting to use it has to connect to cloudflare IPs outside of Iran, it doesn't matter if you test your speed with a server in Iran because you first connect to www.speedtest.net to get the server information and then connect to a different server with a different domain (most likely ending in ooklaserver.net) to test the speed.

[sambali9]

my question: did you have a sever that got blocked even though you(or the person who owned the IP before you) HAD NEVER used speedtest.net as dest on that server?

I used different SNIs (nixos.org, www.speedtest.net, my own domain etc..) all got blocked on MCI network

[MrVb0]

my question: did you have a sever that got blocked even though you(or the person who owned the IP before you) HAD NEVER used speedtest.net as dest on that server?

I used different SNIs (nixos.org, www.speedtest.net, my own domain etc..) all got blocked on MCI network

me too !
I tested different dest and sni on different servers, but the servers were blocked (within 2.3 days).

[sambali9]

did you use speedtest.net as dest on those servers? doesn't matter when , just tell if you have used it or not.

I used www.speedtest.net only for the server with www.speedtest.net SNI and used other servers with other dests relating to their respective SNIs. I have not used www.speedtest.net as an SNI or dest for other servers.

[irgfw]

Hi @Phoenix-999 and @irgfw,

Your report and findings fascinating to us. We'd be happy to help with this concerning situation. We are a group of researchers that have experiences on revealing the censorship mechanisms, such as real-time traffic analysis and active probing by the GFW of China, in the past few years.

If you think it's a good idea, we'd be happy to help investigate this situation, together with Xray developers and other anti-censorship researchers. We'd like to establish a more private communication channel to exchange information, but we couldn't find your email addresses. Can you drop us an email? Our email address can be found on our Github profile page.

We can still use this thread for public communication for transparency and to let more people aware of the situation in Iran and our latest progress.

Hi @gfw-report. great to hear from you. We are actively investigating new Iran's firewall behavior, especially in one of the operators named MCI (mci.ir)
You can see some of our works here: https://github.com/GFW-knocker
Also, I contacted you via email.

Hope we hear from you soon...

[shakibamoshiri]

Please do ask questions on this thread for configuration, etc
just share your test, there are other issues people asked but there is no thread for sharing tests

---

Guys do not zoom in to just X-Ray or Reality!

The targeted zone by MCI is not realty, it is any kind of SSL-VPN including

• OpenCoonect
• AnyConnect
• OpenVPN
• SSTP
• V2Ray if TLS is used
• X-Ray if TLS is used
• etc

which they have two key features in common

1. TLS
2. port 443

Here are what we know so far (TESTED)

• If you run WireShard it simply shows that TLS Handshake never completes (TCP RTO)
• a blocked IP while port 443 cannot be accessed the port 80 is fine
• since SSH no need for TLS it is fine even if the IP is blocked

how a server is detected (best guess)

• traffic pattern (VPNs traffic are close to 1-to-1 == symmetrical while normal sites are asymmetrical, 1-to-3, 1-to-10, etc)
• a website popularity (if it is a well-known, it is/should bot be blocked)

Clients (TESTED - Android)

Xray

OpenConnect

Client Pro

OpenVPN + obfs

SSTP

affected area (TESTED)

Those provinces protected more

• Kurdistan
• Tehran
• Zahedan

The Kurdistan MIC Phone numbers start with 0918--* and Ardabil starts with 0914-- . At the moment of this witting 0918- cannot access a server that is blocked while 0914 can access . So the issue/throttle differs from City to City.

[tobyxdd]

In fact, it is easy to easily block all connections of reality. Reality manifests as port forwarding (a relatively common service) to the outside. As long as GFW is willing to block all port forwarding, no reality connection can survive

But how do you detect port forwarding? One server forwarding (reverse proxying) another would look exactly like the original with no way for a third party to tell, or am I missing something?

It's very easy For example, if you are using the Hetzner server but your dest is apple.com, it is obvious that Hetzner does not provide network services for Apple Other sites are similar, and there is evidence to suggest that Azure servers and Microsoft's CDN can also be distinguished, so don't have a lucky mentality

It may be obvious to you in your specific example, but this requires someone to manually maintain a database of what IPs a domain should and shouldn't use for every single domain, and make sure it's always up to date no less. I seriously doubt this is the case.

Am I right in assuming that there isn't really an SNI whitelist in Iran, and the reason you guys use common domains like apple.com is just to look less suspicious? If random, niche domains (which certainly aren't in the database mentioned above, if it exists at all) are still allowed, has anyone tested the chances of getting blocked vs. using common domains like Apple & Microsoft?

[MrVb0]

It's very easy
For example, if you are using the Hetzner server but your dest is apple.com, it is obvious that Hetzner does not provide network services for Apple
Other sites are similar, and there is evidence to suggest that Azure servers and Microsoft's CDN can also be distinguished, so don't have a lucky mentality

me too ! I tested different dest and sni on different servers, but the servers were blocked (within 2.3 days).

did you use speedtest.net as dest on those servers? doesn't matter when , just tell if you have used it or not.

no , never

[MrVb0]

Does anyone know about rprx? There is no news of him for several days! I wish him health and well.

[its0ka]

"every time the traffic reached a certain point within a specific period, GFW detected and blocked it" why is this a secret? why can't you just tell the number?

[ashad765]

MCI is working perfectly fine in my area, so I can't test it myself. Can someone run a Reality server according to below and report results :

1. Use your own NEW and FRESH domain name (steal oneself and not steal others)
2. Do not use well-known vps providers like Hetzner, OVH, Linode, Leaseweb, Vultr, Microsoft, Amazon and ...
3. Do no use server in well-known countries like Germany, France, Netherland, UK and US

[The-Erf]

They are more stupid than this
In my opinion, irgfw uses a much simpler method for identification. They might have a list of domains restricted to specific IP ranges. For instance, they might whitelist apple.com only for certain IPs, meaning operations won't proceed if the domain is correct but the IP isn't from Apple's actual range.

The second theory involves a simple test. For example, your server might use SNI like apple.com. Initially, irgfw checks your server's SNI and IP via a client hello. Then, it performs a basic ICMP ping or a DNS request to a reputable DNS server like Cloudflare or Google to see the IP behind that SNI.

If irgfw can detect a discrepancy between the actual IP of the site you SNI and your server's IP through a simple ping or DNS request, it might be able to block it.

(Note: They are confident that a reality behind a CDN can't stay hidden because even with a CDN, two simultaneous users might get two different IPs.)

However, considering the expanding number of websites, it's unlikely they can restrict all sites to specific IP ranges. Yet, it's a possibility.

In summary, I believe irgfw employs a simple mechanism for relative identification: checking the real IP behind your SNI and blocking if the SNI IP doesn't match your server's IP. Providers like Hetzner, DigitalOcean, Linode, and others may have a higher chance of being blocked.

[ashad765]

The second theory involves a simple test. For example, your server might use SNI like apple.com. Initially, irgfw checks your server's SNI and IP via a Hello client. Then, it performs a basic ICMP ping or a DNS request to a reputable DNS server like Cloudflare or Google to see the IP behind that SNI

According to this post this is exactly what happened with MCI right now. Iran GFW resolve SNI in tls client hello and forward Reality handshake to real SNI IP.
It seems using your own domain is the only solution right now.

[ashad765]

Does anyone know about rprx? There is no news of him for several days! I wish him health and well.

I clearly remember that this happened last year too. RPRX was not active from September to the end of December.

[AmirReza2012]

The problem we are currently dealing with might not have much to do with Reality or cryptography in general anyways.

Back when MCI was still throttling upload speeds a few months ago I tried creating a new VPS and I loaded a speed test script on it.

From my browser when I used that script to test my download and upload speeds everything was fine, the upload speed was not limited. But when I created any sort of v2ray inbound (same IP, same/different SNI, TLS/no TLS, tested many protocols) and did a speed test through that VPN, the upload speeds would be limited.

It is almost as if they could identify which connections were made through xray-core and they could effectively throttle the upload speeds to near inexistence.

MTN started throttling upload speeds after MCI but the difference was if you used a "whitelisted" SNI like speedtest.net or fast.com you wouldn't face the upload speed bottleneck.

A little while ago there was this news about gas station networks getting hacked and after that the blocking of VPN servers by MTN and other operators rose to the same levels as MCI. Similar things had happened before with MCI as well.

These same limitations never existed for relay servers hosted inside Iran, only the servers situated outside of Iran ever got blocked by these operators; and looking at the price of internet traffic set by Iranian VPS providers, the financial incentive here is pretty clear.

It seems though as of today, the upload speed limitations are mostly lifted now (by landline ADSL/fiber providers like TCI, Shatel, etc. . There was no throttling on MTN and MCI anyways - More testing still needed)
But I also had my servers IP blocked well over 10 times today.

It would seem as though they are now confident that they can more reliably identify and block proxy servers and they don't need to impose speed limits anymore.

I'm not sure if this has to do with uTLS fingerprints, or perhaps the way xray-core establishes connections, or maybe the traffic behavior in general but it appears that Iran's GFW is not targeting the cryptographic features of the connections, rather the traffic behavior.

Do you think there is any way to verify? @RPRX

[sarinaesmailzadeh]

@AmirReza2012 I agree with you.

Today all my sing-box reality and x-ray reality were detected by GFW. They were detected exactly at the same time (9 am) and I checked with many internet providers in different locations. That wasn't an ongoing process, It was offline batch process.
It's look like they flag my server and register in block list with background process.

My hypothesis:
IRGFW has a machine-learning algorithm. The government bought data centers from Russia and put it in the MCI data center.They capture many feature from every request and response. But they just capture some feature of the servers:

1. IP servers plus port
2. volume of data
3. bandwidth of every servers
4. header (request and response)
5. size of request and response of every request

After one day the background machine learning will start and find outlier of IP address.
The next day @Phoenix-999 said they sent a special request to the server and determined if this was a fake website or not.
Then the next day, they add IP to the blacklist in next day.

What is the problem:
Most of the time extremely recommended to use a server only for 10 people, but when we introduced the server in a public channel group of people used this configuration and suddenly bandwidth of the server increased and IRGFW will detect the server.

[AmirReza2012]

I had considered some other possibilities as well.

I thought maybe they were using Iranian applications like Snapp, Bazar, MyMCI, Digikala, etc. to figure out the IP address of your server and flag it as a VPN server. They could run a server in say Finland for example, use these Iranian apps to send to request to that server and then see if the IP address of the request going out from your phone belongs to a VPS server.
This way even if you were using geosite and geoip to avoid Iranian VPS servers from seeing your servers true IP address, you would still expose your servers address to the Finland server that they used.

This seems like a pretty efficient tactic to me.

So to test this out I started using a different outgoing IP address in my servers for a few days. The IP address which I used to connect to the VPN was different than the IP address that my server used to connect to everything else.

This didn't seem to make any difference though and I got well over 10 different IP addresses from different ranges blocked after doing that.

Also I tried the following:
I added many IP addresses to my server and once every few hours I switched the IP address that was actively being used by changing the DNS records of my domain name.

I thought maybe if each IP address transmitted less traffic then it wouldn't be flagged and blocked by the GFW. But even IP addresses that I used for a very short time got blocked the morning after,

This might mean they don't necessarily need you to transmit a lot of traffic to/from your server.

[farhadsaberi]

We talk all about IP and it gets blocked. So why hasn't the Warp application been blocked? How does it work then? Wouldn't it be better to focus on Warp instead?

[AmirReza2012]

@farhadsaberi I suspect that is because CloudFlare has servers situated inside Iran.

And last time I checked you could make WireGuard, OpenVPN, AnyConnect, etc. connections to Iranian servers just fine.

So either WARP can successfully connect out of pure coincidence or they are intentionally not blocking it.

Either way I doubt this will be helpful for us, since they are currently not blocking CloudFlare WSS CDN connections but as we remember they have shown they are willing to go far enough as to block every single CF IP address possible.

[sarinaesmailzadeh]

Report:
I made a dynamic system and changed UUID, header, response, status code, and message response. I change all these items every 4 hours. After one day my server was blocked by some internet providers.
When I bought a server I ran a small web service with Apache and HTML CSS on an 80 port, I couldn't open it with MCI internet provider.

Thesis:
It wasn't an active probe, It was gray list algorithm.
active probe is an attack that GFW doing on your server and CPU usage of the server will be increased.
In the gray list Algorithm, some internet providers block access to your server IP, and even some hours of the day the government makes noise on some TCP UDP packages.

White list, Black List, Gray list:
IRGFW make three different IP lists:
White list: these IP never blocked and some government's VPNs work on these white list IP
Black List: Couldn't access these IPs with any internet provider.
Gray List: It is not an intelligent system, they search on blacklist IPs and block some IPs in the near range of them.

How can find Gray List:
Ran a small web service with Apache and fake html and checked it by MCI internet connection.
Unfortunately these server ip has ping but they are categorized in grays.

Do not use a direct method to connect gray list IPs, and use CDN to solve this problem.

[sarinaesmailzadeh]

MCI is sensitive to derivative on bandwidth

Today I executed my code on white IP, I used Vmess method and change UUID, header and response and ports every 4 hours. My bandwidth received to 100 MB per second. and Server send 60 GB data to clients in one day. MCI has detected the server and it is block. Other internet provider can access to the server. I have another server and I used 6GB per day and it still working.

Database updated: 2024-01-03 16:35:00

   eth0 since 2024-01-02

          rx:  69.19 GiB      tx:  65.56 GiB      total:  134.75 GiB

   monthly
                     rx      |     tx      |    total    |   avg. rate
     ------------------------+-------------+-------------+---------------
       2024-01     69.19 GiB |   65.56 GiB |  134.75 GiB |    4.98 Mbit/s
     ------------------------+-------------+-------------+---------------
     estimated    797.05 GiB |  755.29 GiB |    1.52 TiB |

   daily
                     rx      |     tx      |    total    |   avg. rate
     ------------------------+-------------+-------------+---------------
     yesterday      6.37 GiB |    6.41 GiB |   12.77 GiB |    1.27 Mbit/s
         today     62.82 GiB |   59.16 GiB |  121.98 GiB |   17.55 Mbit/s
     ------------------------+-------------+-------------+---------------
     estimated     90.92 GiB |   85.61 GiB |  176.53 GiB |

Future work:
Increase bandwidth day by day. we need to increase bandwidth by the below function.
https://en.wikipedia.org/wiki/File:Logistic-curve.svg

If you want share a config in public channel, Block the MCI connection.

My another server didn't detect by the MCI:

Database updated: 2024-01-03 17:20:00

   eth0 since 2023-12-30

          rx:  8.30 GiB      tx:  7.86 GiB      total:  16.16 GiB

   monthly
                     rx      |     tx      |    total    |   avg. rate
     ------------------------+-------------+-------------+---------------
       2023-12      3.24 GiB |    3.15 GiB |    6.39 GiB |   20.50 kbit/s
       2024-01      5.06 GiB |    4.71 GiB |    9.77 GiB |  356.73 kbit/s
     ------------------------+-------------+-------------+---------------
     estimated     57.57 GiB |   53.65 GiB |  111.23 GiB |

   daily
                     rx      |     tx      |    total    |   avg. rate
     ------------------------+-------------+-------------+---------------
     yesterday      1.96 GiB |    1.92 GiB |    3.87 GiB |  384.89 kbit/s
         today      1.47 GiB |    1.33 GiB |    2.81 GiB |  386.18 kbit/s
     ------------------------+-------------+-------------+---------------
     estimated      2.04 GiB |    1.85 GiB |    3.88 GiB |
     

[Phoenix-999]

@sarinaesmailzadeh

In simple terms, the GFW seems to follow specific patterns that may not make much sense to us individually. However, when these patterns are combined, they form a basis for filtering robots to determine whether a server should be blocked or not.

Whatever triggers the GFW to detect a server happens well before the proxy protocol is activated. Whether a VPS server has a new IP address or is an old server reactivated or previously deleted, the activation process initiates. The GFW starts inspecting right at that moment, as soon as the first SSH connection is established.

Upon this initial connection, information about the targeted VPS is collected and stored temporarily. This data is later used by the GFW to check if the VPS IP server is on the White List, Grey List, or previous Black List.

It's crucial to note that the Iranian GFW has been gathering information about proxy VPS servers detected and blocked in the past 15 to 17 months. They likely have a comprehensive list of whitelisted, grey-listed, and blacklisted IPs, along with SNI domain lists from the mass server blockages.

Another factor to consider is the presence of informers blending with the general public, collecting information on platforms like GitHub and Telegram. They share experiences and discuss solutions, contributing to the GFW's adaptive behavior in tweaking its filtering strategy.

Now armed with this information, the GFW can effectively use mathematical probability algorithms to inspect, detect, and block servers.

The second stage involves checking newly activated VPS by IP addresses.

⚫ If an IP is Black Listed due to its past use for VPNs or proxies and its history is not clean, the GFW swiftly blocks the server. This often results in VPS being blocked within 2 to 3 Hours.

🔘 For Grey Listed IPs with limited historical data, indicating the number of times they have been blocked in the past due to being used as a proxy server, the GFW places them on an observation list for about 4 days to a week. During this period, the GFW scrutinizes the traffic and monitors activities. If everything aligns with the WEB server guidelines, the VPS is allowed. Otherwise, if there's any unusual traffic, port activity, or user connections, the VPS will be blocked.

⚪ White Listed Servers fall into two categories: Old IPs with a history of more than a year or two, and those suggested by government or private entities are considered Pure white IPs, exempt from GFW observation.

New IPs without a history of the past 12 months undergo a review after about 20 days into the month of usage and If everything aligns with the WEB server guidelines and all is clean, they're added to the permanent whitelist (Please note this happens very rarely.).

Based on my tests using personal VPS servers from the past and new servers from different providers, it seems this could be the strategy behind the GFW's recent upgrade.

While we might have heard bits and pieces of this information, putting it together forms a more comprehensive understanding of the GFW's filtering mechanism.

However, these are all personal hypotheses regarding my test environments and have nothing to do with the group test we are conducting.

[sarinaesmailzadeh]

Report:
I have a gray IP, when I bought it was blocked by MCI. But it working on other internet providers. (number one)
In another experience, I had a white IP, with the same configuration. When I bought it was white and then MCI detected and blocked it.(number two)

Server number one: 196.20 GB and 19.51 Mbit/s used in one day

Database updated: 2024-01-05 01:30:00

   ens3 since 2024-01-01

          rx:  220.53 GiB      tx:  217.77 GiB      total:  438.30 GiB

   monthly
                     rx      |     tx      |    total    |   avg. rate
     ------------------------+-------------+-------------+---------------
       2023-12    415.41 MiB |  424.50 MiB |  839.91 MiB |    2.63 kbit/s
       2024-01    220.13 GiB |  217.36 GiB |  437.48 GiB |   10.71 Mbit/s
     ------------------------+-------------+-------------+---------------
     estimated      1.64 TiB |    1.62 TiB |    3.26 TiB |

   daily
                     rx      |     tx      |    total    |   avg. rate
     ------------------------+-------------+-------------+---------------
     yesterday     98.69 GiB |   97.51 GiB |  196.20 GiB |   19.51 Mbit/s
         today     14.63 GiB |   14.54 GiB |   29.17 GiB |   46.40 Mbit/s
     ------------------------+-------------+-------------+---------------
     estimated    234.01 GiB |  232.65 GiB |  466.66 GiB |
     

 Server number two: 3.40 GiB and 338.17 kbit/s in one day

     Database updated: 2024-01-05 05:00:00

   eth0 since 2024-01-02

          rx:  83.36 GiB      tx:  79.91 GiB      total:  163.27 GiB

   monthly
                     rx      |     tx      |    total    |   avg. rate
     ------------------------+-------------+-------------+---------------
       2024-01     83.36 GiB |   79.91 GiB |  163.27 GiB |    3.86 Mbit/s
     ------------------------+-------------+-------------+---------------
     estimated    614.06 GiB |  588.65 GiB |    1.17 TiB |

   daily
                     rx      |     tx      |    total    |   avg. rate
     ------------------------+-------------+-------------+---------------
     yesterday      1.76 GiB |    1.64 GiB |    3.40 GiB |  338.17 kbit/s
         today     11.22 MiB |    4.49 MiB |   15.71 MiB |    7.32 kbit/s
     ------------------------+-------------+-------------+---------------
     estimated     53.87 MiB |   21.54 MiB |   75.41 MiB |
     

Conclusion :
MCI has a different censorship mechanism. But whenever MCI detects the server, all other internet providers will block the IP too.

[Phoenix-999]

FYI

USENIX Security '23 - How the Great Firewall of China Detects and Blocks Fully Encrypted Traffic

[sarinaesmailzadeh]

After One month of investigation I have found a solution for the IRGFW problem:

1. They had a large database of foreign IP, domains, and average bandwidth used from Iran per day and month. They can detect your domain and IP that were behind the Cloudflare without any problem.
2. If overused the average of bandwidth, They will going to step 3.
3. In this case, they capture all bits and headers and source IP and destination IP
4. They had a machine learning algorithm and investigated on statistics of a series of bits.
5. If find any abnormal situation, they block it immediately on MCI.
6. They going to send a Chinese spider on IP and port. it is not released to the ssh connection. They gathering information from your server to build better version in future.
7. if your server is more overused than usual, they will DDOS attack on your server.As @Phoenix-999 mentioned.
8. after 3 days they will block your server on other internet providers.

Solution:

1. Fragment your TCP package into small packages, the statistical pattern will be different. and used a high average bandwidth website that used Cloudflare.
2. block your server to access from different countries such as China, Russia, and other dictators countries.

[irgfw]

After One month of investigation I have found a solution for the IRGFW problem:

Really? So I can delete my account? :)) (kidding)

1. They had a large database of foreign IP, domains, and average bandwidth used from Iran per day and month. They can detect your domain and IP that were behind the Cloudflare without any problem.

False. we have tested over 20 IP addresses that were blocked WITHOUT any traffic on them for the past 3 months.

2. If overused the average of bandwidth, They will going to step 3.

3. In this case, they capture all bits and headers and source IP and destination IP

4. They had a machine learning algorithm and investigated on statistics of a series of bits.

Machine learning algorithm? come on! we know many insiders in the firewall to TIC. they use simple hardware and firewalls like pfsense or so! there is no evidence of machine learning in Iran's GFW. They are a bunch of Basijis behind computers without proper knowledge of networking! that's why many many simple websites and services are blocked without any reason. (GFW has simple yet ridiculous rules of blocking)
I highly encourage you and either people to read this website accurately and completely:

1. https://internetabad.factnameh.com
2. https://internetabad.factnameh.com/fa/profiles/douran-software-technologies (look at the leaked images!)

5. If find any abnormal situation, they block it immediately on MCI.

What is an abnormal situation?

6. They going to send a Chinese spider on IP and port. it is not released to the ssh connection. They gathering information from your server to build better version in future.

Please first read this article: https://t.me/irgfw/13 .
There is no evidence of the Chinese helping Iran's GFW. The data you see are some brute-force and exploits from some countries to gain access to the server or misuse the server. that's not relevant. please don't mix them up.

7. if your server is more overused than usual, they will DDOS attack on your server.As @Phoenix-999 mentioned.

What defines overused? and what defines usual?
Active-probes are not DDoS (Please read: https://gfw.report/blog/gfw_shadowsocks). 4 months ago Iran was using probes heavily but after some upgrades in the last week, the presence of probes is going away and the system is using Passive methods to detect and block the suspicious VPN servers.

8. after 3 days they will block your server on other internet providers.

There are some patterns right now. we are testing it.

3or4 Hours / 3or4 Days / 2 Weeks.
If your VPN server could hold up for 4 hours, it's likely to be challenged in 4 Days. and if after 4 days it's not blocked, it's likely to go to 2 weeks. and if the server is not blocked after 2 weeks, it's likely to hold up for quite some time (the IP address is likely white and clean in the GFW database)

Solution:

1. Fragment your TCP package into small packages, the statistical pattern will be different. and used a high average bandwidth website that used Cloudflare.

Fragmentation is a good solution but it's not the final one. as discussed here: #2504

2. block your server to access from different countries such as China, Russia, and other dictators countries.

Good solution but be aware, for example, we are saying our server is an innocent webserver. a web server can be accessed from all around the world. so you should not just Iran-Access-Only the server but rather just block specific countries like China and Russia.

[realartin]

i guess it's better to read this article that is from @uoosef (https://github.com/uoosef) twitter post :
link : https://t.co/r4DZtNQPG7

How does the new filtering system of Hamrah Aval (a mobile operator) work?
This system consists of two components:
1- The active part, responsible for checking the first 1.2 to 17 kilobytes of each connection initiated outside the country. This system examines predefined signatures in the initial packets of each stream; for example, it checks if the first packet starts with 0x16 0x3, indicating a probable TLS type. It then looks for the SNI extension in this packet, which starts with 0x1 along with the packet length. After identifying the SNI, it checks whether the SNI is in its filtering hashtable. If the packet is not of the TLS type, the system looks for other signatures such as SSH and HTTP. Regarding HTTP, the system searches for the Host header. Previously, the system was case-sensitive and sensitive to extra spaces but has been updated to remove all spaces. It seems that this active signature checking is done on specialized ASIC processors due to their high processing load, but even with powerful processors, delays and increased ping occur. This is because in the afternoon, people return home, turn on their VPNs, causing the filtering system to become congested.

It's worth noting that the active part's operators differ from each other, each having their own bugs, indicating that the system is not entirely consistent.

2- The passive part:
Before the recent update, the DPI (Deep Packet Inspection) system was entirely active, and if you could deceive it, it wouldn't bother you anymore. However, with the latest update, Hamrah Aval randomly samples some connections of each person, capturing patterns of circumvention in a passive manner. These patterns include TLS in TLS, authentications, and headers of well-known VPN packets. For example, when using v2ray with vless, vless sends a small authentication packet for each connection before sending the main stream, ensuring that the client is legitimate. Additionally, when establishing a new VPN connection with another TLS connection, the passive filtering system looks for repeating patterns in the small packets with TLS or v2ray patterns. If found, it flags the IP addresses and domains, which are then reported to the filtering system every 8 hours and either throttled or completely blocked.

The solution:
The solution is to disrupt these patterns, making it challenging for the server to be filtered. Simply manipulating the codes of these patterns can prevent filtering. Multiplexing to reduce the number of connections and injecting random packets, especially at the beginning of each stream, can decrease the chances of being sampled. Additionally, injecting random packets into authentication packets, breaking them into several packets with different sizes and paddings, can help.

The effectiveness of filtering relies on the absence of an easy way to modify existing protocols and share the results of modifications among users. The purpose of the Bypass SDK is to simplify this process.

A final note regarding VPS:
When setting up a tunnel to Iran, it is advisable to have a noise generator on your server, sending requests to IPs other than your filtering server to avoid detection. Using an Iranian server for a tunnel should be considered a last resort, as it may contribute to strengthening filtering efforts.

translated by chat gpt 3.5

[realartin]

i invite @uoosef to this article about irgfw and if he decide to help xray community with his idea it would be very appreciated

[RPRX]

@realartin @uoosef 感谢你们，其实我们正计划在中国春节期间发布 Xray-core v1.9.0，为 Vision 加上 Seed 支持，它允许用户自定义 Vision 的一些 padding 参数等，并支持与其它传输方式比如 H2、gRPC、WS 等结合使用，到时你们可以测试一下

[realartin]

@realartin @uoosef 感谢你们，其实我们正计划在中国春节期间发布 Xray-core v1.9.0，为 Vision 加上 Seed 支持，它允许用户自定义 Vision 的一些 padding 参数等，并支持与其它传输方式比如 H2、gRPC、WS 等结合使用，到时你们可以测试一下

tnx for your great work sir , you save a lot of people from censorships ❤️🙏
i guess if you and @uoosef share your thoughts together the final results are going to be extraordinary

[karimi12]

Dears

I had two tests with a domain.

1. I used a domain on Cloudflare and then started downloading a static file from it. after three days and downloading less than 170 GB my domain was banned.

My hypothesis:

1. Iranian GFW don't use machine learning their goal this ban access to the internet. so this reason they ban every unknown server or domain (they don't care about the consequences)

2. Because VPN connections are a point to point. they can capture traffic and find the final IP or SNI because Xray sends all traffic to a server (in my test all of the traffic got from a site (point). my suggestion for this action create fake traffic and send it to a list of site

[irgfw]

@RPRX Good. That "may" help for this situation in Iran. as any combination of VLESS/VMESS/Trojan is blocked within 4 hours or 4 days or 2 weeks in Iran. There is something in these combinations that they could "possibly" fingerprint or DPI the packets of them.
BTW can we have some contact with you? we can share some private info about Iran's GFW for this matter and you could get more context about this problem. you can contact us at irgfw@proton.me .

[irgfw]

1. I used a domain on Cloudflare and then started downloading a static file from it. after three days and downloading less than 170 GB my domain was banned.

Yes. we can confirm this. many normal websites are blocked without any reason we tested this (downloading and refreshing from a normal webserver) and they were blocked after 4 days of constant traffic and downloading. (the VPSs were located at Hetzner/Linode/DigitalOcean/Vultr)

1. Iranian GFW don't use machine learning their goal this ban access to the internet. so this reason they ban every unknown server or domain (they don't care about the consequences)

Correct.

[RPRX]

net4people/bbs#327 (comment)

[RPRX]

BTW can we have some contact with you? we can share some private info about Iran's GFW for this matter and you could get more context about this problem. you can contact us at irgfw@proton.me .

我有个邮箱，但我基本上不看，我经常通过 telegram 和 @yuhan6665 聊天，你可以发给他，他会把这类信息转发给我

[Mahdi-zarei]

I had a rather interesting observation lately that I would like to share. I had a trojan and TUIC config running with a tunneled configuration and it has been running on the same IP for almost 9 10 months now.
A night ago the mentioned configuration nearly stopped working, and while ssh and ping were allowed, iperf and traceroute showed some bizarre behavior, for example in the iperf test, an initial rush of data was permitted from the Iran VPS to the foreign one, but it was quickly stopped, and even though the reverse iperf was working at nearly full speed, traceroute could only complete from the Iran side!
Now what had changed in the last few days? I had setup a very simple website on this same IP which served some really simple statistical data and I was directly accessing it! From what I have experienced the past few month and that my configuration did not get blocked by ANY of the blockage waves, I can strongly guess that they simply throttle or ban ANY TLS based traffic that doesn't comply to some of their parameters, could be initial packet sizes or the amount of data transferred, or count of TLS handshakes a certain client does to the server as normal websites usually do not need a lot of it ( I was refreshing my site a lot ).
So given this observation I believe simply mimicking websites is no longer a way to bypass the firewall as they tend to ban real websites too, maybe if we could extend the REALITY protocol so that at the beginning of a connection a certain amount of data (let's say 1MB) was requested by the client from the website we are using as the dest, so before we even begin acting as the proxy, we serve the dest website and this way the characteristics of the initial packets of the connection would practically be identical to that of the dest website, this could pose an immense amount of pressure on the firewall as analyzing an entire connection is practically almost impossible ( by making the amount of data served configurable at the client side, we could really make the task hard ).
Do you thing a method like this could prove useful @RPRX ?

[irgfw]

Thanks for your information. It certainly will help.

For everyone who has some test results and technical information about Iran's GFW, please email me at irgfw@proton.me

[mkh1364]

[Perlover]

Hi,

Don't you think that identifying the IP address where X-Ray is running or which is the actual public server (www.microsoft.com and etc) - can be easily determined by reverse IP address resolving?

That is, the GFW monitors TLS traffic. It checks first of all addresses with high traffic volume, and then does rozolving (PTR record, in Linux you can use dig -x IP command) and by reverse domain identify. For example, for real www.microsoft.com addresses we see a following pattern:

$ dig a microsoft.com

; <<>> DiG 9.18.18-0ubuntu0.22.04.2-Ubuntu <<>> a microsoft.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 10807
;; flags: qr rd ra; QUERY: 1, ANSWER: 5, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 65494
;; QUESTION SECTION:
;microsoft.com.			IN	A

;; ANSWER SECTION:
microsoft.com.		3030	IN	A	20.112.250.133
microsoft.com.		3030	IN	A	20.231.239.246
microsoft.com.		3030	IN	A	20.76.201.171
microsoft.com.		3030	IN	A	20.70.246.20
microsoft.com.		3030	IN	A	20.236.44.162

;; Query time: 4 msec
;; SERVER: 127.0.0.53#53(127.0.0.53) (UDP)
;; WHEN: Mon Mar 04 11:30:10 CET 2024
;; MSG SIZE  rcvd: 122

$ dig -x 20.112.250.133

; <<>> DiG 9.18.18-0ubuntu0.22.04.2-Ubuntu <<>> -x 20.112.250.133
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 51594
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 65494
;; QUESTION SECTION:
;133.250.112.20.in-addr.arpa.	IN	PTR

;; AUTHORITY SECTION:
250.112.20.in-addr.arpa. 218	IN	SOA	ns1-02.azure-dns.com. azuredns-hostmaster.microsoft.com. 1 3600 300 2419200 300

$ dig -x 20.231.239.246

; <<>> DiG 9.18.18-0ubuntu0.22.04.2-Ubuntu <<>> -x 20.231.239.246
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 43046
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 65494
;; QUESTION SECTION:
;246.239.231.20.in-addr.arpa.	IN	PTR

;; AUTHORITY SECTION:
239.231.20.in-addr.arpa. 84	IN	SOA	ns1-09.azure-dns.com. azuredns-hostmaster.microsoft.com. 1 3600 300 2419200 300

;; Query time: 8 msec
;; SERVER: 127.0.0.53#53(127.0.0.53) (UDP)
;; WHEN: Mon Mar 04 11:30:30 CET 2024
;; MSG SIZE  rcvd: 142

Here we can see the pattern that for IP addresses of real microsoft.com at least there is a pattern - SOA record has email (ns1-09@azure-dns.com) & primary DNS (azuredns-hostmaster.microsoft.com) from microsoft.com zone. Wherever you bought a VPS, it is hard to fake these SOA records by pointing them to microsoft.com servers as well. Often VPS providers, if they allow you to change the PTRs, they only allow you to change them to domain names that also resolve back to the same IP addresses. But in this example, there are no PTR records, but there is a SOA record for the entire reverse zone.

So, if your VPS responds only to port 443, pretending to be a public server www.microsoft.com, then if somebody set a simple task for GFW - find out if this IP really belongs to a pool of Microsoft servers, it will be easy to do it by reverse resolving, just by analyzing SOA and PTR records for the IP or reverse zone of the address block. To this method you can also add a whois query to the same address block.

UPDATE: example about www.speedtest.net:

$ dig a www.speedtest.net

www.speedtest.net.	289	IN	CNAME	www.speedtest.net.cdn.cloudflare.net.
www.speedtest.net.cdn.cloudflare.net. 289 IN A	104.18.202.232
www.speedtest.net.cdn.cloudflare.net. 289 IN A	104.18.203.232

Here I got real IP addresses of SpeedTest. Let's see resolved IP addresses:

$ dig -x 104.18.202.232

18.104.in-addr.arpa.	893	IN	SOA	cruz.ns.cloudflare.com. dns.cloudflare.com. 2288625505 10000 2400 604800 3600

$ dig -x 104.18.203.232

18.104.in-addr.arpa.	757	IN	SOA	cruz.ns.cloudflare.com. dns.cloudflare.com. 2288625505 10000 2400 604800 3600

Same pattern here: although IP addresses are not resolved back to any domain names, the reverse zones themselves have a SOA that clearly has a pattern: email & primary DNS from the cloudflare.com zone. To have the same pattern, you need to have access to the reverse DNS zone where the VPS is purchased. But even if you write a dummy SOA with the same data as cloudflare, the following checks will fail:

$ dig -x 104.18.203.232

18.104.in-addr.arpa.	757	IN	SOA	cruz.ns.cloudflare.com. dns.cloudflare.com. 2288625505 10000 2400 604800 3600

It says which immediate upper zone is responsible for return addresses. Let's find the authoritative DNS for this zone:

$ dig ns 18.104.in-addr.arpa.

18.104.in-addr.arpa.	21600	IN	NS	cruz.ns.cloudflare.com.
18.104.in-addr.arpa.	21600	IN	NS	kevin.ns.cloudflare.com.

Now let's imagine that, for example, the address 8.8.8.8 (I took the public Google DNS example just as an example - you can use your own IP) is pretending to be a www.speedtest.net server, and GFW has a simple task - can the IP address 8.8.8.8 be a www.speedtest.net address (let's hypothetically imagine that Google has prescribed SOA or PTR records for 8.8.8.8 that also point to the cloudflare.com server to be similar to it as much as possible). A simple query to cloudlfare DNS on behalf of GFW will get a response:

$ dig @cruz.ns.cloudflare.com. -x 8.8.8.8

; <<>> DiG 9.18.18-0ubuntu0.22.04.2-Ubuntu <<>> @cruz.ns.cloudflare.com. -x 8.8.8.8
; (6 servers found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: REFUSED, id: 14800
                                                         ^^^^^^^^^^^^^^^^
;; flags: qr rd; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1
                                                  ^^^^^^^^^^^^^^^^^^^^^^^
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
; EDE: 20 (Not Authoritative)
^^^^^^^^^^^^^^^^^^^^^^^^^^
;; QUESTION SECTION:
;8.8.8.8.in-addr.arpa.		IN	PTR

Now compare that to requesting an IP address from his responsibility:

$ dig @cruz.ns.cloudflare.com. -x 104.18.203.232
; <<>> DiG 9.18.18-0ubuntu0.22.04.2-Ubuntu <<>> @cruz.ns.cloudflare.com. -x 104.18.203.232
; (6 servers found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 35797
                                                   ^^^^^^^^^^^^^^^^^^^^^^^^^
;; flags: qr aa rd; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1
                                                                  ^^^^^^^^^^^^^^^^
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;232.203.18.104.in-addr.arpa.	IN	PTR

;; AUTHORITY SECTION:
18.104.in-addr.arpa.	3600	IN	SOA	cruz.ns.cloudflare.com. dns.cloudflare.com. 2288625505 10000 2400 604800 3600

Conclusion: no matter how X-Ray server pretends to be any server via SNI, there is a simple way to check its IP address via reverse resolving, and if it is not there - then through analyzing SOA records and queries to DNS responsible for the reverse zone.

If you look at the first post - it may very well be that the increased traffic triggers a check that was performed in a similar manner. This is just as an idea.

[neo2enigma]

Guys
Have you ever managed to check this video:
https://www.youtube.com/watch?v=By1DflMboUc&t=2801s
It describes why GFW is getting powerful because of TLS leak in Iranian panels!

[irgfw]

@neo2enigma Yes.
This is irrelevant. Those exposed panels are from old versions without SSL domains or panel path.
Our tests are without panels and just pure xray-core with ufw and the results were consistent with or without panels.