Using one UE to Ping another, but failed

[dongzx666]

After referring to issue 232, I tried to use UE on one machine to Ping UE on another machine, but failed.(the command I used is ping -I 10.45.0.31 10.45.0.32)
Here's the architecture.

         Open5GS              UERANSIM
     ----------------     ----------------
VM0 |                |   | uesimtun0      | VM1
    |                |---| 10.45.0.31     |
    |                |   |----------------|
    |  ogstun        |                  
    |  10.45.0.1     |   |----------------|
    |                |   | uesimtun0      | VM2
    |                |---| 10.45.0.32     |
     ----------------     ----------------

Here are the PCAP files.
PCAP.zip
Thank you so much.

[aligungr]

I think ICMP packets successfully leaves the VM1 and goes to the VM0. But probably core network does not know how to route a packet which is targeted to 10.45.0.32

I really don't know if it's expected behaviour or not.

Perhaps Sukchan can tell us if it has something related to Open5gs. Or some network experts in the community may inform us.

[acetcom]

@dongzx666 and @aligungr

Now I am busy with other work. I'll look at this issue later.

Sorry for not help you now!
Sukchan

[acetcom]

@dongzx666

I've reviewed your pcap.

The packet departing from the UE arrived well to open5gs and came out as an ICMP packet in 5gs-ogstun.pcap. The next path is depending on VM0's network configuration. Unfortunately, I don't know how to configure VM0's network in this case.

Sorry for not helping this.
Sukchan

[dongzx666]

@acetcom I agree with you. Does this mean that the UEs belong to the same 5GS can't communicate with each other on the IP layer?

[acetcom]

@dongzx666 Open5GS has nothing to do with communication between the two UEs.

[s5uishida]

Hi @dongzx666

I tried ping and ssh with the following configuration.

• Open5GS v2.2.2 (latest)
• UERANSIM v3.1.4 (latest)

       Open5GS        UERANSIM(gNB)     UERANSIM(UE)
     ------------     -------------     ---------------
VM0 |            |   | VM1         |   | uesimtun0     | VM2
    |            |   |             |---| 10.46.0.2 (X) | user: master
    | ogstun2    |   |             |    ---------------
    | 10.46.0.1  |---|             |
    |            |   |             |    ---------------
    |            |   |             |   | uesimtun0     | VM3
    |            |   |             |---| 10.46.0.3 (Y) | user: master
     ------------     -------------     ---------------

[VM0]
ip tuntap add name ogstun2 mode tun
ip addr add 10.46.0.1/16 dev ogstun2
ip link set ogstun2 up

iptables -t nat -A POSTROUTING -s 10.46.0.0/16 ! -o ogstun2 -j MASQUERADE

a) ping X --> Y [OK]

ping 10.46.0.3 -I ueransim0

b) ssh X --> Y [OK]

ssh -b 10.46.0.2 master@10.46.0.3

[aligungr]

@s5uishida Thanks for the description

[dongzx666]

@s5uishida Thank you，I will try it later.

[dongzx666]

@s5uishida After a long time, I finally tested your step. Unfortunately, it failed.
The configurations I tested were Open5GS(v2.2.6) and UERANSIM(v3.1.6)

       Open5GS        UERANSIM(gNB)     UERANSIM(UE)
     ------------     -------------     ---------------
VM0 |            |   | VM1         |   | uesimtun0     | VM1
    |            |   |10.112.81.222|---| 10.45.0.2 (X) | user: master
    | ogstun     |   |             |    ---------------
    | 10.45.0.1  |---|             |
    |            |   |             |    ---------------
    |            |   |             |   | uesimtun0     | VM2
    |            |   |             |---| 10.45.0.3 (Y) | user: master
     ------------     -------------     ---------------

There are two differences from your description.
One is that my UE1 and GNB are on one machine, and UE2 is on another.
The second is that I have more complex iptables rules.

sudo sysctl -w net.ipv4.ip_forward=1
sudo iptables -I INPUT --source 10.45.0.2/16 -j ACCEPT
sudo iptables -t nat -I POSTROUTING --out-interface eno1 -j MASQUERADE
sudo iptables -I FORWARD --in-interface eno1 --out-interface ogstun -j ACCEPT
sudo iptables -I FORWARD --in-interface ogstun --out-interface eno1 -j ACCEPT
# If the above rules are not implemented, UE will not be able to obtain the traffic of DN.
iptables -t nat -A POSTROUTING -s 10.45.0.0/16 ! -o ogstun -j MASQUERADE

I used Wireshark to capture the PCAP files of ogstun of Open5GS, enps4 of GNB and uesimtun0 of UE. The PCAP files shown in the three are consistent with the figure below.

If you know where the problem is, it would be great. Thank you!

[dongzx666]

@s5uishida Maybe the problem still lies in the NAT rules? I try to use only the rule you mentioned(iptables -t nat -A POSTROUTING -s 10.45.0.0/16 ! -o ogstun -j MASQUERADE) Ping UE with another is still failed. Even UE can't Ping Google. Thank you very much.

[dongzx666]

Here I come again. I tried to deploy the project according to @s5uishida's architecture.
But one UE Ping another UE(ping 10.45.0.2 -I ueransim0) still failed (ping 10.45.0.13 -I ogstun and ping www.google.com -I ueransim0 were all succeed.). Can anyone share some opinions?
I notice that the communication between UE and DN cannot be guaranteed only by using the following NAT rules.

iptables -t nat -A POSTROUTING -s 10.45.0.0/16 ! -o ogstun -j MASQUERADE

[dongzx666]

After a long experiment, I finally found that there were some conflicts in the rules of iptables on my Ubuntu where Open5GS was deployed, which affected the communication between UEs.
I replaced a clean Ubuntu environment and carefully checked that the iptables rules were empty. Then I redeployed Open5GS.
With only two rules provided by the author(acetcom), the UE can successfully access DN and Ping another UE.

sudo sysctl -w net.ipv4.ip_forward=1
sudo iptables -t nat -A POSTROUTING -s 10.45.0.0/16 ! -o ogstun -j MASQUERADE

The following shows the interworking between UEs.

$ sudo bash ./nr-binder 10.45.0.3 traceroute 10.45.0.2

traceroute to 10.45.0.2 (10.45.0.2), 30 hops max, 60 byte packets
 1  10.45.0.1 (10.45.0.1)  1.531 ms  1.497 ms  1.472 ms
 2  10.45.0.2 (10.45.0.2)  2.555 ms  2.544 ms  2.523 ms

@acetcom @aligungr @s5uishida Thank you all for your work, especially the two authors who provide us with two wonderful projects.