Magma GTP Gateway

[Arsenii-Oganov]

Proposal: Magma GTP Gateway

Author(s): [@Arsenii-Oganov]

1. Objectives

The objective of this work is to add a GTP aggregation gateway to the Magma system. This Gateway will aid in scale deployments where S8 inbound roaming is supported by reducing the number of integration points for roaming partners.

Software built to accomplish this will be open source under BSD-3-Clause license and will be committed to the Magma software repository under the governance of the Linux foundation, such that it can be effectively maintained in the future releases.

2. Background

In traditional telecom deployments using a centralized (or non-distributed) core, roaming interfaces are limited in number. These interfaces often connect to an IPX/GRX network provider to aggregate roaming traffic into a small set of interfaces, regardless of the number of roaming partners an operator may have.

In the case of S8 roaming interconnections, the connection supports GTP-C and GTP-U traffic between the SGW in the visited network and the PGW in the home network. The SGW IP for the GTP-U endpoint must be routable from the home network. When using an IPX/GRX provider, this means having a globally routable IP address dedicated to the IPX/GRX network interface of an SGW.

Magma has a distributed core architecture. Each cell site, where the Access Gateway (AGW) element usually resides, has an SGW. The requirement that each site have a globally routable IP dedicated to the IPX/GRX connection (i.e. not a general traffic WAN port ISP provided address) is a challenge for operators using Magma core for roaming. Furthermore, IPX/GRX vendors are reluctant to support large numbers of VPN terminations coming from each AGW.

3. Implementation

3.1 Block Diagram

3.2 GTP Gateway Scope of Change

Below are system requirements for the GTP Gateway. These requirements assume the presence of Magma support for:

3.2.1 Wireguard/IPsec support in AGW for S8 GTP traffic

Random TEID assignment for S8 tunnels across AGWs
Wireguard / IPsec Connection Termination
Pipelined supports configuration of Wireguard tunnels which connect automatically when the service is started. These tunnels will be used for connecting AGWs to the GTP Gateway.

The GTP Gateway will need to support termination of 10k wireguard connections per instance. Instances will be horizontally scalable. This will produce a reduction in IP address usage of 1/10,000.

3.2.2 GTP Traffic Flow Learning

GTP traffic needs to be routed to the correct tunnel in the downlink direction. The GTP Gateway will use OVS rules (or potentially another mapping / CGNAT solution) to keep track of GTP tunnel to Wireguard tunnel mapping.

The initial implementation will use OVS to learn flows based on UE APN IP Address. Learning will occur by seeing uplink traffic originating from a tunnel, learning the UE source IP address, and creating a flow mapping for return traffic.

Alternate solutions may use @mark attribute of the sk_buff struct or other native linux networking stack functions/attributes for flow mapping. Such solutions will be explored for scale performance and evaluated against the OVS solution.

3.2.3 Orc8r Integration

The GTP Gateway will be integrated in the Magma ecosystem. Integration will have the following components:

• GTP Gateway will be deployable via a script prepared for bare metal host deployment similar to AGW.

• GTP Gateway will have a connection to orc8r for configuration, health and performance metrics similar to FeG.

• GTP Gateway will include control proxy to allow for gRPC between GTP Gateway and other services in the Magma deployment.

• GTP Gateway specific API endpoints will be created to support configuration and health monitoring:

  • List all GTP Gateways
  • Register a new GTP Gateway
  • Delete a GTP Gateway
  • Get a specific GTP Gateway

3.2.4 Gateway_Health Service

Gateway health service for GTP Gateway will report on the following metrics:

• GTP Gateway specific health which includes:

  • Number of active VPN connections
  • Number of active GTP connections
  • Status of PGW VPN (if present)
  • Total Ingress throughput per NIC
  • Total Egress throughput per NIC

• Generic Health status which includes:

  • Gateway - Controller connectivity
  • Status for all the running services
  • Number of restarts per each service
  • Number of errors per each service
  • Internet and DNS status
  • Kernel version
  • Magma version

3.2.5 HA Active Active

GTP Gateway will be deployed in HA Active Active configuration. GTP Gateway instances do not need to be colocated. Session loading will be balanced between the available instances.

3.2.6 Infrastructure

GTP Gateway will be deployed on Equinix Metal hosts, or other bare metal hosts. Equinix metal C3.Small will be used to start. These hosts support:

• PROC1 x Xeon E 2278G 8-Core Processor @ 3.4Ghz
• RAM 32GB
• 2 x 480GB SSD
• NIC2 x 10Gbps Bonded Ports

This specification will be considered the base supported specification until further performance data is collected.

NOTE: Equinix Metal chosen for it’s access to IPX/GRX network connections.

3.2.7 NMS GTP Gateway

NMS Elements will be created for display of key configuration and health elements including:

• View configuration
• SW Version
• Gateway - Controller connectivity
• Interface IP addresses
• View health metrics
• Number of active VPN connections
• Number of active GTP connections
• Status of PGW VPN (if present)
• Total Ingress throughput per NIC
• Total Egress throughput per NIC

4. Testing approach

• Main critical parts of the solution will be covered with unit tests
• Lab with real inbound roaming scenario will be created
• E2E tests will be done manually in the lab, test scenarios will be upstreamed to Magma repos

5. Security

There two main aspects regarding Roaming Gateway security architecture:

• External connectivity through the internet: deployment architecture will require secured IPX connection between GTP Gateway and HPLMN.
• GTP Gateway internal security will follow in a two-tiered approach:
  • mutual TLS termination at the Orc8r proxy
  • application-level access enforcement

6. Team

https://github.com/119Vik
https://github.com/isergieienkov
https://github.com/Arsenii-Oganov
https://github.com/sudoki2015
https://github.com/berezovskyi-oleksandr
https://github.com/jpad-freedomfi

7. Roadmap & Schedule

SWE hours approximately - 3000

[Arsenii-Oganov]

As per process:
issue - magma/magma#10882
PR with proposal - magma/magma#10901

[ssanadhya]

@Arsenii-Oganov , could you please review the updated guidelines on https://github.com/magma/grants and revise this proposal?

Specifically, it will be great to have more clarity on:

1. How do you plan to validate the GTP Gateway? What is the plan to add unit/integration tests?
2. In the roadmap section, what is the effort needed in terms of engineering resources?

[ardzoht]

@ssanadhya Changes have been applied and per Arsenii confirmation, Testing plan and Roadmap plans have been added on the grant description. Is it possible for the TSC to do a review?
cc @Arsenii-Oganov

[ssanadhya]

@Arsenii-Oganov , the roadmap section above still mentions each effort in number of months. It will be great if you could specify the number of person hours or number of SWEs that will be dedicated to this effort for each milestone. Because this is a grant proposal, if you expect expenses on licenses/testing infrastructure, please highlight those too. cc: @ardzoht