High Performance Overlay Network using WireGuard

[milux]

I want to explain an approach that we (Fraunhofer AISEC) are currently developing for a research project called "Platform Material Digital (PMD)". The objective of this discussion is to

1. evaluate the potential of a similar solution for the EDC, and
2. find other developers interested in implementing this technology (or, more precisely, the necessary OS interfaces) in the EDC, if the approach is found applicable and useful for the EDC.

Potential Value for EDC

The proposed WireGuard-based approach could be seen as an VPN-like extension of the EDC control plane.
It would allow direct communication between services on Connector instances, featuring transparent encryption and excellent performance for any IP-based (e.g. TCP/UDP) protocol, all whilst avoiding exposure of server interfaces to the web.
Further integration of related technologies (i.e. iptables) would allow to restrict access to certain services based on source IPs within the VPN.

Background

WireGuard is a VPN technology that has recently gained a lot of popularity, also due to its inclusion in the Linux kernel since version 5.6+.
By using low level implementations (i.e. Linux kernel or similar), it avoids the costly context switches known from technologies like OpenVPN, tinc, etc.
The identities used for encryption are created using ECDH on Curve25519 (For details, see https://www.wireguard.com/protocol/), which yields very concise public keys (32 bytes, represented as 44 characters, base64-encoded).

PMD Requirements and Current Overall Approach

We started the development of this approach as a (yet non-official) extension of IDS technologies, based on the following requirements:

1. IP-based "tunnels" between participants (VPN-like)
2. Secure, State-of-the-Art cryptography
3. High throughput (Gbit/s speeds on decent server systems)

Our current prototype looks roughly like this (further details on request):

• The whole network consists of a large IPv6 subnet (we chose fdXX::/112 in our case).
• Each participant (Connector) gets assigned an own IPv6 subnet (we use /64 subnets). This subnet is used locally to "publish" services for other participants. We use a IPv6 docker network here, where published services are containers added to the network.
• Each participant has a X.509-cert-based identity. This identity is provisioned on a central omejdn (a.k.a. DAPS reference implementation, for IDS folks) instance, along with its own (/64) subnet. (This solution could easily be replaced by another suitable technology which can act as a root of trust!)
• Each participant runs a public service, reachable from the internet, that we call the "WireGuard mesh daemon". For our prototype, this is a dedicated service written in Rust, but for the EDC, an extension in a JVM language would obviously make more sense.

The typical connection setup works as follows (simplified, we assume that Alice and Bob know each others public IPs/domains and corresponding WireGuard UDP ports, a.k.a. endpoints):

1. Alice and Bob regularly obtain tokens (a.k.a. a DAT in IDS terms) from the aforementioned omejdn server. Each request contains their current WireGuard public key, and the resulting JWT contains this public key along with their individual, fixed IPv6 subnet.
2. Alice contacts the WireGuard mesh daemon of Bob, presenting her token.
3. Bob checks the signature of the JWT. If valid, Bob will reply with his token to Alice and issue a wg set command, using the public key and subnet of Alice, to setup the WireGuard connection on his side.
4. Alice also checks the signature of Bob's JWT. If valid, Alice will also issue the corresponding wg set command, using the information of Bob's JWT.

Limitations

• Requires WireGuard, obviously, which somehow has to be provided by the operating system.
• Requires the WireGuard mesh daemon to be accessible externally (single, common TCP port). This might be replaced by another already defined communication channel/external interface.
• Requires the used WireGuard UDP port to be publicly accessible.

[paullatzelsperger]

Hey, first, thanks a lot for this detailed write-up, generally it sounds like interesting technology.

However, I struggle to understand what particular problem this attempts to solve.

EDC's current security model is aimed at and designed for employing industry standards like HTTPS and did:web to guarantee not only high levels of security but also the utmost level of interoperability (libraries, clients, etc.) and compatibility with other dataspace participants.

If the goal is to further secure communication, it needs to be opaque for EDC runtime instances.
VPN operates on the network layer rather than on the application layer, therefor it (or technologies like it) should be handled at the infrastructure/deployment level. EDC instances will very likely run in clustered environments, so network management should happen at the cluster level and not in the application, especially if it introduces dependencies onto a particular linux kernel version.

Are there any specific requirements/use-cases for this?

[milux]

Hey, thanks for your reply.

"High levels of security" is exactly my cue. I don't know precisely what the meaning of and the vision for this "high levels of security" is.
However, I have seen that Usage Control (UC) and its negotiation is an integral part of the EDC by now.

Now here is the point: As far as I can see it, the typical EDC deployment happens without any real (technical) guarantees about the correct enforcement of UC whatsoever. Any non-honest actor could easily bypass UC by performing changes to the EDC codebase. There is only a contractual promise that participants will behave honestly. Whether that is convincing for use cases with really sensitive data seems questionable to me.

Trusted Execution Environment (TEE) technologies combined with remote attestation (RA) solve the issue to some extend, but I don't see how this can be done in a way that is opaque to the applications exchanging data. In order to protect sensitive data, RA requires the transport channel (its cryptography, more precisely) to be "bound" to the remote attestation of the communicating parties.

Furthermore, there is still the separation of control plane and data plane, where the actual data transfer is delegated to some protocol after the contract negotiation has been done. Is there any further control over the transfer after that point? How would e.g. an ongoing, continuous data transfer be interrupted if a "stop condition" is met in UC enforcment?

I believe that at least some of these issues can be solved by a transparent layer 3 overlay similar to what we are proposing here.
As I already mentioned, modern Linux Kernels (5.6+) are featuring WireGuard out of the box, which makes it the most interesting choice here from my perspective.

Is that something worth discussing about?

Maybe use cases requiring "hard" technical enforcement guarantees are out of scope for EDC?
In that case, no such compromises will be necessary, and this idea is not right for the EDC indeed.

[paullatzelsperger]

Running EDC in some sort of trusted enclave (e.g. Intel SGX) is something that has surfaced in the past and we will have to look at down the road. Not a priority rn though.

One way to do this would be deploying an EDC runtime (or a part of it) in - say - a signed docker image in a VM and have it communicate through VPN, which is handled by the VM. You'd have to solve the issue of a) distributing the image and b) allowing the counter party to verify the image, but that's outside the scope of EDC, at least for the forseeable future.

I might not have said this explicitly, but people are free to develop an extension that implements some application-layer-terminated VPN connection that even might require a particular kernel etc. and do all kinds of weird stuff. Nothing stops you from doing so. We just won't adopt this into EDC directly because I believe it is architecturally unsound.

Lastly, implementing a stop condition can be done e.g. through a contract's expiration, either "naturally", or through an external event (e.g. an API call). The latter is not trivial, and probably not even possible all the time, because from a legal perspective it's doubtful if a single party can unilaterally "cancel" a contract.

I will close this discussion now, because it attempts to solve a problem we don't have (at the moment), and in a way that does not comply with EDC architecture. But most importantly, a valid, real-life use case has not been presented.