feat(gateway): respond with ICMP error for filtered packets - #9816
Conversation
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
7516b63 to
a3d788e
Compare
jamilbk
left a comment
jamilbk
left a comment
There was a problem hiding this comment.
Interesting - I've never seen ICMP messages sent in reply to firewall ACLs. Typically the packet is blackholed / dropped to prevent port enumeration - any thoughts on whether that might be an issue here?
cc @bmanifold
We have ourselves been bitten by the complexity of traffic filters where we were suspecting a bug and in the end it was just a traffic filter. In this case, the "firewall" is not publicly reachable which is why I'd think it doesn't hurt to send an ICMP error. If the user gets this far, they are authorized to access the resource but the filter simply doesn't allow TCP/UDP/ICMP. |
|
This is a tough one, because I think we're just stuck in a difficult spot. In a dedicated firewall product I'm pretty sure you'd have the option to specify "drop" or "reject" which would allow the network admin to choose how they wanted to handle things. We sit in this strange position of sort of being a firewall, but not exactly. From a security standpoint this doesn't feel like a big risk, because as Thomas mentioned our gateway isn't publicly exposed. I think what we likely want to be considering is (probably in this order):
|
I am not sure if I can label it as "significant" but one of the benefits I can see is that users are less likely to blame Firezone not working when their traffic gets filtered if we reject it via ICMP. So hopefully less support requests as a result (I don't know how many people use traffic filters).
Hard to say but as I said earlier, given that it is an egress filter rather than an ingress filter (at least from the client's PoV), having some signal back that it is not allowed seems reasonable to expect. |
|
Just to confirm though, only power users that are actively looking for these logs (and then only on Linux?) will see these ICMP responses right? I.e. for regular users using HTTP apps they won't notice. |
Um, why do you think that? I am pretty sure these ICMP messages will lead to a near instant reset of a socket that tries to contact such a resource. So for example, if the traffic filter doesn't allow HTTP, trying to load the website will immediately display some error of a broken connection in the browser instead of loading for however many seconds the timeout is and then displaying the generic "cannot load website" error. It is a bit difficult to test locally because I can only test it from within the docker container and I can launch a graphical browser there but if we deploy this to staging, we can do a few more extended tests on what the enduser experience is. |
39b57e4 to
e8e61bb
Compare
|
Reminder to myself to test this on staging post-merge. |
|
Can confirm that if a resource filter is in place now, attempting to load a webpage that doesn't allow TCP 80 displays an immediate "cannot connect" error. It does not say anything about it being blocked though. |
In Firezone, a Client requests an "access authorization" for a Resource on the fly when it sees the first packet for said Resource going through the tunnel. If we don't have a connection to the Gateway yet, this is also where we will establish a connection and create the WireGuard tunnel. In order for this to work, the access authorization state between the Client and the Gateway MUST NOT get out of sync. If the Client thinks it has access to a Resource, it will just route the traffic to the Gateway. If the access authorization on the Gateway has expired or vanished otherwise, the packets will be black-holed. Starting with #9816, the Gateway sends ICMP errors back to the application whenever it filters a packet. This can happen either because the access authorization is gone or because the traffic wasn't allowed by the specific filter rules on the Resource. With this patch, the Client will attempt to create a new flow (i.e. re-authorize) traffic for this resource whenever it sees such an ICMP error, therefore acting as a way of synchronizing the view of the world between Client and Gateway should they ever run out of sync. Testing turned out to be a bit tricky. If we let the authorization on the Gateway lapse naturally, we portal will also toggle the Resource off and on on the Client, resulting in "flushing" the current authorizations. Additionally, it the Client had only access to one Resource, then the Gateway will gracefully close the connection, also resulting in the Client creating a new flow for the next packet. To actually trigger this new behaviour we need to: - Access at least two resources via the same Gateway - Directly send `reject_access` to the Gateway for this particular resource To achieve this, we dynamically eval some code on the API node and instruct the Gateway channel to send `reject_access`. The connection stays intact because there is still another active access authorization but packets for the other resource are answered with ICMP errors. To achieve a safe roll-out, the new behaviour is feature-flagged. In order to still test it, we now also allow feature flags to be set via env variables. Resolves: #10074 --------- Co-authored-by: Mariusz Klochowicz <mariusz@klochowicz.com>
3 participants
When defining a resource, a Firezone admin can define traffic filters to only allow traffic on certain TCP and/or UDP ports and/or restrict traffic on the ICMP protocol.
Presently, when a packet is filtered out on the Gateway, we simply drop it. Dropping packets means the sending application can only react to timeouts and has no other means on error handling. ICMP was conceived to deal with these kind of situations. In particular, the "destination unreachable" type has a dedicated code for filtered packets: "Communication administratively prohibited".
Instead of just dropping the not-allowed packet, we now send back an ICMP error with this particular code set, thus informing the sending application that the packet did not get lost but was in fact not routed for policy reasons.
When setting a traffic filter that does not allow TCP traffic, attempting to
curlsuch a resource now results in the following: