multi-wan: support sticky-address route-to

[syserr0r]

I believe gateway load balancing should be affected by Settings > Misc > 'Sticky Connections' (See #691 regarding unclear description) (see https://forum.pfsense.org/index.php?topic=49054.msg260494#msg260494 for information from pfsense forums)

If this is the case it does not appear to be working (as can be seen below):

note: the background tab in firefox is refreshing every second
note: there are no proxy settings on either browser

If the 'Sticky Connections' option is not intended to affect outgoing connections then this would be a feature request to add a 'Outgoing Sticky Connections' so that load-balanced multi-WAN is able to function with general web browsing (or other protocols that would expect a consistent IP)

[fichtner]

We really need to better distinguish between Multi-WAN and Load Balancing everywhere it seems. :)

It looks like the default for Multi-WAN is "route-to" in pf.conf(5), which will do a round-robin per connection, which is what you see. There are, however, pool options for route-to, which can use "sticky-address". Let's try with your setup on IRC next week? :)

[fichtner]

I found the cause... local traffic does not honour gateway settings, see: https://forum.opnsense.org/index.php?topic=2056

I'm marking this [upstream] and will talk to a FreeBSD dev.

[syserr0r]

The traffic isn't originating from the firewall itself in my case, are we sure this is the same issue?

[fichtner]

Hmm, no transparent proxy configured?

[syserr0r]

Correct, no proxy, transparent or otherwise, configured

[fichtner]

Ok, my mistake. The functionality is there but for some reason it won't work in your case.

[fichtner]

(and I mean really there with rules kewords and all, did we already look through a /tmp/rules.debug dump from you box?)

[syserr0r]

I don't think we've been through it -- I'll PM a dump.

[fichtner]

Config does not contain "sticky-address" tags for the respective pf rules, that means the setup tiptoes around the code that adds it. It looks like a config issue at this point, getting to the bottom of this soon enough now. :)

[syserr0r]

Yay, progress! Let me know if you need anything testing, but bear in mind it's a production firewall.

[fichtner]

@syserr0r what's the status on this? :)

[syserr0r]

I'll try and look at it again in the coming week, but last I looked at it (whch was a fair while ago) the source IP/connection wasn't sticky.

[syserr0r]

After some very preliminary testing it looks like the source address is staying somewhat sticky so I've rolled it out further and await feedback from our users.

[fichtner]

Closing this now.