Performance issue on wag gui

[bluecraank]

Here i'm back again,

I don't want to badmouth WAG - but is there any possibility to boost GUI performance?

We are probably the exception when it comes to the number of devices and users

Devices: 163
Users: ~ 155

Changing something on groups takes about 4-10 Seconds until its saved.

Maybe any idea or solution for this?

FYI
WAG is very nice!

[NHAS]

Howdy!

Unfortunately for some reason changing the ebpf map takes quite a long time. I've tried to do it in a way that changes only individual entries which should be faster, however it's actually slower then this implementation.

Could you give me your kernel and distribution please?

I can't really promise any improvements as it currently works in under a second for our currently business (<50 users)

[NHAS]

If you could give a representative sample of your configuration for rules that would be super useful!

[NHAS]

Alright, so good news and bad news.

Good news, I've made this about 50% faster on my 177 device test deployment.

Bad news, the remaining 50% is due to the kernel syscall forcing global syncronisation before giving the EBPF program the new map containing the policies. Which was introduced recently, so it blocks for quite a while even when batching calls.
(cilium/ebpf#1297)

I've made quite a few changes that now live on unstable. So I'd be keen for you to try them out.

[NHAS]

Actually I spoke a little soon. I've done an optimization which caches some maps, and now I've gotten it down to ~30ms for 177 users.

This is on unstable and you should definitely see if it fixes your problem.

[bluecraank]

Seems to work well!

Editing groups just takes about 16ms atm

Thank you!

[bluecraank]

Oh!

I guess its to late but:

PRETTY_NAME="Debian GNU/Linux 12 (bookworm)"
Kernel 6.1.0-13-amd64

sample of our config:

"Acls": {
        "Groups": {
            "group:replaced": [
                "fake.usr",
                "fake.usr",
                "fake.usr"
            ],
            "group:replaced": [
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr"
            ],
            "group:replaced": [
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr"
            ],
            "group:replaced": [
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr"
            ],
            "group:replaced": [
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr"
            ],
            "group:replaced": [
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr"
            ],
            "group:ity": [
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr"
            ],
            "group:itz": [
                "fake.usr",
                "fake.usr"
            ],
            "group:replaced": [
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr"
            ],
            "group:replaced": [
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr"
            ],
            "group:replaced": [
                "fake.usr"
            ],
            "group:replaced": [
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr"
            ],
            "group:replaced": [
                "fake.usr"
            ],
            "group:replaced": [
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr"
            ],
            "group:rdp-clients": [
                "fake.usr",
                "fake.usr"
            ],
            "group:replaced": [
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr"
            ],
            "group:replaced": [
                "fake.usr",
                "fake.usr"
            ],
            "group:replaced": [
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr",
                "fake.usr"
            ]
        },
        "Policies": {
            "*": {
                "Allow": [
                    "fake.domain.local",
                    "fake.domain.local",
                    "fake.domain.local",
                    "fake.domain.local",
                    "fake.domain.local",
                    "fake.domain.local",
                    "fake.domain.local 10092/tcp 445/tcp 137/udp icmp",
                    "fake.domain.local",
                    "192.192.192.2/24"
                ]
            },
            "group:replaced": {
                "Mfa": [
                    "fake.domain.local 445/tcp"
                ]
            },
            "group:replaced": {
                "Mfa": [
                    "fake.domain.local 3389/tcp",
                    "fake.domain.local 80/tcp 443/tcp"
                ]
            },
            "group:replaced": {
                "Allow": [
                    "fake.domain.local"
                ]
            },
            "group:replaced": {
                "Mfa": [
                    "fake.domain.local",
                    "fake.domain.local",
                    "fake.domain.local 445/tcp 137/any",
                    "fake.domain.local 3389/tcp 443/tcp 80/tcp 445/tcp 137/any",
                    "fake.domain.local",
                    "fake.domain.local",
                    "fake.domain.local 445/tcp",
                    "fake.domain.local",
                    "fake.domain.local",
                    "fake.domain.local",
                    "fake.domain.local",
                    "fake.domain.local 443/tcp 80/tcp",
                    "fake.domain.local 80/tcp 443/tcp",
                    "fake.domain.local 80/tcp 443/tcp",
                    "fake.domain.local 80/tcp 443/tcp",
                    "fake.domain.local 443/tcp",
                    "fake.domain.local 80/tcp 443/tcp",
                    "fake.domain.local 80/tcp 443/tcp",
                    "fake.domain.local 80/tcp 443/tcp",
                    "fake.domain.local",
                    "fake.domain.local",
                    "fake.domain.local 4443/tcp"
                ],
                "Allow": [
                    "fake.domain.local",
                    "fake.domain.local"
                ]
            },
            "group:replaced": {
                "Mfa": [
                    "192.192.192.2/24"
                ]
            },
            "group:replaced": {
                "Mfa": [
                    "fake.domain.local 443/tcp",
                    "fake.domain.local 443/tcp 22/tcp",
                    "fake.domain.local 22/tcp 443/tcp 3306/tcp",
                    "fake.domain.local 22/tcp 443/tcp ",
                    "fake.domain.local 443/tcp 22/tcp",
                    "fake.domain.local 3389/tcp",
                    "fake.domain.local 3389/tcp 443/tcp",
                    "fake.domain.local 3389/tcp 1433/tcp",
                    "fake.domain.local 3317/tcp 22/tcp",
                    "fake.domain.local 3306/tcp 22/tcp",
                    "fake.domain.local 3306/tcp 22/tcp 80/tcp 443/tcp",
                    "fake.domain.local",
                    "fake.domain.local",
                    "fake.domain.local 80/tcp 443/tcp",
                    "fake.domain.local 80/tcp 443/tcp",
                    "fake.domain.local",
                    "fake.domain.local 10085/tcp",
                    "fake.domain.local 22/tcp 443/tcp",
                    "fake.domain.local 22/tcp 443/tcp",
                    "fake.domain.local 22/tcp 443/tcp 80/tcp",
                    "fake.domain.local 1433/tcp 3389/tcp",
                    "fake.domain.local 443/tcp",
                    "fake.domain.local",
                    "fake.domain.local 3389/tcp",
                    "192.192.192.2/22 3900/any",
                    "192.192.192.2/24 443/tcp 22/tcp 4444/tcp"
                ],
                "Allow": [
                    "192.192.192.2/24"
                ]
            },
            "group:ita": {
                "Mfa": [
                    "fake.domain.local 3389/tcp",
                    "fake.domain.local 3389/tcp",
                    "fake.domain.local 3389/tcp",
                    "fake.domain.local 3389/tcp",
                    "fake.domain.local 3389/tcp",
                    "fake.domain.local 3389/tcp",
                    "fake.domain.local 22/tcp",
                    "fake.domain.local 22/tcp",
                    "fake.domain.local 22/tcp",
                    "fake.domain.local 22/tcp",
                    "fake.domain.local 4444/tcp",
                    "fake.domain.local 4444/tcp",
                    "fake.domain.local 4444/tcp",
                    "fake.domain.local 4444/tcp",
                    "fake.domain.local 4444/tcp 443/tcp",
                    "192.192.192.2/24",
                    "192.192.192.2/24",
                    "192.192.192.2/24"
                ]
            },
            "group:ita2": {
                "Mfa": [
                    "fake.domain.local 3389/tcp"
                ],
                "Allow": [
                    "fake.domain.local 3389/tcp",
                    "fake.domain.local",
                    "fake.domain.local",
                    "fake.domain.local",
                    "fake.domain.local",
                    "fake.domain.local 443/tcp",
                    "fake.domain.local",
                    "192.192.192.2/24",
                    "fake.domain.local 3389/any"
                ]
            },
            "group:replaced": {
                "Mfa": [
                    "fake.domain.local",
                    "fake.domain.local",
                    "fake.domain.local"
                ]
            },
            "group:replaced": {
                "Mfa": [
                    "fake.domain.local 445/tcp"
                ]
            },
            "group:replaced": {
                "Mfa": [
                    "fake.domain.local 3389/tcp"
                ]
            },
            "group:replaced": {
                "Allow": [
                    "fake.domain.local 3389/any",
                    "192.192.192.2 80/tcp",
                    "fake.domain.local",
                    "fake.domain.local",
                    "fake.domain.local",
                    "fake.domain.local 443/tcp",
                    "fake.domain.local 80/tcp 443/tcp",
                    "fake.domain.local 80/tcp 443/tcp",
                    "fake.domain.local 80/tcp 443/tcp",
                    "fake.domain.local 443/tcp",
                    "fake.domain.local 80/tcp 443/tcp",
                    "fake.domain.local 80/tcp 443/tcp",
                    "fake.domain.local 80/tcp 443/tcp",
                    "fake.domain.local 80/tcp 443/tcp",
                    "fake.domain.local 443/tcp"
                ]
            },
            "group:replaced": {
                "Mfa": [
                    "fake.domain.local"
                ]
            },
            "group:replaced": {
                "Mfa": [
                    "fake.domain.local",
                    "fake.domain.local",
                    "fake.domain.localr"
                ]
            },
            "group:rdp-clients": {
                "Mfa": [
                    "fake.domain.local 3389/tcp",
                    "fake.domain.local 3389/tcp"
                ]
            },
            "group:replaced": {
                "Mfa": [
                    "fake.domain.local 443/tcp"
                ]
            },
            "group:replaced": {
                "Mfa": [
                    "fake.domain.local 443/tcp",
                    "fake.domain.local"
                ]
            },
            "group:replaced": {
                "Mfa": [
                    "fake.domain.local 3389/tcp"
                ]
            }
        }
    }

[NHAS]

Thanks for that! Really glad I could get significant performance improvement sorted it was quite fun.

It's awesome to see that you're using Wag successfully in a professional environment. I'd just like to let you know that I do accept donations on my support page and cryptocurrency wallets listed below, which helps keep this work going, and I tend to expedite issues opened by supporters.

Monero (XMR):
8A8TRqsBKpMMabvt5RxMhCFWcuCSZqGV5L849XQndZB4bcbgkenH8KWJUXinYbF6ySGBznLsunrd1WA8YNPiejGp3FFfPND

Bitcoin (BTC):
bc1qm9e9sfrm7l7tnq982nrm6khnsfdlay07h0dxfr

[NHAS]

A note on this, I have just found a security bug that occurs now as the map is not being recreated. Updating routes will fail to remove old routes from a users firewall. Thus leaving them able to hit routes they otherwise should not be able to.

As I am currently in the middle of a huge change from moving to sqlite3 to etcd for high availability I cannot go back and fix this right at the second.

[bluecraank]

Thanks for the info. Do you have any commit which fix it?