[bug] [proxy] Blacklist is evaluated before custom whitelist, groups acl broken

[inetshell]

Issue

Currently, the plugin os-web-proxy-useracl do not work if a local and/or remote blacklist is used. (This is an issue with opnsense core webproxy config).

How to replicate

I tried the following case:

• Install os-web-proxy-useracl plugin
• Add a remote access control list and block all video and file transfer sites (I'm using http://www.shallalist.de/Downloads/shallalist.tar.gz)
  
• Enable proxy local authentication
  
• Create three groups for different access levels: file_sharing, youtube, all_access
• Create three users with "Proxy: Login" privilege and add them to one group
  
• Create the following whitelist in "Services: Web Proxy: Groups and Users"
  

Expected behavior

• user1: should access wetransfer.com
• user2: should access youtube.com
• user3: should access any site, even the remote ACL

Actual behavior

No user can access the specified sites in their whitelists.

Cause

In squid.conf, the blacklists (marked in red) are evaluated before the "Auth plugins" include, which has the custom whitelists (marked in blue):

Solution

Move the blacklists after "Auth plugins" include. (Tested in OPNsense 18.7.7-amd64):

[fichtner]

@inetshell perfect bug report, thanks ❤️

[inetshell]

I create the following PR to resolve this issue:
opnsense/core#2899

[inetshell]

Thanks @fichtner !
Keep up the good work!

[AdSchellevis]

@inetshell nice report indeed, thanks!

I'm not using the user-acl plugin myself, so I'm not entirely sure what it's trying to add to the acl at the moment. I understand that for your case you would like the plugin rules to prevail above the core rules, but the question is if that's the most suitable solution.
My concern is that when we move the acl around, someone else files a bug report that the general blacklists aren't functional anymore....

Maybe we should ask @kekek2 first about the intended behaviour for the plugin he wrote and decide what to do next, since his plugin seems to be the only one affected by this change.

[kekek2]

General blacklists should take precedence over user-acl. To avoid such issues, I redid the plugin. Now, in the plugin itself, you can set rules similar to the general ones. And you can set their priority relative to each other. You can add rules mixed with the desired priority.

[inetshell]

Hi @kekek2,
How can I use a remote ACL in the plugin? For example:
-I want to block streaming and filetransfer sites. I can use Shalla list for that.
-But a couple of users needs to access youtube, so I create a group an use a whitelist with youtube on it.
-I don't want to allow this group to access all internet sites (Like filetransfer sites and other sites in blacklist).
-I would prefer to do not have to add manually every site to the black list (Remote ACL can do it for me)

The current plugin can not do this. But if we use the PR I sent, is possible. Is there any other way to solve this?

Thanks!

[inetshell]

I would love to have a way to use the ACL like this. Its my last requirement to be able to replace a 3 years old pfSense box with OPNsense :)

[kekek2]

In the new version of the plugin you can already do all this. I also added support Remote ACL.

[fichtner]

The problem is the new version has gained core features and needs redesign because we can't just remove core features.

[kekek2]

I can make a simplified version that does not affect core features.

[inetshell]

Any news about this? As an alternative solution, is it possible to create a switch so the user can enable/disable this?

Thanks!

[AdSchellevis]

@inetshell not from our side, it's a problem with the plugin, which isn't maintained by us.
It's probably better to open the ticket in plugins and close this one.

[kekek2]

@fichtner whats about plugin simplified version that does not affect core features ?

[mimugmail]

Everything not touching core has a high chance getting in. I'd also Like to see some progress with the new version of you plugin :)

[inetshell]

Thanks anyway

[elvysmarchon]

In the new version of the plugin you can already do all this. I also added support Remote ACL.

@kekek2 this new version its available?

[kekek2]

pull-request not accepted. 06.09.2019, 14:33, "elvysmarchon" <notifications@github.com>:In the new version of the plugin you can already do all this. I also added support Remote ACL.@kekek2 this new version its available?—You are receiving this because you were mentioned.Reply to this email directly, view it on GitHub, or mute the thread.  --    

[rafaelolivei]

Good Morning! @inetshell I have exactly this problem. I did the procedure to try to resolve as pointed out here in this bug, but also did not have good results. @kekek2 Is there a more updated version of plugins? I am wanting to migrate from endian to opnsense and just missing it.

[AdSchellevis]

This issue has been automatically timed-out (after 180 days of inactivity).

For more information about the policies for this repository,
please read https://github.com/opnsense/plugins/blob/master/CONTRIBUTING.md for further details.

If someone wants to step up and work on this issue,
just let us know, so we can reopen the issue and assign an owner to it.

[AndyX90]

Is this bug still present? I have setup a remote blacklist and want to whitelist urls for some users/groups via os-web-proxy-useracl and the settings have no effect?!

EDIT:
I have tested it and the problem persists.

In my opinion it would be better to rework the acl-thing.
In the current situation (gerneral blacklist implementation in core and group/user-acl as plugin) i would expect to create a global blacklist in the webproxy and then use individual whitelists for groups/users via plugin.

Maybe it would be better to move the general blacklist into plugin or the groups/user-acl in core?

@kekek2 can you please give some hints or rework the plugin?

@AdSchellevis Could you please consider internally to remove the old plugin and migrate the new one (opnsense/core#2572 and #754) into core?
I think the general blacklist and also the management for users/groups are overlapping and are core features of modern firewalls.
For Example Sophos UTM, there is a base Policy and there are so called Filter-Policies with their own Filteractions which can be assigned to users or groups.
It would be really great to have such a feature.
Maybe @kekek2 would help migrating..

This would also concern #748.