New system check to warn that Piwik is not compatible with mod_security

[mattab]

Reported in: #2997, some work was done in the early days in #1460

• Hostgator users need to contact their host to disable mod_security. Hundreds of piwik users have had to contact their hosts to disable mod_security.
• There are several known issues with Piwik and mod_security!
  • this thread and here

I have found that almost all rules in modsecurity_crs_41_sql_injection_attacks.conf need 

!REQUEST_COOKIES:/^_pk_ref.*/|!REQUEST_COOKIES:/^__utmz$/|!ARGS:gclid  
for google adwords, google analytics  and piwik to work ok with mod_security.

• this one

Pattern match "\\%(?![0-9a-fA-F]{2}|u[0-9a-fA-F]{4})" at ARGS:pattern. [file "/usr/local/apache/conf/modsec2.user.conf"] [line "20"] [id "1234123440"] [msg "URL Encoding Abuse Attack Attempt"] [severity "WARNING"]
  U4n-qUIt@YIADWbyzkUAAAB1 86.112.15.155 35095 66.45.249.132 80
--74545369-B-- 

As a proposed solution to inform users of potential issues early:

• Write a system check entry to check for mod_security and issue Warning if it is detected.
  • Suggest to user that it is OK to disable mod_security for Piwik app.
• Maybe in this system check message we could also link to FAQ and this FAQ could list the rules to disable in the mod_security config. if some users reading here may contribute them?

See similar #5081

[mattab]

It's never going to happen I think!

[mattab]

Also reported here: http://forum.piwik.org/read.php?2,88617,page=1#msg-98619

In the custom rule file, add following lines:

# Allow Piwik queries
  SecRule REQUEST_FILENAME "^/path_to_your_piwik_dir/piwik\.php$" id:99998,log,phase:2,chain,allow
  SecRule ARGS_NAMES "^action_name$"

  SecRule REQUEST_FILENAME "^/path_to_your_piwik_dir/index\.php$" id:99999,log,phase:2,chain,allow
  SecRule ARGS_NAMES "^module$"

[napa-web-designer]

Is there any update to this? I have my hosting with HostGator shared hosting and they are not willing to disable mod_sec for the whole server. The one tech I talked with said he could disable some rules for my domain as long as they are not flagged as required for them - but he would need to know which rules to disable. Is there any progress in knowing what rules to disable? This could be a path to allowing many more users to install Piwik. God Willing. Thanks.

[elsonsolano]

Same problem here with napa-web-designer. HostGator told us that they can't disable mod_sec for the whole server because i'm on a shared hosting. They just need to know which rules to disable. I don't know what rule it is to tell them.

[UVLabs]

Same, i'm on hostgator

[ordex]

This problem exists also on my onlydomains.com host. Any chance we can get the right custom rules that we can suggest to the hoster ?

[mattab]

Hello guys, at this point we do not have enough information to know how to make progress re: this issue.

The short answer: your web host should ideally not enable software that break stuff. mod_security rules are really breaking Piwik and it's not Piwik's fault (unfortunately, because that also means we can't easily fix it).

Maybe you have some details which mod security rules trigger the warnings/errors? Maybe we could contact Hostgator to get them to disable such rules for Piwik users...

[ordex]

I do agree with you: it's the way they have configured mod_sec that is breaking non-malicious applications and should be their duty to fix it.
However, I also do understand their position of not being willing to modify the mod_sec configuration to pleasure one app only (they may not know what else it will break b ychanging the config)...
Honestly, I don't know what to suggest :) But at least I got a log from my hoster OnlyDomains about my IP being blocked. Posting it here just in case it can be useful in a way or another:

 ~/scripts/modgrep.pl -s x.x.x.x -t $(date --date="Thu Jul  7 03:16:38 2016" +"%s") -r 300 -n 1234123440
Last result:

--f9ceae7f-A-- [07/Jul/2016:03:16:32 --0500] V34P4HdRQuoAAronW7MAAAAS
x.x.x.x 48624 x.x.x.x 80 --f9ceae7f-B-- POST
/piwik/index.php?date=today&format=JSON2&idSite=1&limit=15&method=SitesManager.getPatternMatchSites&module=API&pattern=%25&period=day
HTTP/1.1 Host: [1]www.yyyyyy.com User-Agent: Mozilla/5.0 (X11;
Linux x86_64; rv:47.0) Gecko/20100101 Firefox/47.0 Accept: application/json,
text/plain, */* Accept-Language: en-US,en;q=0.5 Accept-Encoding: gzip, deflate
DNT: 1 Content-Type: application/x-www-form-urlencoded Cache-Control: no-cache
Referer:
[2]http://www.yyyyyy.com/piwik/index.php?module=UsersManager&action=userSettings&idSite=1&period=day&date=today&ignoreSalt=XXXXXXXXXXXXXXXXXXXXX
Content-Length: 43 Cookie:
piwik_auth=login%3DXXXXXXXXXXXXXX%3Atoken_auth%3DXXXXXXXXXXXXXXXXXXXXXXXXXXX%3D%3D%3A_%3DXXXXXXXXXXXXXXXXXXXXXx;
piwik_ignore=ignore%3DXXXXX%3D%3A_%3DXXXXXXXXXXXXXXXXXXXXXXx;
PIWIK_SESSID=XXXXXXXXXXXXXXXX Via: 1.1 x.x.x.x Connection:
keep-alive --f9ceae7f-F-- HTTP/1.1 406 Not Acceptable Content-Length: 532
Keep-Alive: timeout=5, max=91 Connection: Keep-Alive Content-Type: text/html;
charset=iso-8859-1 --f9ceae7f-H-- Message: Access denied with code 406 (phase
2). Pattern match "\\%(?![0-9a-fA-F]{2}|u[0-9a-fA-F]{4})" at ARGS:pattern.
[file "/usr/local/apache/conf/modsec2.user.conf"] [line "50"] [id
"1234123440"] [msg "URL Encoding Abuse Attack Attempt"] [severity "WARNING"]
Apache-Error: [file "core.c"] [line 3722] [level 3] File does not exist:
/home/wzoq9i8f/public_html/406.shtml, referer:
[3]http://www.yyyyyy.com/piwik/index.php?module=UsersManager&action=userSettings&idSite=1&period=day&date=today&ignoreSalt=XXXXXXXXXXXXXXXXXXXX
Action: Intercepted (phase 2) Stopwatch: Thu Jul 7 03:16:32 2016 985 (- - -)
Stopwatch2: Thu Jul 7 03:16:32 2016 985; combined=114, p1=67, p2=45, p3=0,
p4=0, p5=2, sr=31, sw=0, l=0, gc=0 Producer: ModSecurity for Apache/2.9.0
(http://www.modsecurity.org/). Server: Apache/2.2.31 (Unix) mod_ssl/2.2.31
OpenSSL/1.0.1e-fips mod_bwlimited/1.4 Engine-Mode: "ENABLED" --f9ceae7f

does it ring any bell ? something that can be changed in piwik to workaround this ? I know this has already be answered...but just in case this log has something different..

Thanks!

[mattab]

Hi @ordex thanks for the log line. In this case I believe the issue is that the Piwik URL index.php?date=today&format=JSON2&idSite=1&limit=15&method=SitesManager.getPatternMatchSites&module=API&pattern=%25&period=day matches the rule %(?![0-9a-fA-F]{2}|u[0-9a-fA-F]{4}) via the %25 in pattern=%25... it is the URL encoded of %, likely someone searches for % in the Search bar in the Site selector...

If anyone has some other log lines matching some other mod security rules, feel free to post them here.

[dune73]

Hi there, I do not know much about PiWik but quite a bit about ModSecurity. The log reported by @ordex points to rule id 1234123440 which is a custom rule outside of the official rule id ranges. So this is a local rule designed by the hoster. They should know how to disable it.

Otherwise, we are looking into publishing a brief guide on how to run and secure PiWik in combination with ModSecurity.

[JonTheWong]

Hey guys;

I just wanted to follow up on this. Here is a list of rules for piwik to serve its files properly

950120
950901
981257
981245
981240
981246
981243
950109
981248

The only issue i've been seeing is; that end-users who have piwik tracking enabled are also seeing issues, and disabling all of the above is not idea.

ps: i also found this

http://www.ashworthconsulting.com/scripts-code/using-piwik-when-mod_security-is-enabled-on-your-site.html

But haven't tested it.

[dune73]

ModSecurity Core Rule Set 3.0 has been released in the meantime. Last week actually. The new release brings a huge reduction in false positives. Most likely all or almost all of the ones mentioned by @zmjwong.

https://lists.owasp.org/pipermail/owasp-modsecurity-core-rule-set/2016-November/002265.html

Please upgrade.

In the meantime I have been continuing on my ModSec/Piwik integration. Complete blogpost coming soon now.

[falzard]

http://www.ashworthconsulting.com/scripts-code/using-piwik-when-mod_security-is-enabled-on-your-site.html

Is it enough? It seems quite simple

[dune73]

They configure ModSecurity to not look at POST requests. It depends on the setup, but generally this does not sound like a safe practice.

[falzard]

Sorry if I misunderstand but they don't configure ModSecurity, they configure Piwik, no? or atleast they configure ModSecurity to not look at POST requests but for Piwik? like a whitelist?

piwikTracker.setRequestMethod('POST');

I didn't enable mod_security yet but I intend to and I just installed Piwik.

[dune73]

Sorry for not making myself clear.

Yes, that's what I meant. They configure it in a way that piwik requests coming in as POST requests bypass ModSecurity.

[Findus23]

Inspired by https://twitter.com/julianguttzeit/status/1107794236701925377
I want to dig out this old Matomo bug (feel free to move it back a few releases if they are already full) because mod_security related issues make up quite a fraction of all support requests and broken Matomo instances.

I think it would be worth it to find out common mod_security rules that break Matomo and make a system check that makes requests to URLs matching these patterns and checks if it returns 403.
If so a short text could be shown to users explaining what's the issue.

[tsteur]

I'll move it to the priority backlog. 3.10, 3.11, and pretty much 3.12 are already quite full and I think it doesn't have too much of a priority compared to all the other things.

[Findus23]

@tsteur Okay, I added the Help wanted label as implementing this shouldn't require much Matomo knowledge. (And more mod_security knowledge)

[tsteur]

👍 be for sure good to have

[tsteur]

see #18064 we won't work on this for now as it's hard to impossible to identify if mod_security is installed or not and mod_security seems to be no longer supported in 2024. I'll close this issue for now as wontfix. If you are reading this and are using mod_security, I recommend you disable it.

[dune73]

Nah, it's just that Trustwave wants to hand over ModSecurity to an open source community. Talks are happening as we speak.

In the end, the engine for the SecLanguage does not matter too much. What is important and what "incompatible" means is that the OWASP ModSecurity Core Rule Set has rules that bite Matomo and these rules run on ModSecurity or a compatible web application firewall. And there are several compatible options.

[michelamarie]

Sorry to contradict your (now rather old) comment, @tsteur, but disabling web application firewalls or other security controls weakens security for web sites and their users. Modsecurity deflects all sorts of horrible activity that web sites are inundated with these days, and it would be a shame to go without that protection. So, it is far better to identify and work out any conflicts your software has with those things, or to at least make some recommendations on configuration adjustments or exceptions administrators can make in order for these things to work together.

As @dune73 pointed out, Modsecurity is still actively maintained, and there are several other web application firewalls in wide use that are based on it and/or use CRS. I should also mention that a lot has changed since 2012, when this ticket was first opened, and both Modsecurity and CRS generally (not just with Matomo) produce far fewer false-positive rule violations than they did in years past.

For my part, I am running the latest Matomo with the latest CRS on Modsecurity 2.9.7-1+b1 on Debian 12 Bookworm in one of our environments, and it appears to be running without issue. I haven't tested this exact configuration thoroughly yet, and I may have had an issue with it recently that I can't remember, but Matomo is tracking web activity properly with Modsecurity and CRS enabled. I have also been running Matomo with recent versions of Modsecurity and CRS on Rocky Linux for several months in another environment with no trouble what so ever.

For those seeking a quick solution to this sort of issue, making a host-based exclusion for the host, where the web sites that are being tracked by Matomo are (the web server), and for hosts that connect to the Matomo dashboard (such as your computer), should prevent most or all false positives, while keeping Modsecurity and CRS fully enabled in protecting the public web sites and Matomo iteself from attack.

An example of such an exclusion rule for those running Apache (put this in the Apache virtual host configuration for Matomo):
SecRule REMOTE_ADDR "@ipmatch 192.168.2.123,10.0.0.20" "phase:1,id:4000100,allow,nolog,ctl:ruleEngine=Off"

You can use CIDR notation to exclude an entire network in the above rule: '192.168.2.0/24,10.0.0.0/16', for example.

If you want to see rule violations by excluded hosts, so that you may make more precise exclusions later, change the above-shown 'nolog' bit to 'log'. You may have to also omit the 'ctl:ruleEngine=Off' bit for that logging to work.

A more precise way to perform exclusions would be to identify the rules being breached during the use and operation or Matomo, by checking the web server logs and modsec_audit.log. Then make exclusions for these rules under the tightest possible conditions -- only for requests for specific locations and/or by specified hosts/nets, etc. This can be done with the Apache and directives, among others, along with Apache variables to serve as conditions for Modsecurity exception configuration parameters.

Hopefully, that helps anyone else running Matomo with Modsecurity! So far, my experience has been very positive (but no false-positives, heh), and I quite like Matomo . . . Modsecurity, too, of course! ;)

[dune73]

Very nice writeup @michelamarie. If you need anything from the ModSecurity / CRS side, please get in touch.

[michelamarie]

Thank you, @dune73!! Thanks for all your effort on Modsecurity and CRS as well! I use them quite a lot, and they definitely help us sleep better at my workplace. If you ever need anything from me on the project, I'm happy to help, too! :)