New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New system check to warn that Piwik is not compatible with mod_security #3371

Open
mattab opened this Issue Sep 8, 2012 · 16 comments

Comments

Projects
None yet
8 participants
@mattab
Member

mattab commented Sep 8, 2012

Reported in: #2997, some work was done in the early days in #1460

  • Hostgator users need to contact their host to disable mod_security. Hundreds of piwik users have had to contact their hosts to disable mod_security.
  • There are several known issues with Piwik and mod_security!

I have found that almost all rules in modsecurity_crs_41_sql_injection_attacks.conf need 

!REQUEST_COOKIES:/^_pk_ref.*/|!REQUEST_COOKIES:/^__utmz$/|!ARGS:gclid  
for google adwords, google analytics  and piwik to work ok with mod_security.
Pattern match "\\%(?![0-9a-fA-F]{2}|u[0-9a-fA-F]{4})" at ARGS:pattern. [file "/usr/local/apache/conf/modsec2.user.conf"] [line "20"] [id "1234123440"] [msg "URL Encoding Abuse Attack Attempt"] [severity "WARNING"]
  U4n-qUIt@YIADWbyzkUAAAB1 86.112.15.155 35095 66.45.249.132 80
--74545369-B-- 

As a proposed solution to inform users of potential issues early:

  • Write a system check entry to check for mod_security and issue Warning if it is detected.
    • Suggest to user that it is OK to disable mod_security for Piwik app.
  • Maybe in this system check message we could also link to FAQ and this FAQ could list the rules to disable in the mod_security config. if some users reading here may contribute them?

See similar #5081

@mattab

This comment has been minimized.

Show comment
Hide comment
@mattab

mattab Dec 14, 2012

Member

It's never going to happen I think!

Member

mattab commented Dec 14, 2012

It's never going to happen I think!

@mattab

This comment has been minimized.

Show comment
Hide comment
@mattab

mattab Jun 2, 2014

Member

Also reported here: http://forum.piwik.org/read.php?2,88617,page=1#msg-98619

In the custom rule file, add following lines:

# Allow Piwik queries
  SecRule REQUEST_FILENAME "^/path_to_your_piwik_dir/piwik\.php$" id:99998,log,phase:2,chain,allow
  SecRule ARGS_NAMES "^action_name$"

  SecRule REQUEST_FILENAME "^/path_to_your_piwik_dir/index\.php$" id:99999,log,phase:2,chain,allow
  SecRule ARGS_NAMES "^module$"

Member

mattab commented Jun 2, 2014

Also reported here: http://forum.piwik.org/read.php?2,88617,page=1#msg-98619

In the custom rule file, add following lines:

# Allow Piwik queries
  SecRule REQUEST_FILENAME "^/path_to_your_piwik_dir/piwik\.php$" id:99998,log,phase:2,chain,allow
  SecRule ARGS_NAMES "^action_name$"

  SecRule REQUEST_FILENAME "^/path_to_your_piwik_dir/index\.php$" id:99999,log,phase:2,chain,allow
  SecRule ARGS_NAMES "^module$"

@mattab mattab added this to the 2.x - The Great Piwik 2.x Backlog milestone Jul 8, 2014

@mattab mattab added T: Task labels Jul 8, 2014

@mattab mattab removed the Major label Aug 3, 2014

@napa-web-designer

This comment has been minimized.

Show comment
Hide comment
@napa-web-designer

napa-web-designer Jan 20, 2016

Is there any update to this? I have my hosting with HostGator shared hosting and they are not willing to disable mod_sec for the whole server. The one tech I talked with said he could disable some rules for my domain as long as they are not flagged as required for them - but he would need to know which rules to disable. Is there any progress in knowing what rules to disable? This could be a path to allowing many more users to install Piwik. God Willing. Thanks.

napa-web-designer commented Jan 20, 2016

Is there any update to this? I have my hosting with HostGator shared hosting and they are not willing to disable mod_sec for the whole server. The one tech I talked with said he could disable some rules for my domain as long as they are not flagged as required for them - but he would need to know which rules to disable. Is there any progress in knowing what rules to disable? This could be a path to allowing many more users to install Piwik. God Willing. Thanks.

@elsonsolano

This comment has been minimized.

Show comment
Hide comment
@elsonsolano

elsonsolano Mar 24, 2016

Same problem here with napa-web-designer. HostGator told us that they can't disable mod_sec for the whole server because i'm on a shared hosting. They just need to know which rules to disable. I don't know what rule it is to tell them.

elsonsolano commented Mar 24, 2016

Same problem here with napa-web-designer. HostGator told us that they can't disable mod_sec for the whole server because i'm on a shared hosting. They just need to know which rules to disable. I don't know what rule it is to tell them.

@UVLabs

This comment has been minimized.

Show comment
Hide comment
@UVLabs

UVLabs Apr 9, 2016

Same, i'm on hostgator

UVLabs commented Apr 9, 2016

Same, i'm on hostgator

@ordex

This comment has been minimized.

Show comment
Hide comment
@ordex

ordex Jul 7, 2016

This problem exists also on my onlydomains.com host. Any chance we can get the right custom rules that we can suggest to the hoster ?

ordex commented Jul 7, 2016

This problem exists also on my onlydomains.com host. Any chance we can get the right custom rules that we can suggest to the hoster ?

@mattab

This comment has been minimized.

Show comment
Hide comment
@mattab

mattab Jul 8, 2016

Member

Hello guys, at this point we do not have enough information to know how to make progress re: this issue.

The short answer: your web host should ideally not enable software that break stuff. mod_security rules are really breaking Piwik and it's not Piwik's fault (unfortunately, because that also means we can't easily fix it).

Maybe you have some details which mod security rules trigger the warnings/errors? Maybe we could contact Hostgator to get them to disable such rules for Piwik users...

Member

mattab commented Jul 8, 2016

Hello guys, at this point we do not have enough information to know how to make progress re: this issue.

The short answer: your web host should ideally not enable software that break stuff. mod_security rules are really breaking Piwik and it's not Piwik's fault (unfortunately, because that also means we can't easily fix it).

Maybe you have some details which mod security rules trigger the warnings/errors? Maybe we could contact Hostgator to get them to disable such rules for Piwik users...

@ordex

This comment has been minimized.

Show comment
Hide comment
@ordex

ordex Jul 8, 2016

I do agree with you: it's the way they have configured mod_sec that is breaking non-malicious applications and should be their duty to fix it.
However, I also do understand their position of not being willing to modify the mod_sec configuration to pleasure one app only (they may not know what else it will break b ychanging the config)...
Honestly, I don't know what to suggest :) But at least I got a log from my hoster OnlyDomains about my IP being blocked. Posting it here just in case it can be useful in a way or another:

 ~/scripts/modgrep.pl -s x.x.x.x -t $(date --date="Thu Jul  7 03:16:38 2016" +"%s") -r 300 -n 1234123440
Last result:

--f9ceae7f-A-- [07/Jul/2016:03:16:32 --0500] V34P4HdRQuoAAronW7MAAAAS
x.x.x.x 48624 x.x.x.x 80 --f9ceae7f-B-- POST
/piwik/index.php?date=today&format=JSON2&idSite=1&limit=15&method=SitesManager.getPatternMatchSites&module=API&pattern=%25&period=day
HTTP/1.1 Host: [1]www.yyyyyy.com User-Agent: Mozilla/5.0 (X11;
Linux x86_64; rv:47.0) Gecko/20100101 Firefox/47.0 Accept: application/json,
text/plain, */* Accept-Language: en-US,en;q=0.5 Accept-Encoding: gzip, deflate
DNT: 1 Content-Type: application/x-www-form-urlencoded Cache-Control: no-cache
Referer:
[2]http://www.yyyyyy.com/piwik/index.php?module=UsersManager&action=userSettings&idSite=1&period=day&date=today&ignoreSalt=XXXXXXXXXXXXXXXXXXXXX
Content-Length: 43 Cookie:
piwik_auth=login%3DXXXXXXXXXXXXXX%3Atoken_auth%3DXXXXXXXXXXXXXXXXXXXXXXXXXXX%3D%3D%3A_%3DXXXXXXXXXXXXXXXXXXXXXx;
piwik_ignore=ignore%3DXXXXX%3D%3A_%3DXXXXXXXXXXXXXXXXXXXXXXx;
PIWIK_SESSID=XXXXXXXXXXXXXXXX Via: 1.1 x.x.x.x Connection:
keep-alive --f9ceae7f-F-- HTTP/1.1 406 Not Acceptable Content-Length: 532
Keep-Alive: timeout=5, max=91 Connection: Keep-Alive Content-Type: text/html;
charset=iso-8859-1 --f9ceae7f-H-- Message: Access denied with code 406 (phase
2). Pattern match "\\%(?![0-9a-fA-F]{2}|u[0-9a-fA-F]{4})" at ARGS:pattern.
[file "/usr/local/apache/conf/modsec2.user.conf"] [line "50"] [id
"1234123440"] [msg "URL Encoding Abuse Attack Attempt"] [severity "WARNING"]
Apache-Error: [file "core.c"] [line 3722] [level 3] File does not exist:
/home/wzoq9i8f/public_html/406.shtml, referer:
[3]http://www.yyyyyy.com/piwik/index.php?module=UsersManager&action=userSettings&idSite=1&period=day&date=today&ignoreSalt=XXXXXXXXXXXXXXXXXXXX
Action: Intercepted (phase 2) Stopwatch: Thu Jul 7 03:16:32 2016 985 (- - -)
Stopwatch2: Thu Jul 7 03:16:32 2016 985; combined=114, p1=67, p2=45, p3=0,
p4=0, p5=2, sr=31, sw=0, l=0, gc=0 Producer: ModSecurity for Apache/2.9.0
(http://www.modsecurity.org/). Server: Apache/2.2.31 (Unix) mod_ssl/2.2.31
OpenSSL/1.0.1e-fips mod_bwlimited/1.4 Engine-Mode: "ENABLED" --f9ceae7f

does it ring any bell ? something that can be changed in piwik to workaround this ? I know this has already be answered...but just in case this log has something different..

Thanks!

ordex commented Jul 8, 2016

I do agree with you: it's the way they have configured mod_sec that is breaking non-malicious applications and should be their duty to fix it.
However, I also do understand their position of not being willing to modify the mod_sec configuration to pleasure one app only (they may not know what else it will break b ychanging the config)...
Honestly, I don't know what to suggest :) But at least I got a log from my hoster OnlyDomains about my IP being blocked. Posting it here just in case it can be useful in a way or another:

 ~/scripts/modgrep.pl -s x.x.x.x -t $(date --date="Thu Jul  7 03:16:38 2016" +"%s") -r 300 -n 1234123440
Last result:

--f9ceae7f-A-- [07/Jul/2016:03:16:32 --0500] V34P4HdRQuoAAronW7MAAAAS
x.x.x.x 48624 x.x.x.x 80 --f9ceae7f-B-- POST
/piwik/index.php?date=today&format=JSON2&idSite=1&limit=15&method=SitesManager.getPatternMatchSites&module=API&pattern=%25&period=day
HTTP/1.1 Host: [1]www.yyyyyy.com User-Agent: Mozilla/5.0 (X11;
Linux x86_64; rv:47.0) Gecko/20100101 Firefox/47.0 Accept: application/json,
text/plain, */* Accept-Language: en-US,en;q=0.5 Accept-Encoding: gzip, deflate
DNT: 1 Content-Type: application/x-www-form-urlencoded Cache-Control: no-cache
Referer:
[2]http://www.yyyyyy.com/piwik/index.php?module=UsersManager&action=userSettings&idSite=1&period=day&date=today&ignoreSalt=XXXXXXXXXXXXXXXXXXXXX
Content-Length: 43 Cookie:
piwik_auth=login%3DXXXXXXXXXXXXXX%3Atoken_auth%3DXXXXXXXXXXXXXXXXXXXXXXXXXXX%3D%3D%3A_%3DXXXXXXXXXXXXXXXXXXXXXx;
piwik_ignore=ignore%3DXXXXX%3D%3A_%3DXXXXXXXXXXXXXXXXXXXXXXx;
PIWIK_SESSID=XXXXXXXXXXXXXXXX Via: 1.1 x.x.x.x Connection:
keep-alive --f9ceae7f-F-- HTTP/1.1 406 Not Acceptable Content-Length: 532
Keep-Alive: timeout=5, max=91 Connection: Keep-Alive Content-Type: text/html;
charset=iso-8859-1 --f9ceae7f-H-- Message: Access denied with code 406 (phase
2). Pattern match "\\%(?![0-9a-fA-F]{2}|u[0-9a-fA-F]{4})" at ARGS:pattern.
[file "/usr/local/apache/conf/modsec2.user.conf"] [line "50"] [id
"1234123440"] [msg "URL Encoding Abuse Attack Attempt"] [severity "WARNING"]
Apache-Error: [file "core.c"] [line 3722] [level 3] File does not exist:
/home/wzoq9i8f/public_html/406.shtml, referer:
[3]http://www.yyyyyy.com/piwik/index.php?module=UsersManager&action=userSettings&idSite=1&period=day&date=today&ignoreSalt=XXXXXXXXXXXXXXXXXXXX
Action: Intercepted (phase 2) Stopwatch: Thu Jul 7 03:16:32 2016 985 (- - -)
Stopwatch2: Thu Jul 7 03:16:32 2016 985; combined=114, p1=67, p2=45, p3=0,
p4=0, p5=2, sr=31, sw=0, l=0, gc=0 Producer: ModSecurity for Apache/2.9.0
(http://www.modsecurity.org/). Server: Apache/2.2.31 (Unix) mod_ssl/2.2.31
OpenSSL/1.0.1e-fips mod_bwlimited/1.4 Engine-Mode: "ENABLED" --f9ceae7f

does it ring any bell ? something that can be changed in piwik to workaround this ? I know this has already be answered...but just in case this log has something different..

Thanks!

@mattab

This comment has been minimized.

Show comment
Hide comment
@mattab

mattab Jul 11, 2016

Member

Hi @ordex thanks for the log line. In this case I believe the issue is that the Piwik URL index.php?date=today&format=JSON2&idSite=1&limit=15&method=SitesManager.getPatternMatchSites&module=API&pattern=%25&period=day matches the rule %(?![0-9a-fA-F]{2}|u[0-9a-fA-F]{4}) via the %25 in pattern=%25... it is the URL encoded of %, likely someone searches for % in the Search bar in the Site selector...

If anyone has some other log lines matching some other mod security rules, feel free to post them here.

Member

mattab commented Jul 11, 2016

Hi @ordex thanks for the log line. In this case I believe the issue is that the Piwik URL index.php?date=today&format=JSON2&idSite=1&limit=15&method=SitesManager.getPatternMatchSites&module=API&pattern=%25&period=day matches the rule %(?![0-9a-fA-F]{2}|u[0-9a-fA-F]{4}) via the %25 in pattern=%25... it is the URL encoded of %, likely someone searches for % in the Search bar in the Site selector...

If anyone has some other log lines matching some other mod security rules, feel free to post them here.

@dune73

This comment has been minimized.

Show comment
Hide comment
@dune73

dune73 Sep 27, 2016

Hi there, I do not know much about PiWik but quite a bit about ModSecurity. The log reported by @ordex points to rule id 1234123440 which is a custom rule outside of the official rule id ranges. So this is a local rule designed by the hoster. They should know how to disable it.

Otherwise, we are looking into publishing a brief guide on how to run and secure PiWik in combination with ModSecurity.

dune73 commented Sep 27, 2016

Hi there, I do not know much about PiWik but quite a bit about ModSecurity. The log reported by @ordex points to rule id 1234123440 which is a custom rule outside of the official rule id ranges. So this is a local rule designed by the hoster. They should know how to disable it.

Otherwise, we are looking into publishing a brief guide on how to run and secure PiWik in combination with ModSecurity.

@JonTheWong

This comment has been minimized.

Show comment
Hide comment
@JonTheWong

JonTheWong Nov 16, 2016

Hey guys;

I just wanted to follow up on this. Here is a list of rules for piwik to serve its files properly

950120
950901
981257
981245
981240
981246
981243
950109
981248

The only issue i've been seeing is; that end-users who have piwik tracking enabled are also seeing issues, and disabling all of the above is not idea.

ps: i also found this

http://www.ashworthconsulting.com/scripts-code/using-piwik-when-mod_security-is-enabled-on-your-site.html

But haven't tested it.

JonTheWong commented Nov 16, 2016

Hey guys;

I just wanted to follow up on this. Here is a list of rules for piwik to serve its files properly

950120
950901
981257
981245
981240
981246
981243
950109
981248

The only issue i've been seeing is; that end-users who have piwik tracking enabled are also seeing issues, and disabling all of the above is not idea.

ps: i also found this

http://www.ashworthconsulting.com/scripts-code/using-piwik-when-mod_security-is-enabled-on-your-site.html

But haven't tested it.

@dune73

This comment has been minimized.

Show comment
Hide comment
@dune73

dune73 Nov 16, 2016

ModSecurity Core Rule Set 3.0 has been released in the meantime. Last week actually. The new release brings a huge reduction in false positives. Most likely all or almost all of the ones mentioned by @zmjwong.

https://lists.owasp.org/pipermail/owasp-modsecurity-core-rule-set/2016-November/002265.html

Please upgrade.

In the meantime I have been continuing on my ModSec/Piwik integration. Complete blogpost coming soon now.

dune73 commented Nov 16, 2016

ModSecurity Core Rule Set 3.0 has been released in the meantime. Last week actually. The new release brings a huge reduction in false positives. Most likely all or almost all of the ones mentioned by @zmjwong.

https://lists.owasp.org/pipermail/owasp-modsecurity-core-rule-set/2016-November/002265.html

Please upgrade.

In the meantime I have been continuing on my ModSec/Piwik integration. Complete blogpost coming soon now.

@falzard

This comment has been minimized.

Show comment
Hide comment
@dune73

This comment has been minimized.

Show comment
Hide comment
@dune73

dune73 Nov 19, 2016

They configure ModSecurity to not look at POST requests. It depends on the setup, but generally this does not sound like a safe practice.

dune73 commented Nov 19, 2016

They configure ModSecurity to not look at POST requests. It depends on the setup, but generally this does not sound like a safe practice.

@falzard

This comment has been minimized.

Show comment
Hide comment
@falzard

falzard Nov 19, 2016

Sorry if I misunderstand but they don't configure ModSecurity, they configure Piwik, no? or atleast they configure ModSecurity to not look at POST requests but for Piwik? like a whitelist?

piwikTracker.setRequestMethod('POST');

I didn't enable mod_security yet but I intend to and I just installed Piwik.

falzard commented Nov 19, 2016

Sorry if I misunderstand but they don't configure ModSecurity, they configure Piwik, no? or atleast they configure ModSecurity to not look at POST requests but for Piwik? like a whitelist?

piwikTracker.setRequestMethod('POST');

I didn't enable mod_security yet but I intend to and I just installed Piwik.

@dune73

This comment has been minimized.

Show comment
Hide comment
@dune73

dune73 Nov 20, 2016

Sorry for not making myself clear.

Yes, that's what I meant. They configure it in a way that piwik requests coming in as POST requests bypass ModSecurity.

dune73 commented Nov 20, 2016

Sorry for not making myself clear.

Yes, that's what I meant. They configure it in a way that piwik requests coming in as POST requests bypass ModSecurity.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment