Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Workaround broken mod_security rules causing Piwik to not track visits #1460

Closed
halfdan opened this Issue · 19 comments

3 participants

@halfdan
Collaborator

Ticket for http://forum.piwik.org/index.php?showtopic=11321

I haven't had the time to investigate this any deeper, but the number of people reporting that Piwik is not tracking any visits is increasing and we should take a look at it.

Quote from mutor:
I am suspecting that the problem is that the tracking URL structure (query string after piwik.php) has changed slightly with the new version (0.6.3) and many hosts don't allow to have an URL as query string variable because of security reasons (injecting malicious code from external websites).

@robocoder
Collaborator

One of the posts referenced HostGator ... sure enough, I verified they have mod_security and are filtering any URL that contains "http:/" or "https:/" (and escaped variants; note: one forward slash) Reiterating/paraphrasing #564, mod_security users don't do the due diligence to inspect the rules before installing them -- some rules are absolute sh*t. The following return a 403 Forbidden om HostGator:

Vote wontfix.

@mattab
Owner

I think we did something earlier, strip http:// or https:// (maybe mailto: as well?) in front of the URLs, and put them back in the tracker.

This could easily be done in trunk, probably would have to wait for a next release coming in maybe 2 weeks?

@robocoder
Collaborator

We used to strip the scheme/protocol on the server-side.

If we simply strip the protocol on the client-side, we'd lose outlinks that begin with ftp:// (also blocked by HostGator), svn://, mailto:, etc.

To reassemble this on the server implies the workaround would be to separate the schema from the rest of the URL, e.g.,

  • http;//example.com/piwik.php?url=site1.com%2fpath&proto=http&urlref=site2.com%path&protoref=http

Note: this should be affecting the installer (first website setup) and the sitemanager -- as it would explain http://forum.piwik.org/index.php?showtopic=11291

@robocoder
Collaborator

HostGator and users in the forum confirm whitelisting the mod_security rule as the workaround.

@robocoder
Collaborator

At the moment, the HG mod_security rule (purportedly as a security measure against "XSS accounts") affects only $_GET parameters.

@mattab
Owner

maybe it is possible to white list mod_security in .htaccess?

Otherwise, is it wontfix? if we update piwik.js to not pass http:// , we also have to update the PiwikTracker client.

@robocoder
Collaborator

I proposed that we would still send the protocol (ftp or http) but as separate parameters. The parameters (containing the protocol) can be optional. If specified, the server concatenates it with the url. This allow for backward compatibility and cases where the browser has cached piwik.js.

Newer versions of mod_security (since 2008?) can no longer be disabled via .htaccess (eg "SecFilterEngine off" causes error code 500 Internal Server Error).

@robocoder
Collaborator

I'm leaning towards making this an installation systemcheck since the mod_security rules broadly impact Piwik, eg tracker, REST API functions that accept URLs, customData that might contain URLs, etc.

@robocoder
Collaborator

The installation system check will test an actual URL to see if it fails. If so, it's an error that can't be skipped.

(An alternative would be to query a remote server to get this server's IP address, then reverse whois to see if the IP range is assigned to HostGator or The Planet. One disadvantage of this approach is that it is host provider specific...not that they should be singled out, but they are the leading source for support requests.)

These workarounds are ugly, but do-able.

  • piwik.js: strip http:// and https:// from url, urlref, link, download, and redirect parameters; server-side: prepend urls with http:// if protocol is missing (to maintain backwards compatibility); make a similar change to PiwikTracker for consistency; similar changes to SitesManager (API.php and SitesManager.js) required

    • doesn't help with customdata that might contain user-specified URLs
  • piwik.js: add option to POST the request to the server; add a similar option to PiwikTracker; in both cases, the default is GET; note: subject to same-origin policy

    • some headers may be suppressed, so Cookie and UserAgent must be sent in the payload
    • some browsers provide limited information, e.g., no version number in Safari's user agent; Accept-Language not accessible via JavaScript -- might be able to fake it with navigator.language, navigator.browserLanguage, navigator.systemLanguage, and/or navigator.userLanguage

References:

@robocoder
Collaborator

(In [3231]) refs #1460 - add .setRequestMethod("POST") to piwik.js

Use cases:

  • workaround GET request length limit
  • workaround mod_security or .htaccess rules that block request containing "ht$

Notes:

  • subject to same origin policy
@robocoder
Collaborator

(In [3301]) fixes #1460 - SitesManager - add/update site uses POST

@mattab
Owner

Thanks vipsoft, great work on this one. Reopening to remember to document the procedure in a FAQ answer, and also on http://piwik.org/docs/javascript-tracking/

@robocoder
Collaborator

I already documented .setRequestMethod().

I just updated the FAQ http://piwik.org/faq/troubleshooting/#faq_58.

@robocoder
Collaborator

(In [3304]) fixes #1460 - add Installation check for the tracker (GET piwik.php?url=http://example.com); fix the tracker proxy when using POST

@robocoder
Collaborator

(In [3305]) refs #1460 - comment out checkForErrors on Installation welcome screen as it gets a false positive from the inline JavaScript source

@robocoder
Collaborator

(In [3306]) refs #1460 - add retry loop and pause for tracking test to complete

@robocoder
Collaborator

(In [3307]) refs #1460 - typo

@mattab
Owner

I'll update the warning message to a new FAQ answer about mod_security support

  • disable mod_security
  • or set POST method
@mattab
Owner

Anthon's message is fine

@halfdan halfdan added this to the Piwik 1.2 milestone
@robocoder robocoder was assigned by halfdan
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.