modsecurity_crs_35_bad_robots.conf - Execution error - PCRE limits exceeded

[remotehelp]

[Mon Nov 04 15:49:43 2013] [error] [client 37.115.184.70] ModSecurity: Rule 7f18
532e53f8 [id "-"][file "/etc/httpd/modsecurity.d/activated_rules/modsecurity_crs
_35_bad_robots.conf"][line "28"] - Execution error - PCRE limits exceeded (-8):
(null). [hostname "example.com"] [uri "/index.php"] [unique_id "UneJ112qgHI
AAAguT-EAAAAA"]

== OWASP ModSecurity Core Rule Set (CRS) CHANGES ==

== Report Bugs/Issues to GitHub Issues Tracker ==

• https://github.com/SpiderLabs/owasp-modsecurity-crs/issues

== Version 2.2.8 - 06/30/2013 ==

[remotehelp]

less /etc/httpd/modsecurity.d/modsecurity.conf
....
SecPcreMatchLimit 2000
SecPcreMatchLimitRecursion 2000

[rcbarnett-zz]

Please paste in an audit log entry for this issue which contains the full inbound request data.

[ipauldev]

I'm having the same issues. Here is sanitized request data for a similar type of request, although it happens on other rules for me... top offenders are 981257, 981242, and 973302:

Error:
Rule 7f76b1a55f00 [id "973302"][file "/usr/share/modsecurity-crs/base_rules/modsecurity_crs_41_xss_attacks.conf"][line "309"] - Execution error - PCRE limits exceeded (-8): (null).
- Location:
- Tags:
- Transaction: Us7czX8AAAIAAFIyU0wAAAD1
- Rule ID:
- Session:
- User:
- Handler: dispatcher-handler
- Response Body Transformation: Dechunked
- Rule File: :0
- Producer: ModSecurity for Apache/2.7.4 (http://www.modsecurity.org/); OWASP_CRS/2.2.8.
- Server: Apache

Rule 7f76b1a55f00 [id "973302"][file "/usr/share/modsecurity-crs/base_rules/modsecurity_crs_41_xss_attacks.conf"][line "309"] - Execution error - PCRE limits exceeded (-8): (null).


Security Rule it is trying to process [modsecurity_crs_41_xss_attacks.conf"][line "309"]:
SecRule REQUEST_COOKIES|!REQUEST_COOKIES:/__utm/|REQUEST_COOKIES_NAMES|ARGS_NAMES|ARGS|XML:/* ".+application/x-shockwave-flash|image/svg\+xml|text/(css|html|ecmascript|javascript|vbscript|x-(javascript|scriptlet|vbscript)).+" \
"phase:2,rev:'2',ver:'OWASP_CRS/2.2.9',maturity:'8',accuracy:'8',id:'973302',capture,t:none,t:htmlEntityDecode,t:lowercase,block,msg:'XSS Attack Detected',logdata:'Matched Data: %{TX.0} found within %{MATCHED_VAR_NAME}: %{MATCHED_VAR}',tag:'OWASP_CRS/WEB_ATTACK/XSS',tag:'WASCTC/WASC-8',tag:'WASCTC/WASC-22',tag:'OWASP_TOP_10/A2',tag:'OWASP_AppSensor/IE1',tag:'PCI/6.5.1',setvar:'tx.msg=%{rule.msg}',setvar:tx.xss_score=+%{tx.critical_anomaly_score},setvar:tx.anomaly_score=+%{tx.critical_anomaly_score},setvar:tx.%{rule.id}-OWASP_CRS/WEB_ATTACK/XSS-%{matched_var_name}=%{tx.0}"


Request Headers:
GET /en/my-test-topic/test/test.html HTTP/1.1
host: qa.mytestdomain.com
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Encoding: gzip
Accept-Language: en-US,en;q=0.5
Akamai-Origin-Hop: 2
Authorization: Basic ZAAAAAAAAAAAAAAAAAAAAAAAAA==
Cache-Control: no-cache, max-age=0
Cookie: SessionPersistence-publish-qa=MYSTORE%3A%3DmembershipType%3D%2ClastName%3D%2CpreferredAreaID%3D0%2CpreferredArea%3D%2CmemberState%3Dnewmember%2CpreferredClubState%3D%2CareaVisits%3D0%2CmyTestVisibility%3Dfalse%2CfirstName%3D%7CCLIENTCONTEXT%3A%3DvisitorId%3Danonymous%2CvisitorId_xss%3Danonymous%7CPROFILEDATA%3A%3Davatar%3D%2Fetc%2Fmydesigns%2Fdefault%2Fimages%2Fusers%2Favatar.png%2Cpath%3D%2Fhome%2Fusers%2Fa%2Fanonymous%2Fprofile%2CisLoggedIn%3Dfalse%2CisLoggedIn_xss%3Dfalse%2CauthorizableId%3Danonymous%2CauthorizableId_xss%3Danonymous%2CformattedName%3D%2CformattedName_xss%3D%2Cage%3D%2Cage_xss%3D%7CGEOLOCATION%3A%3Dlatitude%3D37.333333%2Clongitude%3D-120.888888%2Caddress%2Fregion%3Dundefined%2Caddress%2Fregion_xss%3D%2Caddress%2FcountryCode%3Dundefined%2Caddress%2FcountryCode_xss%3D%2Caddress%2Fcountry%3Dundefined%2Caddress%2Fcountry_xss%3D%7CTAGCLOUD%3A%3Dinterests%3Atraining%3D4%2Cadditional%3Amy-test%3D2%2Cinterests%3Atest%3D4%2Cinterests%3Atesting-again%3D4%2Cinterests%3Atest%3D4%2Cinterests%3Atesting%3D2%2Cinterests%3Atesting-again2%3D4%2Cinterests%3Atest%3D4%2Cinterests%3Atest%3D4%2Cinterests%3Atest%3D4%2Cinterests%3Atest%3D4%2Cinterests%3Atest%3D4%2Cinterests%3Atest-again%3D2%2Cadditional%3Atest-another%3D4%2Cinterests%3Atest%3D4%2Cinterests%3Atest%3D4%2Cinterests%3Atest%3D2%2Cinterests%3Atest%3D2%2Cproperties%3Atype%2FsearchForm%3D120%2Cadditional%3Amytest%3D2%2Cadditional%3AmyTesting%3D1%2Ctest%3Atx%2FALVH%3D1%2Cadditional%3Amma%3D1%2Cadditional%3Atesting-again%3D2%7C; __utma=123456789.123456789.1234567890.1234567890.1234567890.1; __utmz=123456789.1234567890.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); s_nr=1234567890012-Repeat; oo_event_entry=a1b1c1d1e1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1

    [Fri Jan 10 16:21:34 2014] [notice] ModSecurity for Apache/2.7.5 (http://www.modsecurity.org/) configured.
    [Fri Jan 10 16:21:34 2014] [notice] ModSecurity: APR compiled version="1.4.6"; loaded version="1.4.6"
    [Fri Jan 10 16:21:34 2014] [notice] ModSecurity: PCRE compiled version="8.12 "; loaded version="8.12 2011-01-15"
    [Fri Jan 10 16:21:34 2014] [notice] ModSecurity: LIBXML compiled version="2.7.8"


    Server version: Apache/2.2.22 (Ubuntu)
    Server built: Mar 8 2013 15:53:20
    Server's Module Magic Number: 20051115:30
    Server loaded: APR 1.4.6, APR-Util 1.3.12
    Compiled using: APR 1.4.6, APR-Util 1.3.12
    Architecture: 64-bit
    Server MPM: Worker
    threaded: yes (fixed thread count)
    forked: yes (variable process count)
    Server compiled with....
    -D APACHE_MPM_DIR="server/mpm/worker"
    -D APR_HAS_SENDFILE
    -D APR_HAS_MMAP
    -D APR_HAVE_IPV6 (IPv4-mapped addresses enabled)
    -D APR_USE_SYSVSEM_SERIALIZE
    -D APR_USE_PTHREAD_SERIALIZE
    -D SINGLE_LISTEN_UNSERIALIZED_ACCEPT
    -D APR_HAS_OTHER_CHILD
    -D AP_HAVE_RELIABLE_PIPED_LOGS
    -D DYNAMIC_MODULE_LIMIT=128
    -D HTTPD_ROOT="/etc/apache2"
    -D SUEXEC_BIN="/usr/lib/apache2/suexec"
    -D DEFAULT_PIDLOG="/var/run/apache2.pid"
    -D DEFAULT_SCOREBOARD="logs/apache_runtime_status"
    -D DEFAULT_ERRORLOG="logs/error_log"
    -D AP_TYPES_CONFIG_FILE="mime.types"
    -D SERVER_CONFIG_FILE="apache2.conf"

[leonelr]

I am also having pretty much the same issue here ..... what was the outcome of this conversation ?

Error Sample:
[error] 23056#0: [client x.y.z.z] ModSecurity: Rule 7f3bf8cd1500 [id "970003"][file "/etc/nginx/modsecurity/activated_rules/modsecurity_crs_50_outbound.conf"][line "123"] - Execution error - PCRE limits exceeded (-8): (null). [hostname "HostName"] [uri "/_PATH/myfile.xml"] [unique_id "RcGc0cAc5cAcAcAFAsARlcAc"]

[error] 23625#0: [client p.r.s.t] ModSecurity: Rule 7f3c05866490 [id "981243"][file "/etc/nginx/modsecurity/activated_rules/modsecurity_crs_41_sql_injection_attacks.conf"][line "245"] - Execution error - PCRE limits exceeded (-8): (null). [hostname "HostName"] [uri "/AJAX/myfile.asp"] [unique_id "AcAcecAcAcAc8cAoAcACAcvS"]

nginx version: nginx/1.6.2
built by gcc 4.6.3
(Ubuntu/Linaro 4.6.3-1ubuntu5)

TLS SNI support enabled
configure arguments:
--with-openssl=../openssl-1.0.1j
--with-openssl-opt=--openssldir=/usr/local/openssl
--prefix=/usr/local
--conf-path=/etc/nginx/nginx.conf
--error-log-path=/var/log/nginx/error.log
--http-log-path=/var/log/nginx/access.log
--pid-path=/var/run/nginx.pid
--lock-path=/var/run/nginx.lock
--http-client-body-temp-path=/var/cache/nginx/client_temp
--http-proxy-temp-path=/var/cache/nginx/proxy_temp
--http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp
--http-uwsgi-temp-path=/var/cache/nginx/uwcgi_temp
--http-scgi-temp-path=/var/cache/nginx/scgi_temp
--user=nginx
--group=nginx
--with-file-aio
--with-ipv6
--with-http_ssl_module
--add-module=../modsecurity-2.8.0/nginx/modsecurity

---

2014/12/19 16:42:02 [notice] 539#0: ModSecurity for nginx (STABLE)/2.8.0 (http://www.modsecurity.org/) configured.
2014/12/19 16:42:02 [notice] 539#0: ModSecurity: APR compiled version="1.4.6"; loaded version="1.4.6"
2014/12/19 16:42:02 [notice] 539#0: ModSecurity: PCRE compiled version="8.12 "; loaded version="8.12 2011-01-15"

2014/12/19 16:42:02 [notice] 539#0: ModSecurity: LIBXML compiled version="2.7.8"

Any help would be much appreciated. Cheers,

[couchfault]

Exact same issue using nginx, a bunch of reports that like this:

--2d51bd24-H--
Message: Rule 7feff7710a68 [id "950901"][file "/etc/nginx/conf/modsecurity.conf"][line "1683"] - Execution error - PCRE limits exceeded (-8): (null).
Message: Rule 7feff7710a68 [id "950901"][file "/etc/nginx/conf/modsecurity.conf"][line "1683"] - Execution error - PCRE limits exceeded (-8): (null).
Message: Rule 7feff7710a68 [id "950901"][file "/etc/nginx/conf/modsecurity.conf"][line "1683"] - Execution error - PCRE limits exceeded (-8): (null).
Message: Rule 7feff7710a68 [id "950901"][file "/etc/nginx/conf/modsecurity.conf"][line "1683"] - Execution error - PCRE limits exceeded (-8): (null).
Message: Rule 7feff7710a68 [id "950901"][file "/etc/nginx/conf/modsecurity.conf"][line "1683"] - Execution error - PCRE limits exceeded (-8): (null).
Apache-Handler: IIS
Stopwatch: 1421699005000687 913586 (- - -)
Stopwatch2: 1421699005000687 913586; combined=12865, p1=241, p2=12375, p3=3, p4=160, p5=85, sr=23, sw=1, l=0, gc=0
Response-Body-Transformed: Dechunked
Producer: ModSecurity for nginx (STABLE)/2.9.0-RC2 (http://www.modsecurity.org/); OWASP_CRS/2.2.5.
Server: ModSecurity Standalone
Engine-Mode: "DETECTION_ONLY"

--2d51bd24-Z--

[couchfault]

The error appears to be in the last line of:

# -=[ SQL Tautologies ]=-
#
SecRule REQUEST_COOKIES|REQUEST_COOKIES_NAMES|REQUEST_FILENAME|ARGS_NAMES|ARGS|XML:/* "(?i:([\s'\"`´’‘\(\)]*)?([\d\w]+)([\s'\"`´’‘\(\)]*)?(?:=|<=>|r?like|sounds\s+like|regexp)([\s'\"`´’‘\(\)]*)?\2|([\s'\"`´’‘\(\)]*)?([\d\w]+)([\s'\"`´’‘\(\)]*)?(?:!=|<=|>=|<>|<|>|\^|is\s+not|not\s+like|not\s+regexp)([\s'\"`´’‘\(\)]*)?(?!\6)([\d\w]+))" \
        "phase:2,rev:'2.2.5',capture,multiMatch,t:none,t:urlDecodeUni,t:replaceComments,ctl:auditLogParts=+E,block,msg:'SQL Injection Attack',id:'950901',logdata:'%{TX.0}',severity:'2',tag:'WEB_ATTACK/SQL_INJECTION',tag:'WASCTC/WASC-19',tag:'OWASP_TOP_10/A1',tag:'OWASP_AppSensor/CIE1',tag:'PCI/6.5.2',setvar:'tx.msg=%{rule.msg}',setvar:tx.sql_injection_score=+%{tx.critical_anomaly_score},setvar:tx.anomaly_score=+%{tx.critical_anomaly_score},setvar:tx.%{rule.id}-WEB_ATTACK/SQL_INJECTION-%{matched_var_name}=%{tx.0}"

[couchfault]

What completely fixed the issue for me was just cloning the latest version of modsecurity crs from this repository instead of just using the one from my package manager

(which I honestly never should have used in the first place no matter how lazy I was feeling)

[Spiritux]

@735tesla, when I update the base rules to the last version (downloaded from owasp-git), the apache2 failed to start. The error points to new line formats, for example the new line of -=[ SQL Tautologies ]=-, which change from:

SecRule REQUEST_COOKIES|REQUEST_COOKIES_NAMES|REQUEST_FILENAME|ARGS_NAMES|ARGS|XML:/* "(?i:([\s'\"`´’‘\(\)]*)?([\d\w]+)([\s'\"`´’‘\(\)]*)?(?:=|<=>|r?like|sounds\s+like|regexp)([\s'\"`´’‘\(\)]*)?\2|([\s'\"`´’‘\(\)]*)?([\d\w]+)([\s'\"`´’‘\(\)]*)?(?:!=|<=|>=|<>|<|>|\^|is\s+not|not\s+like|not\s+regexp)([\s'\"`´’‘\(\)]*)?(?!\6)([\d\w]+))" \
        "phase:2,rev:'2.2.5',capture,multiMatch,t:none,t:urlDecodeUni,t:replaceComments,ctl:auditLogParts=+E,block,msg:'SQL Injection Attack',id:'950901',logdata:'%{TX.0}',severity:'2',tag:'WEB_ATTACK/SQL_INJECTION',tag:'WASCTC/WASC-19',tag:'OWASP_TOP_10/A1',tag:'OWASP_AppSensor/CIE1',tag:'PCI/6.5.2',setvar:'tx.msg=%{rule.msg}',setvar:tx.sql_injection_score=+%{tx.critical_anomaly_score},setvar:tx.anomaly_score=+%{tx.critical_anomaly_score},setvar:tx.%{rule.id}-WEB_ATTACK/SQL_INJECTION-%{matched_var_name}=%{tx.0}"

to:

SecRule REQUEST_COOKIES|!REQUEST_COOKIES:/__utm/|!REQUEST_COOKIES:/_pk_ref/|REQUEST_COOKIES_NAMES|ARGS_NAMES|ARGS|XML:/* "(?i:([\s'\"`´’‘\(\)]*?)\b([\d\w]++)([\s'\"`´’‘\(\)]*?)(?:(?:=|<=>|r?like|sounds\s+like|regexp)([\s'\"`´’‘\(\)]*?)\2\b|(?:!=|<=|>=|<>|<|>|\^|is\s+not|not\s+like|not\s+regexp)([\s'\"`´’‘\(\)]*?)(?!\2)([\d\w]+)\b))" \
          "phase:2,rev:'2',ver:'OWASP_CRS/2.2.9',maturity:'9',accuracy:'8',capture,multiMatch,t:none,t:urlDecodeUni,t:replaceComments,ctl:auditLogParts=+E,block,msg:'SQL Injection Attack: SQL Tautology Detected.',id:'950901',logdata:'Matched Data: %{TX.0} found within %{MATCHED_VAR_NAME}: %{MATCHED_VAR}',severity:'2',tag:'OWASP_CRS/WEB_ATTACK/SQL_INJECTION',tag:'WASCTC/WASC-19',tag:'OWASP_TOP_10/A1',tag:'OWASP_AppSensor/CIE1',tag:'PCI/6.5.2',setvar:'tx.msg=%{rule.msg}',setvar:tx.sql_injection_score=+%{tx.critical_anomaly_score},setvar:tx.anomaly_score=+%{tx.critical_anomaly_score},setvar:tx.%{rule.id}-OWASP_CRS/WEB_ATTACK/SQL_INJECTION-%{matched_var_name}=%{tx.0}"

Of course, a lot of other rules has been updated in same way.. I mean, including !REQUEST_COOKIES:/__utm/ and so on..

Could you please tell me if you had this problem when you updated your rules? Maybe is mandatory to update modsecurity to the last version?
I am running the last stable version of modsecurity and for apache as well, via apt-get in Debian.

Thanks

[dune73]

The PCRE match limits are here to prevent Regular Expression Denial of Service attacks. That is the CPU getting eaten up completely calculating evil regexes...

Now the defaults are very low, which leads to false positives in the sense that rule execution on regular requests run into the PCRE match limit failure as reported above several times.

I have successfully configured productive servers with PCRE match limits of 500000 or even 1000000. This makes most, if not all PCRE match limit failures go away and is still not a value, that would allow an easy DDoS. However, let' make this clear: You need to monitor your server and then - in case of an attack - you may need to lower the match limit to save CPU power.

An alternative is to improve the regular expressions to make them less prone to this sort of problem. This is an ongoing process and @csanders-git has made some progress with this.

For now, I am closing this issue as a more sane PCRE match limit makes the problem go away. Please reopen, if you still encounter this problem with benign requests and a PCRE match limit in the range of several hundred thousand.