Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rule MULTIPART_UNMATCHED_BOUNDARY, id:'200003' fails on valid multipart/form-data submission #652

Closed
temyers opened this issue Jan 30, 2014 · 25 comments

Comments

Projects
None yet
@temyers
Copy link

commented Jan 30, 2014

The MULTIPART_UNMATCHED_BOUNDARY rule fires, denying the request if the multipart/form data includes "--" at the beginning of a line. If there is anything else at the start of the line (e.g. " --") then the rule does not trigger.

This issue was identified against a Liferay 6.2 installation fronted by apache+modsecurity, submitting a freemarker code snippet.

A sample curl command:

curl 'https://localhost/group/control_panel/manage?p_auth=MWq0gmZw&p_p_id=166&p_p_lifecycle=1&p_p_state=pop_up&p_p_mode=view&doAsGroupId=10328&refererPlid=10331&controlPanelCategory=current_site.content&_166_refererPortletName=15&_166_refererWebDAVToken=journal&_166_scopeTitle=Templates&_166_struts_action=%2Fdynamic_data_mapping%2Fedit_template' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' -H 'Accept-Encoding: gzip, deflate' -H 'Accept-Language: en-US,en;q=0.5' -H 'Connection: keep-alive' -H 'Cookie: JSESSIONID=E10AB37213FD6D61DE10A72AB8FF1A3A; LFR_SESSION_STATE_10345=1390888801890' -H 'DNT: 1' -H 'Host: localhost' -H 'Referer: https://localhost/group/control_panel/manage?p_p_id=166&p_p_lifecycle=0&p_p_state=pop_up&doAsGroupId=10328&refererPlid=10331&controlPanelCategory=current_site.content&_166_refererPortletName=15&_166_refererWebDAVToken=journal&_166_scopeTitle=Templates&_166_cmd=update&_166_struts_action=%2Fdynamic_data_mapping%2Fedit_template&_166_redirect=https%3A%2F%2Flocalhost%2Fgroup%2Fcontrol_panel%2Fmanage%3Fp_p_id%3D166%26p_p_lifecycle%3D0%26p_p_state%3Dpop_up%26p_p_mode%3Dview%26doAsGroupId%3D10328%26refererPlid%3D10331%26controlPanelCategory%3Dcurrent_site.content%26_166_refererPortletName%3D15%26_166_refererWebDAVToken%3Djournal%26_166_scopeTitle%3DTemplates%26_166_groupId%3D10328%26_166_showHeader%3D0%26_166_classNameId%3D10102%26_166_eventName%3DselectStructure%26_166_struts_action%3D%252Fdynamic_data_mapping%252Fview_template&_166_templateId=10850&_166_groupId=10328&_166_classNameId=10102&_166_classPK=0&_166_type=display&_166_structureAvailableFields=' -H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:26.0) Gecko/20100101 Firefox/26.0' -H 'Content-Type: multipart/form-data; boundary=---------------------------1835282785777842564651277339' --data-binary $'Content-Type: multipart/form-data; boundary=---------------------------1835282785777842564651277339\r\nContent-Length: 3492\r\n\r\n-----------------------------1835282785777842564651277339\r\nContent-Disposition: form-data; name="_166_formDate"\r\n\r\n1390888796288\r\n-----------------------------1835282785777842564651277339\r\nContent-Disposition: form-data; name="_166_cmd"\r\n\r\nupdate\r\n-----------------------------1835282785777842564651277339\r\nContent-Disposition: form-data; name="_166_redirect"\r\n\r\nhttps://localhost/group/control_panel/manage?p_p_id=166&p_p_lifecycle=0&p_p_state=pop_up&p_p_mode=view&doAsGroupId=10328&refererPlid=10331&controlPanelCategory=current_site.content&_166_refererPortletName=15&_166_refererWebDAVToken=journal&_166_scopeTitle=Templates&_166_groupId=10328&_166_showHeader=0&_166_classNameId=10102&_166_eventName=selectStructure&_166_struts_action=%2Fdynamic_data_mapping%2Fview_template\r\n-----------------------------1835282785777842564651277339\r\nContent-Disposition: form-data; name="_166_closeRedirect"\r\n\r\n\r\n-----------------------------1835282785777842564651277339\r\nContent-Disposition: form-data; name="_166_portletResource"\r\n\r\n\r\n-----------------------------1835282785777842564651277339\r\nContent-Disposition: form-data; name="_166_templateId"\r\n\r\n10850\r\n-----------------------------1835282785777842564651277339\r\nContent-Disposition: form-data; name="_166_groupId"\r\n\r\n10328\r\n-----------------------------1835282785777842564651277339\r\nContent-Disposition: form-data; name="_166_classNameId"\r\n\r\n10102\r\n-----------------------------1835282785777842564651277339\r\nContent-Disposition: form-data; name="_166_classPK"\r\n\r\n0\r\n-----------------------------1835282785777842564651277339\r\nContent-Disposition: form-data; name="_166_type"\r\n\r\ndisplay\r\n-----------------------------1835282785777842564651277339\r\nContent-Disposition: form-data; name="_166_structureAvailableFields"\r\n\r\n\r\n-----------------------------1835282785777842564651277339\r\nContent-Disposition: form-data; name="_166_saveAndContinue"\r\n\r\n1\r\n-----------------------------1835282785777842564651277339\r\nContent-Disposition: form-data; name="_166_name"\r\n\r\nTest\r\n-----------------------------1835282785777842564651277339\r\nContent-Disposition: form-data; name="_166_name_en_AU"\r\n\r\nTest\r\n-----------------------------1835282785777842564651277339\r\nContent-Disposition: form-data; name="_166_language"\r\n\r\nftl\r\n-----------------------------1835282785777842564651277339\r\nContent-Disposition: form-data; name="_166_description"\r\n\r\n\r\n-----------------------------1835282785777842564651277339\r\nContent-Disposition: form-data; name="_166_description_en_AU"\r\n\r\n\r\n-----------------------------1835282785777842564651277339\r\nContent-Disposition: form-data; name="_166_smallImage"\r\n\r\nfalse\r\n-----------------------------1835282785777842564651277339\r\nContent-Disposition: form-data; name="_166_type"\r\n\r\nfalse\r\n-----------------------------1835282785777842564651277339\r\nContent-Disposition: form-data; name="_166_smallImageFile"; filename=""\r\nContent-Type: application/octet-stream\r\n\r\n\r\n-----------------------------1835282785777842564651277339\r\nContent-Disposition: form-data; name="_166_scriptContent"\r\n\r\n<#--\r\nDisplay templates are used to lay out the fields defined in a data\r\ndefinition.\r\n\r\nPlease use the left panel to quickly add commonly used variables.\r\nAutocomplete is also available and can be invoked by typing "${".\r\n-->\r\n-----------------------------1835282785777842564651277339\r\nContent-Disposition: form-data; name="_166_script"; filename=""\r\nContent-Type: application/octet-stream\r\n\r\n\r\n-----------------------------1835282785777842564651277339--\r\n'

Environment:

$ yum list installed | grep httpd
httpd.x86_64                     2.4.6-2.fc18       @updates                    
httpd-tools.x86_64               2.4.6-2.fc18       @updates                    
libmicrohttpd.x86_64             0.9.22-1.fc18      @koji-override-0/$releasever
$ yum list installed | grep mod_
mod_security.x86_64              2.7.3-2.fc18       @updates                    
mod_ssl.x86_64                   1:2.4.6-2.fc18     @updates          

The form submission is valid, with this an obvious false positive. The workaround is to disable the rule. However, it would be better if this was catered for an didn't trigger the rule in the first place.

@Hadrianus

This comment has been minimized.

Copy link

commented Feb 13, 2014

Even a parameter with a value starting with "--" causes the request being denied.
A sample curl command:

curl https://localhost/app/ -Fparam=--value
@akrherz

This comment has been minimized.

Copy link

commented Apr 2, 2014

any hopes of triaging this and getting it fixed? This is causing us great grief at the moment :)

@rcbarnett

This comment has been minimized.

Copy link
Contributor

commented Apr 2, 2014

Is this the rule causing the alert?
https://github.com/SpiderLabs/ModSecurity/blob/master/modsecurity.conf-recommended#L83-86
If so, you might want to disable it in the meantime until we can investigate.

@akrherz

This comment has been minimized.

Copy link

commented Apr 2, 2014

Thanks for the response. Yes that is the rule and we are not permitted to disable the rule :(

@rcbarnett

This comment has been minimized.

Copy link
Contributor

commented Apr 2, 2014

Are you not the website (ModSecurity) owner?

@akrherz

This comment has been minimized.

Copy link

commented Apr 2, 2014

Nope, just the developer

@ghost

This comment has been minimized.

Copy link

commented Jul 30, 2014

I am affected by this bug as well.
A false positive trigger occurs when uploading a pdf which contains a line that starts with '--abc'.

The rule that triggers this is:
https://github.com/SpiderLabs/ModSecurity/blob/master/modsecurity.conf-recommended#L83-86

@akrherz

This comment has been minimized.

Copy link

commented Nov 20, 2014

Any updated comments on resolving this bug? :)

@AndreyMZ

This comment has been minimized.

Copy link

commented Feb 17, 2015

I am not familiar with Modsecurity code, but...
It is an obvious bug in Modsecurity in defining MULTIPART_UNMATCHED_BOUNDARY variable. And it have taken 2 minutes for me to locate it.
Could you fix it please, or correct me if I am wrong?

Problem description:
If all boundaries are OK but POST data contains "--" at the begining of line, then MULTIPART_UNMATCHED_BOUNDARY is wrongly set to 1.

Affected versions:
Modsecurity 2.9.0 (downloaded from https://www.modsecurity.org/tarball/2.9.0/modsecurity-2.9.0.tar.gz).

Root cause:
File modsecurity-2.9.0\apache2\msc_multipart.c, function multipart_process_chunk, line 23.

/* Do we have something that looks like a boundary? */
if (   msr->mpd->buf_contains_line
    && (strlen(msr->mpd->buf) > 3)
    && (*(msr->mpd->buf) == '-')
    && (*(msr->mpd->buf + 1) == '-') )
{
    /* Does it match our boundary? */
    if (   (strlen(msr->mpd->buf) >= strlen(msr->mpd->boundary) + 2)
        && (strncmp(msr->mpd->buf + 2, msr->mpd->boundary, strlen(msr->mpd->boundary)) == 0) )
    {
        // ...
    } else { /* It looks like a boundary but we couldn't match it. */
        // ...
        msr->mpd->flag_unmatched_boundary = 1;
    }
} else { /* We do not think the buffer contains a boundary. */

Possible soulution:
Maybe just this?

         }

-        msr->mpd->flag_unmatched_boundary = 1;
     }
 } else { /* We do not think the buffer contains a boundary. */
@manuel-sousa

This comment has been minimized.

Copy link

commented Mar 30, 2015

Same thing as rutgerbazen, with a couple of PDF uploads this rule triggers and had to disable it since the PDF encoding is out of our control.

@driehuis

This comment has been minimized.

Copy link

commented Jun 6, 2015

@AndreyMZ: This rule does what it says on the tin. Your suggested fix would essentially remove the entire check. The MULTIPART_UNMATCHED_BOUNDARY check is intended to detect uploads that would cause problems for a web application that does not properly check the full MIME boundary. If you are 100% confident that your application properly checks MIME boundaries, just disable the rule in your modsecurity.conf with

SecRuleRemoveById 200003

Remember that many rules that try to protect against common programming errors will also trip on legitimate traffic, and as a security administrator, you will have to make the judgement call on whether or not the added security of keeping the rule enabled is worth the occasional false positive (or indeed, that you are sure that this particular programming error was not made anywhere on your system).

I have personally seen badly written MIME parsers that fail when processing a file that contains a newline-dash-dash sequence, so I have seen the use case for having this rule first hand.

@goyocasero

This comment has been minimized.

Copy link

commented Nov 8, 2016

We are experiencing the same issue (some uploaded files are being denied by this rule).
The log:
Message: Access denied with code 403 (phase 2). Match of "eq 0" against "MULTIPART_UNMATCHED_BOUNDARY" required. [file "E:/Apache2.2.31/conf/httpd.conf"] [line "561"] [id "000003"] [msg "Multipart parser detected a possible unmatched boundary."]

and the rule:
SecRule MULTIPART_UNMATCHED_BOUNDARY "!@eq 0" \ "phase:2,id:'000003',t:none,log,deny,msg:'Multipart parser detected a possible unmatched boundary.'"

Some files are being denied but some others not...

We're using modsecurity version 2.8.0 in Apache2.2.31 and I'm finding this issue as opened, so I'm wondering if this could be the reason of our problem. If it's already resolved, anyone has any clue?

@AndreyMZ

This comment has been minimized.

Copy link

commented Nov 8, 2016

The solution is to disable this rule (it's id is 200003).

According to @driehuis, this rule does what it should do.
So the problem is that the documentation is not clear. It should say that this rule in particular blocks uploading of some legitimate files, and it should be disabled unless some badly written MIME parser is used.

@victorhora

This comment has been minimized.

Copy link
Member

commented Feb 27, 2018

Documentation for MULTIPART_UNMATCHED_BOUNDARY which is checked by that rule currently states the following:

"Change the rule from blocking to logging-only if many false positives are encountered."

Also, the check with MULTIPART_UNMATCHED_BOUNDARY variable used to have ID 200003. The current version of modsecurity.conf-recommended uses ID 200004 for the same check.

This was changed / updated with ModSecurity 2.8.x and it's worth upgrading the modsecurity.conf as there's a number of fixes and improvements since 2.7.9

@iamenderst

This comment has been minimized.

Copy link

commented Jul 2, 2018

Also happens on 3.x. For 3.x, instead of disabling 200003 or 200004, why not change 200004 to allow and log? Works for me.
Change:
SecRule MULTIPART_UNMATCHED_BOUNDARY "!@eq 0" \ "id:'200004',phase:2,t:none,log,deny,msg:'Multipart parser detected a possible unmatched boundary.'"
To:
SecRule MULTIPART_UNMATCHED_BOUNDARY "!@eq 0" \ "id:'200004',phase:2,t:none,log,allow,msg:'Multipart parser detected a possible unmatched boundary.'"

@victorhora

This comment has been minimized.

Copy link
Member

commented Aug 30, 2018

@iamenderst, I believe this should be solved as of 7def498.

Check with the latest code from master. If it's still an issue please let us know and we'll have another look. Thanks!

@defanator

This comment has been minimized.

Copy link
Contributor

commented Nov 1, 2018

@victorhora @zimmerle just got a similar issue with a simplest multipart payload:

root@vagrant:/var/log/nginx# curl -iv -F param1=1 http://localhost/modsec-full/
*   Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 80 (#0)
> POST /modsec-full/ HTTP/1.1
> Host: localhost
> User-Agent: curl/7.47.0
> Accept: */*
> Content-Length: 142
> Expect: 100-continue
> Content-Type: multipart/form-data; boundary=------------------------9fef5902ba3178e2
> 
< HTTP/1.1 100 Continue
HTTP/1.1 100 Continue

< HTTP/1.1 403 Forbidden
HTTP/1.1 403 Forbidden
< Server: nginx/1.15.5
Server: nginx/1.15.5
< Date: Thu, 01 Nov 2018 15:36:01 GMT
Date: Thu, 01 Nov 2018 15:36:01 GMT
< Content-Type: text/html
Content-Type: text/html
< Content-Length: 153
Content-Length: 153
< Connection: keep-alive
Connection: keep-alive

* HTTP error before end of send, stop sending
< 
<html>
<head><title>403 Forbidden</title></head>
<body>
<center><h1>403 Forbidden</h1></center>
<hr><center>nginx/1.15.5</center>
</body>
</html>
* Closing connection 0

Error log:

2018/11/01 08:36:01 [warn] 28095#28095: *3 [client 127.0.0.1] ModSecurity: Access denied with code 403 (phase 2). Matched "Operator `Eq' with parameter `0' against variable `MULTIPART_UNMATCHED_BOUNDARY' (Value: `2' ) [file "/etc/nginx/modsec/modsecurity.conf"] [line "15"] [id "200004"] [rev ""] [msg "Multipart parser detected a possible unmatched boundary."] [data ""] [severity "0"] [ver ""] [maturity "0"] [accuracy "0"] [hostname "127.0.0.1"] [uri "/modsec-full/"] [unique_id "154108656148.310734"] [ref "v206,1"], client: 127.0.0.1, server: , request: "POST /modsec-full/ HTTP/1.1", host: "localhost"

I'm on a recent v3/master, nginx/1.15.5, connector from master.

@zimmerle

This comment has been minimized.

Copy link
Member

commented Nov 1, 2018

Hi @defanator,

Please check #1924.

@airween

This comment has been minimized.

Copy link
Contributor

commented Nov 2, 2018

Issue of @defanator is valid, the well-formed multipart upload is denied with "!@eq 0" operator.
There is a PR - should I sent it to directly to ModSecurity (instead of @victorhora)?

@chrisconlan

This comment has been minimized.

Copy link

commented Jan 8, 2019

Same issue here.

A sequence of dashes -------- in markdown is shorthand for <hr>. My users are triggering eq 0 when writing markdown.

@airween

This comment has been minimized.

Copy link
Contributor

commented Jan 8, 2019

@chrisconlan could you show an example - headers, content and your relevant config?

@chrisconlan

This comment has been minimized.

Copy link

commented Jan 8, 2019

OS: AWS Linux
Apache version: httpd24.x86_64 2.4.37-1.83.amzn1 @amzn-updates
Mod security version: mod24_security.x86_64 2.8.0-5.27.amzn1 @amzn-main

HTML form sending POST processing as multipart/form-data. Only <textarea> and <input> tags. No files.

That's the best I can do as for details right now. Hope it helps.

@airween

This comment has been minimized.

Copy link
Contributor

commented Jan 8, 2019

Yes, helped - sorry, 2.8 is (I guess) a little bit old version, I don't know that the fix exists for that... :(

@chrisconlan

This comment has been minimized.

Copy link

commented Jan 8, 2019

No problem. Thanks.

@dwreski

This comment has been minimized.

Copy link

commented Feb 3, 2019

I'm seeing this problem with 2.9.2 on fedora28. I've had this version installed since October, but only started seeing the error today after changing the virtual domain name. It looks like this was built back in Feb 2018. Is there something I can do to fix this without upgrading?

[Sun Feb 03 11:31:58.614670 2019] [:error] [pid 9771:tid 140119080138496] [client 185.136.156.195:40896] [client 185.136.156.195] ModSecurity: Access denied with code 44 (phase 2). Match of "eq 0" against "MULTIPART_UNMATCHED_BOUNDARY" required. [file "/etc/httpd/conf.d/mod_security.conf"] [line "32"] [id "200003"] [msg "Multipart parser detected a possible unmatched boundary."] [hostname "www.linuxsecurity.com"] [uri "/index.php"] [unique_id "XFcXfoXhqCfzhUP5s9byowAAAIw"], referer: https://www.linuxsecurity.com/index.php?option=com_content&task=new&sectionid=8&itemid=0

# rpm -qva --last |grep -E 'mod_security|httpd' mod_security-2.9.2-5.fc28.x86_64 Tue 09 Oct 2018 10:11:44 AM EDT httpd-2.4.34-3.fc28.x86_64 Sat 29 Sep 2018 05:58:50 PM EDT fedora-logos-httpd-28.0.3-1.fc28.noarch Sat 29 Sep 2018 05:58:49 PM EDT httpd-tools-2.4.34-3.fc28.x86_64 Sat 29 Sep 2018 05:58:44 PM EDT httpd-filesystem-2.4.34-3.fc28.noarch Sat 29 Sep 2018 05:58:43 PM EDT

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.