-
Notifications
You must be signed in to change notification settings - Fork 725
JSON Payloads process significantly slower (600%) than XML Payloads of a similar size and format #1728
Comments
The question buried in here is: Is there any significant set of rules which runs by default on JSON, but not XML-parsed fields? If not (and I couldn't find any), I am at a total loss to explain the roughly 650% increase in processing time in JSON compared to XML requests |
Question for the CRS team now: Why do JSON Lists cause so much latency with CRS rules? |
Payload for the "Fast" JSON test outlined above:
|
I'm afraid we can't help with the available information. The JSON contents above are very different: the first one has so many keys, and all child have more child items. The second one has 10 keys (as I see) without any sub-child. If a rule has an argument Yes, you're right - the XML content doesn't trigger this latency - I have no idea why. But I think this means this issue is not CRS related. You should start to turn off each rule set, start with 901. If this modification has no effect, turn back and take next one. Iterate this while you get a better result. Then you found the source of your problem, and can check each rule in that file. |
Thanks @airween ! Agree, there's not enough info yet to draw conclusions. For now, my current theory is that something about lists with multiple items, multiple nested keys, might cause high latency in some particular rule(s). I'm pretty sure XML vs JSON doesn't really matter. I just happened to run my original test with a particularly "bad" JSON. I ran a "List" version of this payload earlier today too, and that did not produce similar high latency. I also attempted running Modsecurity with CRS disabled on the original "bad" JSON, and that too did not produce high latency - that's the only reason I was eyeing something in CRS. Will attempt to identify which rule(s) seem to take excess time on the "bad" JSON. |
I made some researches, let me share with you the results. I sent all of three payloads above to my test Nginx with The first column in the line is the line number in file (all requests logged into same file). The first line of three pairs is the beginning of random rule, the last is same with next one. Just see the difference between the first and last line numbers. That means, how many steps required to execute a rule.
3861 lines.
15 lines.
129 lines. The other lines between two quotes lines above contains all steps, including transformations, and all operator evaluations. Please check these values at your side - I think you can see the reason, why is so slowly the JSON payload what you included. Hope this helps. |
@airween thank you for the analysis! Just one last Q - is this behavior consistent between all rules for the "High Latency" JSON? Or just for the |
No, not just for the rule It's consistent between all rules (which affected on your chosed PL), just check your log. Just check your |
Describe the bug
In comparison testing of different payload types while running the OWASP CRS Ruleset 3.2, with Modsecurity 3.0.4 on nginx; we've noticed a pretty huge discrepancy in processing times between JSON and XML payloads.
Please find a screenshot of a reasonable comparison, as well as the raw JSON and XML payloads below.
Summary of above graphic:
Important Note: We've ran a test of Modsecurity without the CRS enabled, but still including Modsecurity request parsing of both MIMEs and have confirmed little difference between the parsing times for JSON and XML; so that leads us to the CRS.
Steps to reproduce
With both
XML
andJSON
request body processing types enabled, send requests using the following XML and JSON payloads to an Nginx webserver running OWASP CRS Ruleset 3.2, with Modsecurity 3.0.4.The discrepancy in processing times between JSON and XML payloads is reproducible with any throughput.
XML Payload
JSON Payload
The text was updated successfully, but these errors were encountered: