Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

AuditLog parity with libmodsecurity3 #856

Open
Tracked by #945
amsnek opened this issue Aug 5, 2023 · 7 comments
Open
Tracked by #945

AuditLog parity with libmodsecurity3 #856

amsnek opened this issue Aug 5, 2023 · 7 comments
Labels
help wanted Extra attention is needed v3.1

Comments

@amsnek
Copy link

amsnek commented Aug 5, 2023

Summary

Corazas current AuditLog format in json has empty values for important fields and is overall less detailed compared to libmodsecurity3. Since the stated goal is to be a "drop in replacement" for modsecurity, logging parity would be beneficial.

Basic example

Below are AuditLogs from coraza-spoa ccompared to libmodsecurity3 with the following http request intended to trigger the WAF.
curl -v http://127.0.0.100/?x\=/etc/passwd

AuditLog generated from coraza-spoa

{
  "transaction": {
    "timestamp": "2023/08/05 13:58:06",
    "unix_timestamp": 1691236686919190605,
    "id": "622c3c2b-073f-43c8-9878-657be5d8a69f",
    "client_ip": "127.0.0.1",
    "client_port": 6203,
    "host_ip": "127.0.0.100",
    "host_port": 80,
    "server_id": "",
    "request": {
      "method": "",
      "protocol": "",
      "uri": "",
      "http_version": "",
      "headers": {
        "accept": [
          "*/*"
        ],
        "host": [
          "127.0.0.100"
        ],
        "user-agent": [
          "curl/7.61.1"
        ]
      },
      "body": "",
      "files": null
    },
    "response": {
      "protocol": "",
      "status": 0,
      "headers": {},
      "body": ""
    },
    "producer": {
      "connector": "",
      "version": "",
      "server": "",
      "rule_engine": "On",
      "stopwatch": "1691236686919190605 1091256; combined=1046168, p1=360327, p2=646155, p3=0, p4=0, p5=39686",
      "rulesets": [
        "OWASP_CRS/4.0.0-rc1"
      ]
    }
  }
}

There are details in coraza-spoa default error log but this is only partially json and not compareable to the auditlog:

{
  "level": "error",
  "ts": 1691236686.9202244,
  "msg": "[client \"127.0.0.1\"] Coraza: Access denied (phase 2). Inbound Anomaly Score Exceeded (Total Score: 13) [file \"/path/to/rules/REQUEST-949-BLOCKING-EVALUATION.conf\"] [line \"11001\"] [id \"949110\"] [rev \"\"] [msg \"Inbound Anomaly Score Exceeded (Total Score: 13)\"] [data \"\"] [severity \"emergency\"] [ver \"OWASP_CRS/4.0.0-rc1\"] [maturity \"0\"] [accuracy \"0\"] [tag \"anomaly-evaluation\"] [hostname \"127.0.0.100\"] [uri \"/?x=/etc/passwd\"] [unique_id \"622c3c2b-073f-43c8-9878-657be5d8a69f\"]\n"
}

AuditLog from libmodsecurity3

  "transaction": {
    "client_ip": "127.0.0.1",
    "time_stamp": "Thu Aug  3 22:42:14 2023",
    "server_id": "5dab6b80f157",
    "client_port": 0,
    "host_ip": "127.0.0.100",
    "host_port": 80,
    "unique_id": "d181c234abcd1be01421",
    "request": {
      "method": "GET",
      "http_version": 1.1,
      "uri": "/?x=/etc/passwd",
      "body": "",
      "headers": {
        "host": "example.tld",
        "user-agent": "curl/7.61.1",
        "accept": "*/*",
        "x-forwarded-for": "127.0.0.1"
      }
    },
    "response": {
      "body": "<!doctype html>\n<head><meta charset=\"utf-8\">\n<title>403 Forbidden</title>\n</head>\n<style> .... ",
      "http_code": 403,
      "headers": {
        "Server": "nginx",
        "Date": "Thu, 03 Aug 2023 20:42:14 GMT",
        "Content-Type": "text/html",
        "Connection": "keep-alive"
      }
    },
    "producer": {
      "modsecurity": "ModSecurity v3.0.10 (Linux)",
      "connector": "ModSecurity-nginx v1.0.3",
      "secrules_engine": "Enabled",
      "components": [
        "OWASP_CRS/3.3.5\""
      ]
    },
    "messages": [
      {
        "message": "OS File Access Attempt",
        "details": {
          "match": "Matched \"Operator `PmFromFile' with parameter `lfi-os-files.data' against variable `ARGS:x' (Value: `/etc/passwd' )",
          "reference": "o1,10v8,11t:utf8toUnicode,t:urlDecodeUni,t:normalizePathWin,t:lowercase",
          "ruleId": "930120",
          "file": "/path/to/rules/REQUEST-930-APPLICATION-ATTACK-LFI.conf",
          "lineNumber": "79",
          "data": "Matched Data: etc/passwd found within ARGS:x: /etc/passwd",
          "severity": "2",
          "ver": "OWASP_CRS/3.3.5",
          "rev": "",
          "tags": [
            "application-multi",
            "language-multi",
            "platform-multi",
            "attack-lfi",
            "paranoia-level/1",
            "OWASP_CRS",
            "capec/1000/255/153/126",
            "PCI/6.5.4"
          ],
          "maturity": "0",
          "accuracy": "0"
        }
      },
      {
        "message": "Remote Command Execution: Unix Shell Code Found",
        "details": {
          "match": "Matched \"Operator `PmFromFile' with parameter `unix-shell.data' against variable `ARGS:x' (Value: `/etc/passwd' )",
          "reference": "o1,10v8,11t:urlDecodeUni,t:cmdLine,t:normalizePath,t:lowercase",
          "ruleId": "932160",
          "file": "/path/to/rules/REQUEST-932-APPLICATION-ATTACK-RCE.conf",
          "lineNumber": "481",
          "data": "Matched Data: etc/passwd found within ARGS:x: /etc/passwd",
          "severity": "2",
          "ver": "OWASP_CRS/3.3.5",
          "rev": "",
          "tags": [
            "application-multi",
            "language-shell",
            "platform-unix",
            "attack-rce",
            "paranoia-level/1",
            "OWASP_CRS",
            "capec/1000/152/248/88",
            "PCI/6.5.2"
          ],
          "maturity": "0",
          "accuracy": "0"
        }
      },
      {
        "message": "Inbound Anomaly Score Exceeded (Total Score: 10)",
        "details": {
          "match": "Matched \"Operator `Ge' with parameter `5' against variable `TX:ANOMALY_SCORE' (Value: `10' )",
          "reference": "",
          "ruleId": "949110",
          "file": "/path/to/rules/REQUEST-949-BLOCKING-EVALUATION.conf",
          "lineNumber": "81",
          "data": "",
          "severity": "2",
          "ver": "OWASP_CRS/3.3.5",
          "rev": "",
          "tags": [
            "application-multi",
            "language-multi",
            "platform-multi",
            "attack-generic"
          ],
          "maturity": "0",
          "accuracy": "0"
        }
      }
    ]
  }
}

Motivation

Detailed Logs in JSON format make it easy to store and analyze the logs in tools like elasticsearch or similiar. This increases the visibility of attacks, aids with filtering of false positives and greatly improves overall usefullness.
@jptosso
Copy link
Member

jptosso commented Aug 7, 2023 •
edited
Loading

Hey @amsnek , actually, Coraza supports a format called JSONLEGACY that is compatible with ModSecurity.

coraza/internal/auditlog/legacy.go

Line 9 in 4f30afe

type logLegacy struct {

Just replace the JSON format in SecAuditLogFormat with JSONLEGACY.

SecAuditLogFormat JSONLEGACY

@amsnek
Copy link
Author

amsnek commented Aug 7, 2023 

SecAuditLogFormat JSONLEGACY

Hey @jptosso
I tried with "SecAuditLogFormat JSONLEGACY" but the output did not change at all? 🤔

{
  "transaction": {
    "time": "2023/08/07 10:02:15",
    "transaction_id": "c6c51035-6dc0-4edf-9e27-f9328397f04d",
    "remote_address": "127.0.0.1",
    "remote_port": 44113,
    "local_address": "127.0.0.1",
    "local_port": 80
  },
  "request": {
    "request_line": "  ",
    "headers": {
      "accept": "*/*",
      "host": "127.0.0.1",
      "user-agent": "curl/7.61.1"
    }
  },
  "response": {
    "status": 0,
    "protocol": "",
    "headers": {}
  },
  "audit_data": {
    "messages": null,
    "error_messages": null,
    "handler": "",
    "stopwatch": {
      "Combined": 0,
      "P1": 0,
      "P2": 0,
      "P3": 0,
      "P4": 0,
      "P5": 0,
      "Sr": 0,
      "Sw": 0,
      "L": 0,
      "Gc": 0
    },
    "response_body_dechunked": false,
    "producer": [
      "OWASP_CRS/4.0.0-rc1"
    ],
    "server": "",
    "engine_mode": "On"
  }
}

@jptosso
Copy link
Member

jptosso commented Aug 21, 2023

Apparently jsonlegacy is using modsecurity 2 format. We would need another JSON formatter for libmodsecurity

@jptosso jptosso mentioned this issue Aug 21, 2023
Closed
@amsnek amsnek mentioned this issue Aug 21, 2023
Closed
@amsnek
Copy link
Author

amsnek commented Aug 21, 2023

modsecurity2 was only "partial json". libmodsecurity3 implemented full json support. a "full" json formatter for libmodsecurity would be most usefull.

@jcchavezs jcchavezs added v3.1 help wanted Extra attention is needed labels Aug 30, 2023
@jcchavezs
Copy link
Member

Up to work on this @amsnek ?

@amsnek
Copy link
Author

amsnek commented Sep 26, 2023

hey @jcchavezs
as mentioned in the august 2023 meeting, sadly I do not have the required skills to implement this :(

@jptosso jptosso mentioned this issue Dec 19, 2023
Open
16 tasks
@M4tteoP
Copy link
Member

M4tteoP commented Apr 4, 2024

Just to keep tracking the evolution of this issue: #968 implemented some missing information

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
help wanted Extra attention is needed v3.1
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@jptosso @jcchavezs @M4tteoP @amsnek