MongoDB 4.4 logs are in .json format and not recognized

[julianfortune]

Environment:

• Fail2Ban version (including any possible distribution suffixes): Fail2Ban v0.11.1
• OS, including release name/version: Ubuntu 20.04
• Fail2Ban installed via OS/distribution mechanisms
• You have not applied any additional foreign patches to the codebase
• Some customizations were done to the configuration (provide details below is so)

The issue:

MongoDB 4.4 introduced output logging in structures json format. This change in formatting means the existing failure regex no longer recognizes failures.

Steps to reproduce

• Install MongoDB 4.4
• Install fail2ban (sudo apt install fail2ban)
• Enable the default mongodb-auth filter.

  [mongodb-auth]
  enabled   = true
  filter    = mongodb-auth
  

Expected behavior

{"t":{"$date":"2021-02-11T01:38:33.454+00:00"},"s":"I",  "c":"ACCESS",   "id":"00000",   "ctx":"conn14","msg":"Authentication failed","attr":{"mechanism":"SCRAM-SHA-256","principalName":"user","authenticationDatabase":"admin","client":"100.0.0.0:45005","result":"AuthenticationFailed: SCRAM authentication failed, storedKey mismatch"}}

Should be recognized by the filter as a failed login.

Observed behavior

Not recognized by the filter.

[sebres]

<breathe-in>
Why must devs continuously change the log format, moreover activate structured logging by default...
</breathe-out>

But, OK...

[Definition]

datepattern = ^\{"t":\{"\$date":"%%Y-%%m-%%dT%%H:%%M:%%S\.%%f%%z"}\s*,\s*
_groupre = (?:"(?!(?:msg|attr|client|remote)\b)\w+":(?:"[^"]+"|\w+)\s*[,\}]\s*)
failregex = ^%(_groupre)s*"msg":"Authentication failed"\s*,\s*%(_groupre)s*"attr"\s*:\s*\{%(_groupre)s*"(?:client|remote)":"<ADDR>:\d+"

^{(RE adjusted to consider #3046 log format - parses both client and remote tags)}

Just... it is a bit weak (since IP is in nested dict after all that foreign input). Sure PCRE allows nesting (recursive parsing rules), but it's somehow ugly (especially if order of tags is not specified).
By the way, I think, because logging in json became trendy now, we can implement json parsing in filter (with some rules to map tags to fail-info).

[julianfortune]

Thank you @sebres ! I really appreciate the quick help.

I think JSON parsing makes sense! Let me know if I can do anything to help.

[raidensakura]

Apologies for bumping a 2 years old issue but this is now an issue again with latest MongoDB image from Docker, currently 6.0.4. The log format has changed again hence breaking the filter.

{"t":{"$date":"2023-02-03T18:47:55.198+00:00"},"s":"I",  "c":"ACCESS",   "id":20249,   "ctx":"conn101","msg":"Authentication failed","attr":{"mechanism":"SCRAM-SHA-256","speculative":true,"principalName":"raidensakura","authenticationDatabase":"admin","remote":"0.0.0.0:62189","extraInfo":{},"error":"AuthenticationFailed: SCRAM authentication failed, storedKey mismatch"}}

[sebres]

The log format has changed again hence breaking the filter.

I don't see it is changed again. The filter from #2932 (comment) still find a match for your example log message.