Improvement/cscli alerts inspect show the matched part

[pierrehenrymuller]

Is your feature request related to a problem? Please describe.
I have a case for alert with crowdsecurity/http-bad-user-agent for an IP of Google. So I think it is a false positive and I launch an cscli alerts inspect ID -d to find details. But I can't see where is the problem, I have IP and metadata but I don't have in which log file the match occured and the matched string.

 - ID         : 46
 - Date       : 2021-02-18T22:55:26+01:00
 - Machine    : XXX
 - Simulation : false
 - Reason     : crowdsecurity/http-bad-user-agent
 - Events Count : 2
 - Scope:Value: Ip:::ffff:35.243.255.156
 - Country    : US
 - AS         : Google LLC

 - Active Decisions  :
+-----+--------------------------+--------+------------------+---------------------------+
| ID  |       SCOPE:VALUE        | ACTION |    EXPIRATION    |        CREATED AT         |
+-----+--------------------------+--------+------------------+---------------------------+
| 535 | Ip:::ffff:35.243.255.156 | ban    | 2h25m22.6607429s | 2021-02-18T22:55:26+01:00 |
+-----+--------------------------+--------+------------------+---------------------------+

 - Events  :

- Date: 2021-02-18 22:55:25 +0100 +0100
+---------------+-----------------------+
|      KEY      |         VALUE         |
+---------------+-----------------------+
| service       | http                  |
| IsoCode       | US                    |
| IsInEU        | false                 |
| ASNNumber     |                 15169 |
| ASNOrg        | Google LLC            |
| http_args_len |                     3 |
| log_type      | http_access-log       |
| source_ip     | ::ffff:35.243.255.156 |
| http_status   |                   200 |
| http_path     | /xmlrpc.php?rsd       |
| SourceRange   | 35.240.0.0/13         |
+---------------+-----------------------+

- Date: 2021-02-18 22:55:25 +0100 +0100
+---------------+-----------------------+
|      KEY      |         VALUE         |
+---------------+-----------------------+
| http_status   |                   304 |
| http_path     | /home/feed/           |
| IsInEU        | false                 |
| ASNOrg        | Google LLC            |
| SourceRange   | 35.240.0.0/13         |
| http_args_len |                     0 |
| log_type      | http_access-log       |
| service       | http                  |
| ASNNumber     |                 15169 |
| source_ip     | ::ffff:35.243.255.156 |
| IsoCode       | US                    |
+---------------+-----------------------+

Describe the solution you'd like
It will better to see the one or multiple logs files who is concerned about this alerts and the matched string of the scenarios.

Exemple for this real case :

 - ID         : 46
 - Date       : 2021-02-18T22:55:26+01:00
 - Machine    : XXX
 - Simulation : false
 - Reason     : crowdsecurity/http-bad-user-agent
 - Events Count : 2
 - Scope:Value: Ip:::ffff:35.243.255.156
 - Country    : US
 - AS         : Google LLC

 - Active Decisions  :
+-----+--------------------------+--------+------------------+---------------------------+
| ID  |       SCOPE:VALUE        | ACTION |    EXPIRATION    |        CREATED AT         |
+-----+--------------------------+--------+------------------+---------------------------+
| 535 | Ip:::ffff:35.243.255.156 | ban    | 2h25m22.6607429s | 2021-02-18T22:55:26+01:00 |
+-----+--------------------------+--------+------------------+---------------------------+

 - Events  :

- Date: 2021-02-18 22:55:25 +0100 +0100
+---------------+-----------------------+
|      KEY      |         VALUE         |
+---------------+-----------------------+
| service       | http                  |
| IsoCode       | US                    |
| IsInEU        | false                 |
| ASNNumber     |                 15169 |
| ASNOrg        | Google LLC            |
| http_args_len |                     3 |
| log_type      | http_access-log       |
| source_ip     | ::ffff:35.243.255.156 |
| http_status   |                   200 |
| http_path     | /xmlrpc.php?rsd       |
| SourceRange   | 35.240.0.0/13         |
| matched_file | /var/www/customer1/www.website1.com/logs/access.log |
| matched_string | ZoominfoBot (zoominfobot at zoominfo dot com) |
+---------------+-----------------------+

- Date: 2021-02-18 22:55:25 +0100 +0100
+---------------+-----------------------+
|      KEY      |         VALUE         |
+---------------+-----------------------+
| http_status   |                   304 |
| http_path     | /home/feed/           |
| IsInEU        | false                 |
| ASNOrg        | Google LLC            |
| SourceRange   | 35.240.0.0/13         |
| http_args_len |                     0 |
| log_type      | http_access-log       |
| service       | http                  |
| ASNNumber     |                 15169 |
| source_ip     | ::ffff:35.243.255.156 |
| IsoCode       | US                    |
| matched_file | /var/www/customer2/www.website2.com/logs/access.log |
| matched_string | ZoominfoBot (zoominfobot at zoominfo dot com) |
+---------------+-----------------------+

In this case with the two extra matched_file and matched_string sysops can investigate more quickly that in this case this a bad user agent for Google / Google bot are not blocked.

What do you think about this?

[buixor]

Hello,

Sorry, completely forgot to answer you 😬
Currently, what appears or not in the cscli alerts inspect output for a given alert depends on the content that was in the Meta dictionary of the events that lead to the given alert. That's why you will see the file paths, but not the bad user-agent (in your example).

It's something that we plan to address in the following way, let me know what you think 😄

Scenarios will allow to specify fields from events (such as evt.Parsed.toto) that needs to be collected an kept. In this way, one can decide what is relevant in the alert when writing the scenario (while know it's unfortunately the parser that decides that).

I take your idea of having the exact source of the event as well, as it's probably very useful !

[buixor]

Hello, better late than never 😄

We have merged "alert context" in 1.5 which solves this issue: https://doc.crowdsec.net/docs/next/user_guides/alert_context

It allows you to specify any variable/expression etc. to be embedded with the alert, and view them ie. in cscli alerts inspect (and soon to view them in the console too)