From a6272c92ebf2954d790855da74e7c2153ded7012 Mon Sep 17 00:00:00 2001 From: Thibault Koechlin Date: Tue, 30 Sep 2025 11:45:14 +0200 Subject: [PATCH 1/7] clarify intent --- crowdsec-docs/unversioned/console/remediation_metrics.mdx | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/crowdsec-docs/unversioned/console/remediation_metrics.mdx b/crowdsec-docs/unversioned/console/remediation_metrics.mdx index 6a02a6ddd..03d60e12a 100644 --- a/crowdsec-docs/unversioned/console/remediation_metrics.mdx +++ b/crowdsec-docs/unversioned/console/remediation_metrics.mdx @@ -18,7 +18,7 @@ It provides key insights into: - The blocklist that contribute to remediate malicious traffic The page is divided into three main sections: -- **Malicious Intents** – A breakdown of attack types associated over time and the total number of attacks prevented. +- **Malicious Intents** – A breakdown of attack types associated with the IPs that your remediation components blocked. - **Malicious Traffic Dropped/Discarded** – Raw and estimated data showing how much malicious traffic has been dropped by your remediation components. - **Projected Resources Saved** – An estimate of the resources preserved thanks to traffic being dropped (e.g., storage, bandwidth, log volume). @@ -30,7 +30,7 @@ At the top of the page, you'll see the **Total Prevented Attacks** for the selec ![Total Prevented Attacks](/img/console/remediation_metrics/rc-metrics-total-prevented-attacks.png) -The **Malicious Intents** section provides a detailed breakdown of the types of attacks that were prevented. These are based on the behavior and typology of IPs remediated by your remediation components, including blocklists and security engines. +The **Malicious Intents** section provides a detailed breakdown of the typical behaviour associated with the blocked IPs. It shows the behaviour and typology of IPs remediated by your remediation components, including blocklists and security engines. The displayed intent might differ from the behaviours observed from this IP in your context, as it reflects the most commonly observed attacks by the CrowdSec network for a given IP. ![Malicious Intents Breakdown](/img/console/remediation_metrics/rc-metrics-malicious-intents.png) From dc0c759c679d06be4c6d10b9407955844bbd0d68 Mon Sep 17 00:00:00 2001 From: Thibault Koechlin Date: Tue, 30 Sep 2025 13:56:01 +0200 Subject: [PATCH 2/7] up --- crowdsec-docs/unversioned/console/remediation_metrics.mdx | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/crowdsec-docs/unversioned/console/remediation_metrics.mdx b/crowdsec-docs/unversioned/console/remediation_metrics.mdx index 03d60e12a..860bad121 100644 --- a/crowdsec-docs/unversioned/console/remediation_metrics.mdx +++ b/crowdsec-docs/unversioned/console/remediation_metrics.mdx @@ -30,7 +30,9 @@ At the top of the page, you'll see the **Total Prevented Attacks** for the selec ![Total Prevented Attacks](/img/console/remediation_metrics/rc-metrics-total-prevented-attacks.png) -The **Malicious Intents** section provides a detailed breakdown of the typical behaviour associated with the blocked IPs. It shows the behaviour and typology of IPs remediated by your remediation components, including blocklists and security engines. The displayed intent might differ from the behaviours observed from this IP in your context, as it reflects the most commonly observed attacks by the CrowdSec network for a given IP. +The **Malicious Intents** section summarizes what a blocked IP was *likely* attempting (for example spam, scanning, or brute-force), based on behaviour patterns observed across the CrowdSec network. Because these IPs are blocked, we may not have direct evidence of what they tried in **your** environment. Treat the intent as an informed estimate derived from community data rather than a record of actions on your system. + +> **Note:** We cannot determine the exact action an IP took against your service. The displayed intent is an estimate inferred from broader network observations. ![Malicious Intents Breakdown](/img/console/remediation_metrics/rc-metrics-malicious-intents.png) From f84d9396ce369e5c48384eee82e78441581d0dd7 Mon Sep 17 00:00:00 2001 From: Thibault Koechlin Date: Tue, 30 Sep 2025 14:45:17 +0200 Subject: [PATCH 3/7] up --- crowdsec-docs/docs/appsec/rules_deploy.md | 100 ++++++++++++++++++++++ crowdsec-docs/sidebars.ts | 3 +- 2 files changed, 102 insertions(+), 1 deletion(-) create mode 100644 crowdsec-docs/docs/appsec/rules_deploy.md diff --git a/crowdsec-docs/docs/appsec/rules_deploy.md b/crowdsec-docs/docs/appsec/rules_deploy.md new file mode 100644 index 000000000..ae493467e --- /dev/null +++ b/crowdsec-docs/docs/appsec/rules_deploy.md @@ -0,0 +1,100 @@ +--- +id: rules_deploy +title: Custom Rules deployment +sidebar_position: 81 +--- + +# WAF Rules Deployment + +This guide starts once you have already authored and validated a custom AppSec (WAF) rule locally. The steps below focus on packaging that rule so CrowdSec can load it, wiring the configuration into the acquisition pipeline, deploying the files, and finally smoke-testing the result. + +## Step 1 — Stage the Rule File + +CrowdSec loads AppSec rules from `/etc/crowdsec/appsec-rules/`. Copy your YAML rule into that directory (create a `custom/` subfolder to keep things tidy if you manage several rules): + +```bash +sudo install -d -m 750 /etc/crowdsec/appsec-rules/custom +sudo install -m 640 ./my-virtual-patch.yaml \ + /etc/crowdsec/appsec-rules/custom/my-virtual-patch.yaml +``` + +Make sure the `name` inside the rule file matches the file name convention you plan to reference (for example `custom/my-virtual-patch`). + +:::tip +If you run CrowdSec in a container, copy the file into the volume that is mounted at `/etc/crowdsec/appsec-rules/` inside the container. +::: + +## Step 2 — Create an AppSec Configuration + +An AppSec configuration lists which rules to load and how to handle matches. Create a new file under `/etc/crowdsec/appsec-configs/` that targets your custom rule: + +```yaml title="/etc/crowdsec/appsec-configs/custom-virtual-patching.yaml" +name: custom/virtual-patching +default_remediation: ban +inband_rules: + - custom/my-virtual-patch +# Add outofband_rules or hooks here if needed +``` + +Key points: +- `name` is how you will reference this configuration from the acquisition file and in logs. +- `inband_rules` (and/or `outofband_rules`) accept glob patterns, so you can load multiple rules with a single entry such as `custom/my-virtual-patch-*`. +- Keep file permissions restrictive (`640`) so only the `crowdsec` user can read the file. +- During the reload step CrowdSec validates the syntax; if anything is off, the reload fails and the service logs the parsing error. + +## Step 3 — Reference the Configuration in the Acquisition File + +The AppSec acquisition file (`/etc/crowdsec/acquis.d/appsec.yaml`) controls which configurations are active for the WAF component. Add your configuration to the `appsec_configs` list. Order matters: later entries override conflicting defaults such as `default_remediation`. + +```yaml title="/etc/crowdsec/acquis.d/appsec.yaml" +appsec_configs: + - crowdsecurity/appsec-default + - custom/virtual-patching +labels: + type: appsec +listen_addr: 127.0.0.1:7422 +source: appsec +``` + +If you only want to run your custom configuration, remove other entries and keep the list with a single item. + +## Step 4 — Reload CrowdSec and Validate the Load + +Apply the changes by reloading the CrowdSec service: + +```bash +sudo systemctl reload crowdsec +``` + +If your init system does not support reload, perform a restart instead. Then verify the rule and configuration are active: + +```bash +sudo cscli appsec-rules list | grep my-virtual-patch +sudo cscli appsec-configs list | grep virtual-patching +sudo journalctl -u crowdsec --since "5 minutes ago" | grep appsec +``` + +The rule should appear as `enabled`, and the configuration should show up in the list. CrowdSec logs confirm the configuration was loaded without errors. + +## Step 5 — Functional Test with `curl` + +Trigger the behaviour your rule is meant to catch to ensure it blocks as expected. For example, if the rule protects `/admin` against a malicious header, you can test from the WAF host: + +```bash +curl -i -H 'X-Foobar-Bypass: 1' \ + -d 'user_id=123;cat /etc/passwd&do=yes' \ + http://127.0.0.1/admin +``` + +A successful block returns an HTTP status such as `403 Forbidden`, and CrowdSec logs a matching alert: + +```bash +sudo journalctl -u crowdsec -n 20 | grep "my-virtual-patch" +``` + +If the request is not blocked, double-check that the rule `name` matches the pattern in your AppSec configuration, that the acquisition file lists your configuration, and that the CrowdSec service picked up the changes. + +## Next Steps + +- Add automated regression tests with `cscli hubtest` so future updates do not break the rule. +- Version-control your custom rule and configuration files to keep track of changes. diff --git a/crowdsec-docs/sidebars.ts b/crowdsec-docs/sidebars.ts index 25050c0b5..02a67e709 100644 --- a/crowdsec-docs/sidebars.ts +++ b/crowdsec-docs/sidebars.ts @@ -737,8 +737,9 @@ const sidebarsConfig: SidebarConfig = { label: "Rules", items: [ { type: "doc", id: "appsec/rules_syntax" }, - { type: "doc", id: "appsec/hooks" }, { type: "doc", id: "appsec/create_rules" }, + { type: "doc", id: "appsec/rules_deploy" }, + { type: "doc", id: "appsec/hooks" }, { type: "doc", id: "appsec/rules_examples" }, ], }, From 2d563964a1aaae1b620043524280e4633aaa2d44 Mon Sep 17 00:00:00 2001 From: Thibault Koechlin Date: Tue, 30 Sep 2025 17:01:18 +0200 Subject: [PATCH 4/7] improved the guide --- crowdsec-docs/docs/appsec/rules_deploy.md | 64 +++++++++++++++++------ 1 file changed, 47 insertions(+), 17 deletions(-) diff --git a/crowdsec-docs/docs/appsec/rules_deploy.md b/crowdsec-docs/docs/appsec/rules_deploy.md index ae493467e..1e093f5db 100644 --- a/crowdsec-docs/docs/appsec/rules_deploy.md +++ b/crowdsec-docs/docs/appsec/rules_deploy.md @@ -1,12 +1,44 @@ --- id: rules_deploy -title: Custom Rules deployment +title: Rules deployment sidebar_position: 81 --- # WAF Rules Deployment -This guide starts once you have already authored and validated a custom AppSec (WAF) rule locally. The steps below focus on packaging that rule so CrowdSec can load it, wiring the configuration into the acquisition pipeline, deploying the files, and finally smoke-testing the result. +This walkthrough assumes you already wrote and validated a custom AppSec (WAF) rule. We will deploy a concrete example so you can mirror the exact commands on your host. + +## Example Rule We Will Deploy + +The example blocks any `GET` request whose `user_id` query argument contains non-numeric characters. While you iterate locally, keep it in a working directory as `./block-nonnumeric-user-id.yaml`: + +```yaml title="./block-nonnumeric-user-id.yaml" +name: custom/block-nonnumeric-user-id +description: Block GET requests with a non-numeric user_id parameter. +rules: + - and: + - zones: + - METHOD + match: + type: equals + value: GET + - zones: + - ARGS + variables: + - user_id + match: + type: regex + value: "[^0-9]" +labels: + type: exploit + service: http + confidence: 2 + spoofable: 0 + behavior: "http:exploit" + label: "Non numeric user id" +``` + +Once the rule behaves as expected, the remaining steps package it for CrowdSec, wire it into the acquisition pipeline, and test it end to end. ## Step 1 — Stage the Rule File @@ -14,11 +46,11 @@ CrowdSec loads AppSec rules from `/etc/crowdsec/appsec-rules/`. Copy your YAML r ```bash sudo install -d -m 750 /etc/crowdsec/appsec-rules/custom -sudo install -m 640 ./my-virtual-patch.yaml \ - /etc/crowdsec/appsec-rules/custom/my-virtual-patch.yaml +sudo install -m 640 ./block-nonnumeric-user-id.yaml \ + /etc/crowdsec/appsec-rules/custom/block-nonnumeric-user-id.yaml ``` -Make sure the `name` inside the rule file matches the file name convention you plan to reference (for example `custom/my-virtual-patch`). +Make sure the `name` inside the rule file matches the file name convention you plan to reference (in our example `custom/block-nonnumeric-user-id`). :::tip If you run CrowdSec in a container, copy the file into the volume that is mounted at `/etc/crowdsec/appsec-rules/` inside the container. @@ -28,17 +60,17 @@ If you run CrowdSec in a container, copy the file into the volume that is mounte An AppSec configuration lists which rules to load and how to handle matches. Create a new file under `/etc/crowdsec/appsec-configs/` that targets your custom rule: -```yaml title="/etc/crowdsec/appsec-configs/custom-virtual-patching.yaml" -name: custom/virtual-patching +```yaml title="/etc/crowdsec/appsec-configs/custom-block-nonnumeric-user-id.yaml" +name: custom/block-nonnumeric-user-id default_remediation: ban inband_rules: - - custom/my-virtual-patch + - custom/block-nonnumeric-user-id # Add outofband_rules or hooks here if needed ``` Key points: - `name` is how you will reference this configuration from the acquisition file and in logs. -- `inband_rules` (and/or `outofband_rules`) accept glob patterns, so you can load multiple rules with a single entry such as `custom/my-virtual-patch-*`. +- `inband_rules` (and/or `outofband_rules`) accept glob patterns, so you can load multiple rules with a single entry such as `custom/block-*`. - Keep file permissions restrictive (`640`) so only the `crowdsec` user can read the file. - During the reload step CrowdSec validates the syntax; if anything is off, the reload fails and the service logs the parsing error. @@ -49,7 +81,7 @@ The AppSec acquisition file (`/etc/crowdsec/acquis.d/appsec.yaml`) controls whic ```yaml title="/etc/crowdsec/acquis.d/appsec.yaml" appsec_configs: - crowdsecurity/appsec-default - - custom/virtual-patching + - custom/block-nonnumeric-user-id labels: type: appsec listen_addr: 127.0.0.1:7422 @@ -69,8 +101,8 @@ sudo systemctl reload crowdsec If your init system does not support reload, perform a restart instead. Then verify the rule and configuration are active: ```bash -sudo cscli appsec-rules list | grep my-virtual-patch -sudo cscli appsec-configs list | grep virtual-patching +sudo cscli appsec-rules list | grep block-nonnumeric-user-id +sudo cscli appsec-configs list | grep block-nonnumeric-user-id sudo journalctl -u crowdsec --since "5 minutes ago" | grep appsec ``` @@ -78,18 +110,16 @@ The rule should appear as `enabled`, and the configuration should show up in the ## Step 5 — Functional Test with `curl` -Trigger the behaviour your rule is meant to catch to ensure it blocks as expected. For example, if the rule protects `/admin` against a malicious header, you can test from the WAF host: +Trigger the behaviour your rule is meant to catch to ensure it blocks as expected. For the example rule, send a request with a non-numeric `user_id` value: ```bash -curl -i -H 'X-Foobar-Bypass: 1' \ - -d 'user_id=123;cat /etc/passwd&do=yes' \ - http://127.0.0.1/admin +curl -i 'http://127.0.0.1/profile?user_id=abc123' ``` A successful block returns an HTTP status such as `403 Forbidden`, and CrowdSec logs a matching alert: ```bash -sudo journalctl -u crowdsec -n 20 | grep "my-virtual-patch" +sudo journalctl -u crowdsec -n 20 | grep "block-nonnumeric-user-id" ``` If the request is not blocked, double-check that the rule `name` matches the pattern in your AppSec configuration, that the acquisition file lists your configuration, and that the CrowdSec service picked up the changes. From c5bb5185351ef282e852d853c4998fd430b31880 Mon Sep 17 00:00:00 2001 From: Thibault Koechlin Date: Tue, 30 Sep 2025 17:16:00 +0200 Subject: [PATCH 5/7] up --- crowdsec-docs/docs/appsec/create_rules.md | 2 +- crowdsec-docs/docs/appsec/rules_deploy.md | 2 +- crowdsec-docs/docs/appsec/rules_syntax.md | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/crowdsec-docs/docs/appsec/create_rules.md b/crowdsec-docs/docs/appsec/create_rules.md index c04cfb89d..e46265118 100644 --- a/crowdsec-docs/docs/appsec/create_rules.md +++ b/crowdsec-docs/docs/appsec/create_rules.md @@ -1,6 +1,6 @@ --- id: create_rules -title: Rules Creation & Testing +title: Creation & Testing sidebar_position: 3 --- diff --git a/crowdsec-docs/docs/appsec/rules_deploy.md b/crowdsec-docs/docs/appsec/rules_deploy.md index 1e093f5db..04915d6eb 100644 --- a/crowdsec-docs/docs/appsec/rules_deploy.md +++ b/crowdsec-docs/docs/appsec/rules_deploy.md @@ -1,6 +1,6 @@ --- id: rules_deploy -title: Rules deployment +title: Deployment sidebar_position: 81 --- diff --git a/crowdsec-docs/docs/appsec/rules_syntax.md b/crowdsec-docs/docs/appsec/rules_syntax.md index d194046f6..32c903298 100644 --- a/crowdsec-docs/docs/appsec/rules_syntax.md +++ b/crowdsec-docs/docs/appsec/rules_syntax.md @@ -1,6 +1,6 @@ --- id: rules_syntax -title: Rules syntax +title: Syntax sidebar_position: 8 --- From d6526738aa75535259c870f31b5aaeae148a8e97 Mon Sep 17 00:00:00 2001 From: Thibault Koechlin Date: Tue, 30 Sep 2025 17:55:37 +0200 Subject: [PATCH 6/7] up --- crowdsec-docs/docs/appsec/rules_deploy.md | 1 - 1 file changed, 1 deletion(-) diff --git a/crowdsec-docs/docs/appsec/rules_deploy.md b/crowdsec-docs/docs/appsec/rules_deploy.md index 04915d6eb..ecb9139e6 100644 --- a/crowdsec-docs/docs/appsec/rules_deploy.md +++ b/crowdsec-docs/docs/appsec/rules_deploy.md @@ -71,7 +71,6 @@ inband_rules: Key points: - `name` is how you will reference this configuration from the acquisition file and in logs. - `inband_rules` (and/or `outofband_rules`) accept glob patterns, so you can load multiple rules with a single entry such as `custom/block-*`. -- Keep file permissions restrictive (`640`) so only the `crowdsec` user can read the file. - During the reload step CrowdSec validates the syntax; if anything is off, the reload fails and the service logs the parsing error. ## Step 3 — Reference the Configuration in the Acquisition File From 5a77ff229a015190434da8e9ea9c0db248c4c37b Mon Sep 17 00:00:00 2001 From: Thibault Koechlin Date: Tue, 30 Sep 2025 18:00:30 +0200 Subject: [PATCH 7/7] up --- crowdsec-docs/docs/appsec/rules_deploy.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/crowdsec-docs/docs/appsec/rules_deploy.md b/crowdsec-docs/docs/appsec/rules_deploy.md index ecb9139e6..f2aa80dd2 100644 --- a/crowdsec-docs/docs/appsec/rules_deploy.md +++ b/crowdsec-docs/docs/appsec/rules_deploy.md @@ -102,7 +102,6 @@ If your init system does not support reload, perform a restart instead. Then ver ```bash sudo cscli appsec-rules list | grep block-nonnumeric-user-id sudo cscli appsec-configs list | grep block-nonnumeric-user-id -sudo journalctl -u crowdsec --since "5 minutes ago" | grep appsec ``` The rule should appear as `enabled`, and the configuration should show up in the list. CrowdSec logs confirm the configuration was loaded without errors. @@ -118,7 +117,7 @@ curl -i 'http://127.0.0.1/profile?user_id=abc123' A successful block returns an HTTP status such as `403 Forbidden`, and CrowdSec logs a matching alert: ```bash -sudo journalctl -u crowdsec -n 20 | grep "block-nonnumeric-user-id" +sudo cscli alerts list -s custom/block-nonnumeric-user-id ``` If the request is not blocked, double-check that the rule `name` matches the pattern in your AppSec configuration, that the acquisition file lists your configuration, and that the CrowdSec service picked up the changes.