v1.6.0 container fails to start on crowdsec/whiltelist installation #2786

bufanda · 2024-01-25T23:31:58Z

What happened?

When I updated from 1.5.5 to 1.6.0 crowdsec failed with follwoing log

Check if lapi needs to register an additional agent
time="2024-01-25T22:57:55Z" level=info msg="hub index is up to date"
time="2024-01-25T22:57:55Z" level=info msg="Ignoring file /etc/crowdsec/parsers/s02-enrich/whitelists.yaml of type parsers"
time="2024-01-25T22:57:55Z" level=info msg="Ignoring file /etc/crowdsec/hub/parsers/s02-enrich/crowdsecurity/whitelists.yaml of type parsers"
time="2024-01-25T22:57:55Z" level=info msg="crowdsecurity/sshd is tainted by missing contexts:crowdsecurity/bf_base"
time="2024-01-25T22:57:55Z" level=info msg="crowdsecurity/linux is tainted by collections:crowdsecurity/sshd"
Object collections/crowdsecurity/linux is tainted, skipping
Running: cscli  parsers upgrade "crowdsecurity/whitelists" 
time="2024-01-25T22:57:55Z" level=info msg="Ignoring file /etc/crowdsec/parsers/s02-enrich/whitelists.yaml of type parsers"
time="2024-01-25T22:57:55Z" level=info msg="Ignoring file /etc/crowdsec/hub/parsers/s02-enrich/crowdsecurity/whitelists.yaml of type parsers"
time="2024-01-25T22:57:55Z" level=info msg="not upgrading crowdsecurity/whitelists: local item"
Running: cscli  parsers install "crowdsecurity/docker-logs" 
time="2024-01-25T22:57:55Z" level=info msg="Ignoring file /etc/crowdsec/parsers/s02-enrich/whitelists.yaml of type parsers"
time="2024-01-25T22:57:55Z" level=info msg="Ignoring file /etc/crowdsec/hub/parsers/s02-enrich/crowdsecurity/whitelists.yaml of type parsers"
time="2024-01-25T22:57:55Z" level=warning msg="crowdsecurity/docker-logs: overwrite"
time="2024-01-25T22:57:55Z" level=info msg="Enabled crowdsecurity/docker-logs"
time="2024-01-25T22:57:55Z" level=info msg="Run 'sudo systemctl reload crowdsec' for the new configuration to be effective."
Running: cscli  parsers install "crowdsecurity/cri-logs" 
time="2024-01-25T22:57:55Z" level=info msg="Ignoring file /etc/crowdsec/parsers/s02-enrich/whitelists.yaml of type parsers"
time="2024-01-25T22:57:55Z" level=info msg="Ignoring file /etc/crowdsec/hub/parsers/s02-enrich/crowdsecurity/whitelists.yaml of type parsers"
time="2024-01-25T22:57:55Z" level=warning msg="crowdsecurity/cri-logs: overwrite"
time="2024-01-25T22:57:55Z" level=info msg="Enabled crowdsecurity/cri-logs"
time="2024-01-25T22:57:55Z" level=info msg="Run 'sudo systemctl reload crowdsec' for the new configuration to be effective."
Object collections/crowdsecurity/sshd is tainted, skipping
Running: cscli  collections install "crowdsecurity/traefik" 
time="2024-01-25T22:57:56Z" level=info msg="Ignoring file /etc/crowdsec/parsers/s02-enrich/whitelists.yaml of type parsers"
time="2024-01-25T22:57:56Z" level=info msg="Ignoring file /etc/crowdsec/hub/parsers/s02-enrich/crowdsecurity/whitelists.yaml of type parsers"
time="2024-01-25T22:57:56Z" level=warning msg="crowdsecurity/traefik-logs: overwrite"
time="2024-01-25T22:57:56Z" level=warning msg="crowdsecurity/http-logs: overwrite"
time="2024-01-25T22:57:56Z" level=warning msg="crowdsecurity/http-crawl-non_statics: overwrite"
time="2024-01-25T22:57:56Z" level=warning msg="crowdsecurity/http-probing: overwrite"
time="2024-01-25T22:57:56Z" level=warning msg="crowdsecurity/http-bad-user-agent: overwrite"
time="2024-01-25T22:57:56Z" level=warning msg="crowdsecurity/http-path-traversal-probing: overwrite"
time="2024-01-25T22:57:56Z" level=warning msg="crowdsecurity/http-sensitive-files: overwrite"
time="2024-01-25T22:57:56Z" level=warning msg="crowdsecurity/http-sqli-probing: overwrite"
time="2024-01-25T22:57:56Z" level=warning msg="crowdsecurity/http-xss-probing: overwrite"
time="2024-01-25T22:57:56Z" level=warning msg="crowdsecurity/http-backdoors-attempts: overwrite"
time="2024-01-25T22:57:56Z" level=warning msg="ltsich/http-w00tw00t: overwrite"
time="2024-01-25T22:57:56Z" level=warning msg="crowdsecurity/http-generic-bf: overwrite"
time="2024-01-25T22:57:56Z" level=warning msg="crowdsecurity/http-open-proxy: overwrite"
time="2024-01-25T22:57:56Z" level=warning msg="crowdsecurity/http-admin-interface-probing: overwrite"
time="2024-01-25T22:57:56Z" level=warning msg="crowdsecurity/http_base: overwrite"
time="2024-01-25T22:57:56Z" level=warning msg="crowdsecurity/http-cve-2021-41773: overwrite"
time="2024-01-25T22:57:56Z" level=warning msg="crowdsecurity/http-cve-2021-42013: overwrite"
time="2024-01-25T22:57:56Z" level=warning msg="crowdsecurity/grafana-cve-2021-43798: overwrite"
time="2024-01-25T22:57:56Z" level=warning msg="crowdsecurity/vmware-vcenter-vmsa-2021-0027: overwrite"
time="2024-01-25T22:57:56Z" level=warning msg="crowdsecurity/fortinet-cve-2018-13379: overwrite"
time="2024-01-25T22:57:56Z" level=warning msg="crowdsecurity/pulse-secure-sslvpn-cve-2019-11510: overwrite"
time="2024-01-25T22:57:56Z" level=warning msg="crowdsecurity/f5-big-ip-cve-2020-5902: overwrite"
time="2024-01-25T22:57:56Z" level=warning msg="crowdsecurity/thinkphp-cve-2018-20062: overwrite"
time="2024-01-25T22:57:56Z" level=warning msg="crowdsecurity/apache_log4j2_cve-2021-44228: overwrite"
time="2024-01-25T22:57:56Z" level=warning msg="crowdsecurity/jira_cve-2021-26086: overwrite"
time="2024-01-25T22:57:56Z" level=warning msg="crowdsecurity/spring4shell_cve-2022-22965: overwrite"
time="2024-01-25T22:57:56Z" level=warning msg="crowdsecurity/vmware-cve-2022-22954: overwrite"
time="2024-01-25T22:57:56Z" level=warning msg="crowdsecurity/CVE-2022-37042: overwrite"
time="2024-01-25T22:57:56Z" level=warning msg="crowdsecurity/CVE-2022-41082: overwrite"
time="2024-01-25T22:57:56Z" level=warning msg="crowdsecurity/CVE-2022-35914: overwrite"
time="2024-01-25T22:57:56Z" level=warning msg="crowdsecurity/CVE-2022-40684: overwrite"
time="2024-01-25T22:57:56Z" level=warning msg="crowdsecurity/CVE-2022-26134: overwrite"
time="2024-01-25T22:57:56Z" level=warning msg="crowdsecurity/CVE-2022-42889: overwrite"
time="2024-01-25T22:57:56Z" level=warning msg="crowdsecurity/CVE-2022-41697: overwrite"
time="2024-01-25T22:57:56Z" level=warning msg="crowdsecurity/CVE-2022-46169: overwrite"
time="2024-01-25T22:57:56Z" level=warning msg="crowdsecurity/CVE-2022-44877: overwrite"
time="2024-01-25T22:57:56Z" level=warning msg="crowdsecurity/CVE-2019-18935: overwrite"
time="2024-01-25T22:57:56Z" level=warning msg="crowdsecurity/netgear_rce: overwrite"
time="2024-01-25T22:57:56Z" level=warning msg="crowdsecurity/CVE-2023-22515: overwrite"
time="2024-01-25T22:57:56Z" level=warning msg="crowdsecurity/CVE-2023-22518: overwrite"
time="2024-01-25T22:57:56Z" level=warning msg="crowdsecurity/CVE-2023-49103: overwrite"
time="2024-01-25T22:57:56Z" level=warning msg="crowdsecurity/http-cve: overwrite"
time="2024-01-25T22:57:56Z" level=warning msg="crowdsecurity/http-cve: overwrite"
time="2024-01-25T22:57:56Z" level=warning msg="crowdsecurity/base-http-scenarios: overwrite"
time="2024-01-25T22:57:56Z" level=warning msg="crowdsecurity/base-http-scenarios: overwrite"
time="2024-01-25T22:57:56Z" level=warning msg="crowdsecurity/traefik: overwrite"
time="2024-01-25T22:57:56Z" level=info msg="/etc/crowdsec/collections/http-cve.yaml already exists."
time="2024-01-25T22:57:56Z" level=info msg="Enabled collections: crowdsecurity/http-cve"
time="2024-01-25T22:57:56Z" level=info msg="/etc/crowdsec/collections/base-http-scenarios.yaml already exists."
time="2024-01-25T22:57:56Z" level=info msg="Enabled collections: crowdsecurity/base-http-scenarios"
time="2024-01-25T22:57:56Z" level=info msg="/etc/crowdsec/collections/traefik.yaml already exists."
time="2024-01-25T22:57:56Z" level=info msg="Enabled collections: crowdsecurity/traefik"
time="2024-01-25T22:57:56Z" level=info msg="Enabled crowdsecurity/traefik"
time="2024-01-25T22:57:56Z" level=info msg="Run 'sudo systemctl reload crowdsec' for the new configuration to be effective."
Running: cscli  parsers install "crowdsecurity/whitelists" 
time="2024-01-25T22:57:56Z" level=info msg="Ignoring file /etc/crowdsec/parsers/s02-enrich/whitelists.yaml of type parsers"
time="2024-01-25T22:57:56Z" level=info msg="Ignoring file /etc/crowdsec/hub/parsers/s02-enrich/crowdsecurity/whitelists.yaml of type parsers"
time="2024-01-25T22:57:56Z" level=warning msg="crowdsecurity/whitelists is local, can't download"
time="2024-01-25T22:57:56Z" level=fatal msg="error while installing 'crowdsecurity/whitelists': while enabling crowdsecurity/whitelists: crowdsecurity/whitelists is local, won't enable"

What did you expect to happen?

Crowdsec should migrate configuration to new version and start up.

How can we reproduce it (as minimally and precisely as possible)?

Have crowdsec container version 1.5.5 running with parser crowdsec/whitelists running and upgrade to v1.6.0 container

Anything else we need to know?

No response

Crowdsec version

$ cscli version
# paste output here

I would post the version but container isn't running so I can't attach to it to print version string.

But it is the latest container (v1.6.0) with hash

crowdsecurity/crowdsec                    latest                    26f479bef5aa   32 hours ago    264MB

OS version

❯ cat /etc/os-release
NAME="Ubuntu"
VERSION="20.04.6 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.6 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal
❯ uname -a
Linux bufanda 5.4.0-170-generic #188-Ubuntu SMP Wed Jan 10 09:51:01 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Enabled collections and parsers

Enabled on v1.5.0 and so should be on v1.6.0 ```console ❯ docker exec -it crowdsec cscli hub list -o raw crowdsecurity/base-http-scenarios,"enabled,tainted",?,http common : scanners detection,collections crowdsecurity/http-cve,enabled,2.5,Detect CVE exploitation in http logs,collections crowdsecurity/linux,"enabled,tainted",0.2,core linux support : syslog+geoip+ssh,collections crowdsecurity/sshd,"enabled,tainted",?,sshd support : parser and brute-force detection,collections crowdsecurity/traefik,"enabled,tainted",0.1,traefik support: parser and generic http scenarios,collections crowdsecurity/cri-logs,enabled,0.1,CRI logging format parser,parsers crowdsecurity/dateparse-enrich,enabled,0.2,,parsers crowdsecurity/docker-logs,enabled,0.1,docker json logs parser,parsers crowdsecurity/geoip-enrich,enabled,0.2,"Populate event with geoloc info : as, country, coords, source range.",parsers crowdsecurity/http-logs,enabled,1.2,"Parse more Specifically HTTP logs, such as HTTP Code, HTTP path, HTTP args and if its a static ressource",parsers crowdsecurity/sshd-logs,enabled,2.2,Parse openSSH logs,parsers crowdsecurity/syslog-logs,enabled,0.8,,parsers crowdsecurity/traefik-logs,enabled,0.9,Parse Traefik access logs,parsers crowdsecurity/whitelists,enabled,0.2,Whitelist events from private ipv4 addresses,parsers mywhitelists.yaml,"enabled,local",n/a,,parsers crowdsecurity/CVE-2019-18935,enabled,0.2,Detect Telerik CVE-2019-18935 exploitation attempts,scenarios crowdsecurity/CVE-2022-26134,enabled,0.2,Detect CVE-2022-26134 exploits,scenarios crowdsecurity/CVE-2022-35914,enabled,0.2,Detect CVE-2022-35914 exploits,scenarios crowdsecurity/CVE-2022-37042,enabled,0.2,Detect CVE-2022-37042 exploits,scenarios crowdsecurity/CVE-2022-40684,enabled,0.3,Detect cve-2022-40684 exploitation attempts,scenarios crowdsecurity/CVE-2022-41082,enabled,0.4,Detect CVE-2022-41082 exploits,scenarios crowdsecurity/CVE-2022-41697,enabled,0.2,Detect CVE-2022-41697 enumeration,scenarios crowdsecurity/CVE-2022-42889,enabled,0.3,Detect CVE-2022-42889 exploits (Text4Shell),scenarios crowdsecurity/CVE-2022-44877,enabled,0.3,Detect CVE-2022-44877 exploits,scenarios crowdsecurity/CVE-2022-46169,enabled,0.2,Detect CVE-2022-46169 brute forcing,scenarios crowdsecurity/CVE-2023-22515,enabled,0.1,Detect CVE-2023-22515 exploitation,scenarios crowdsecurity/CVE-2023-22518,enabled,0.2,Detect CVE-2023-22518 exploits,scenarios crowdsecurity/CVE-2023-49103,enabled,0.2,Detect owncloud CVE-2023-49103 exploitation attempts,scenarios crowdsecurity/apache_log4j2_cve-2021-44228,enabled,0.5,Detect cve-2021-44228 exploitation attemps,scenarios crowdsecurity/f5-big-ip-cve-2020-5902,enabled,0.2,Detect cve-2020-5902 exploitation attemps,scenarios crowdsecurity/fortinet-cve-2018-13379,enabled,0.3,Detect cve-2018-13379 exploitation attemps,scenarios crowdsecurity/grafana-cve-2021-43798,enabled,0.2,Detect cve-2021-43798 exploitation attemps,scenarios crowdsecurity/http-backdoors-attempts,enabled,0.4,Detect attempt to common backdoors,scenarios crowdsecurity/http-bad-user-agent,enabled,0.9,Detect bad user-agents,scenarios crowdsecurity/http-crawl-non_statics,enabled,0.4,Detect aggressive crawl from single ip,scenarios crowdsecurity/http-cve-2021-41773,enabled,0.2,cve-2021-41773,scenarios crowdsecurity/http-cve-2021-42013,enabled,0.2,cve-2021-42013,scenarios crowdsecurity/http-generic-bf,enabled,0.5,Detect generic http brute force,scenarios crowdsecurity/http-open-proxy,enabled,0.4,Detect scan for open proxy,scenarios crowdsecurity/http-path-traversal-probing,enabled,0.3,Detect path traversal attempt,scenarios crowdsecurity/http-probing,enabled,0.3,Detect site scanning/probing from a single ip,scenarios crowdsecurity/http-sensitive-files,enabled,0.3,"Detect attempt to access to sensitive files (.log, .db ..) or folders (.git)",scenarios crowdsecurity/http-sqli-probing,enabled,0.3,A scenario that detects SQL injection probing with minimal false positives,scenarios crowdsecurity/http-xss-probing,enabled,0.3,A scenario that detects XSS probing with minimal false positives,scenarios crowdsecurity/jira_cve-2021-26086,enabled,0.2,Detect Atlassian Jira CVE-2021-26086 exploitation attemps,scenarios crowdsecurity/netgear_rce,enabled,0.3,Detect Netgear RCE DGN1000/DGN220 exploitation attempts,scenarios crowdsecurity/pulse-secure-sslvpn-cve-2019-11510,enabled,0.3,Detect cve-2019-11510 exploitation attemps,scenarios crowdsecurity/spring4shell_cve-2022-22965,enabled,0.3,Detect cve-2022-22965 probing,scenarios crowdsecurity/ssh-bf,enabled,0.3,Detect ssh bruteforce,scenarios crowdsecurity/ssh-slow-bf,enabled,0.4,Detect slow ssh bruteforce,scenarios crowdsecurity/thinkphp-cve-2018-20062,enabled,0.4,Detect ThinkPHP CVE-2018-20062 exploitation attemps,scenarios crowdsecurity/vmware-cve-2022-22954,enabled,0.3,Detect Vmware CVE-2022-22954 exploitation attempts,scenarios crowdsecurity/vmware-vcenter-vmsa-2021-0027,enabled,0.2,Detect VMSA-2021-0027 exploitation attemps,scenarios ltsich/http-w00tw00t,enabled,0.2,detect w00tw00t,scenarios


</details>


### Acquisition config

<details>
```console
❯ cat /volumes/crowdsec/acquis.yaml
---
filenames:
 - /var/log/auth.log
 - /var/log/syslog
labels:
  type: syslog
---
filenames:
 - /var/log/traefik/access.log
labels:
  type: traefik

Config show

1.5.0 config but should be the same on 1.6.0 ```console ❯ docker exec -it crowdsec cscli config show Global: - Configuration Folder : /etc/crowdsec - Data Folder : /var/lib/crowdsec/data - Hub Folder : /etc/crowdsec/hub - Simulation File : /etc/crowdsec/simulation.yaml - Log Folder : /var/log/ - Log level : info - Log Media : stdout Crowdsec: - Acquisition File : /etc/crowdsec/acquis.yaml - Parsers routines : 1 cscli: - Output : human - Hub Branch : - Hub Folder : /etc/crowdsec/hub API Client: - URL : http://0.0.0.0:8080/ - Login : localhost - Credentials File : /etc/crowdsec/local_api_credentials.yaml Local API Server: - Listen URL : 0.0.0.0:8080 - Profile File : /etc/crowdsec/profiles.yaml

Trusted IPs:
- 127.0.0.1
- ::1
Database:
- Type : sqlite
- Path : /var/lib/crowdsec/data/crowdsec.db
- Flush age : 7d
- Flush size : 5000


</details>


### Prometheus metrics

<details>

```console
❯ docker exec -it crowdsec cscli metrics

Acquisition Metrics:
╭──────────────────────────────────┬────────────┬──────────────┬────────────────┬────────────────────────╮
│              Source              │ Lines read │ Lines parsed │ Lines unparsed │ Lines poured to bucket │
├──────────────────────────────────┼────────────┼──────────────┼────────────────┼────────────────────────┤
│ file:/var/log/auth.log           │ 33         │ 16           │ 17             │ 46                     │
│ file:/var/log/syslog             │ 109        │ -            │ 109            │ -                      │
│ file:/var/log/traefik/access.log │ 341        │ 341          │ -              │ 1                      │
╰──────────────────────────────────┴────────────┴──────────────┴────────────────┴────────────────────────╯

Bucket Metrics:
╭──────────────────────────────────────┬───────────────┬───────────┬──────────────┬────────┬─────────╮
│                Bucket                │ Current Count │ Overflows │ Instantiated │ Poured │ Expired │
├──────────────────────────────────────┼───────────────┼───────────┼──────────────┼────────┼─────────┤
│ crowdsecurity/http-crawl-non_statics │ -             │ -         │ 1            │ 1      │ 1       │
│ crowdsecurity/ssh-bf                 │ -             │ -         │ 8            │ 16     │ 8       │
│ crowdsecurity/ssh-bf_user-enum       │ -             │ -         │ 8            │ 8      │ 8       │
│ crowdsecurity/ssh-slow-bf            │ 3             │ -         │ 5            │ 16     │ 2       │
│ crowdsecurity/ssh-slow-bf_user-enum  │ 3             │ -         │ 5            │ 6      │ 2       │
╰──────────────────────────────────────┴───────────────┴───────────┴──────────────┴────────┴─────────╯

Parser Metrics:
╭──────────────────────────────────┬───────┬────────┬──────────╮
│             Parsers              │ Hits  │ Parsed │ Unparsed │
├──────────────────────────────────┼───────┼────────┼──────────┤
│ child-crowdsecurity/http-logs    │ 1.02k │ 953    │ 70       │
│ child-crowdsecurity/sshd-logs    │ 112   │ 16     │ 96       │
│ child-crowdsecurity/syslog-logs  │ 142   │ 142    │ -        │
│ child-crowdsecurity/traefik-logs │ 341   │ 341    │ -        │
│ crowdsecurity/dateparse-enrich   │ 357   │ 357    │ -        │
│ crowdsecurity/geoip-enrich       │ 357   │ 357    │ -        │
│ crowdsecurity/http-logs          │ 341   │ 341    │ -        │
│ crowdsecurity/non-syslog         │ 341   │ 341    │ -        │
│ crowdsecurity/sshd-logs          │ 20    │ 16     │ 4        │
│ crowdsecurity/syslog-logs        │ 142   │ 142    │ -        │
│ crowdsecurity/traefik-logs       │ 341   │ 341    │ -        │
│ crowdsecurity/whitelists         │ 714   │ 714    │ -        │
╰──────────────────────────────────┴───────┴────────┴──────────╯

Local API Metrics:
╭──────────────────────┬────────┬──────╮
│        Route         │ Method │ Hits │
├──────────────────────┼────────┼──────┤
│ /v1/decisions/stream │ GET    │ 74   │
│ /v1/heartbeat        │ GET    │ 11   │
│ /v1/watchers/login   │ POST   │ 1    │
╰──────────────────────┴────────┴──────╯

Local API Machines Metrics:
╭───────────┬───────────────┬────────┬──────╮
│  Machine  │     Route     │ Method │ Hits │
├───────────┼───────────────┼────────┼──────┤
│ localhost │ /v1/heartbeat │ GET    │ 11   │
╰───────────┴───────────────┴────────┴──────╯

Local API Bouncers Metrics:
╭─────────────────┬──────────────────────┬────────┬──────╮
│     Bouncer     │        Route         │ Method │ Hits │
├─────────────────┼──────────────────────┼────────┼──────┤
│ crowdsecBouncer │ /v1/decisions/stream │ GET    │ 11   │
│ iptables        │ /v1/decisions/stream │ GET    │ 63   │
╰─────────────────┴──────────────────────┴────────┴──────╯

Local API Decisions:
╭────────────────────────────────────────────┬──────────┬────────┬───────╮
│                   Reason                   │  Origin  │ Action │ Count │
├────────────────────────────────────────────┼──────────┼────────┼───────┤
│ crowdsecurity/grafana-cve-2021-43798       │ CAPI     │ ban    │ 60    │
│ crowdsecurity/http-cve-2021-41773          │ CAPI     │ ban    │ 44    │
│ crowdsecurity/netgear_rce                  │ CAPI     │ ban    │ 20    │
│ ltsich/http-w00tw00t                       │ CAPI     │ ban    │ 1     │
│ crowdsecurity/CVE-2022-35914               │ CAPI     │ ban    │ 68    │
│ crowdsecurity/http-bad-user-agent          │ CAPI     │ ban    │ 19853 │
│ crowdsecurity/ssh-bf                       │ CAPI     │ ban    │ 15612 │
│ crowdsecurity/ssh-bf                       │ crowdsec │ ban    │ 1     │
│ crowdsecurity/ssh-slow-bf                  │ CAPI     │ ban    │ 38    │
│ crowdsecurity/ssh-slow-bf                  │ crowdsec │ ban    │ 1     │
│ crowdsecurity/CVE-2023-49103               │ CAPI     │ ban    │ 396   │
│ crowdsecurity/http-cve-2021-42013          │ CAPI     │ ban    │ 3     │
│ crowdsecurity/http-generic-bf              │ CAPI     │ ban    │ 54    │
│ crowdsecurity/http-path-traversal-probing  │ CAPI     │ ban    │ 243   │
│ crowdsecurity/CVE-2019-18935               │ CAPI     │ ban    │ 140   │
│ crowdsecurity/CVE-2022-37042               │ CAPI     │ ban    │ 25    │
│ crowdsecurity/CVE-2022-41082               │ CAPI     │ ban    │ 999   │
│ crowdsecurity/http-probing                 │ CAPI     │ ban    │ 3360  │
│ crowdsecurity/http-probing                 │ crowdsec │ ban    │ 1     │
│ crowdsecurity/CVE-2022-26134               │ CAPI     │ ban    │ 269   │
│ crowdsecurity/jira_cve-2021-26086          │ CAPI     │ ban    │ 30    │
│ crowdsecurity/fortinet-cve-2018-13379      │ CAPI     │ ban    │ 97    │
│ crowdsecurity/http-sensitive-files         │ CAPI     │ ban    │ 75    │
│ crowdsecurity/http-crawl-non_statics       │ CAPI     │ ban    │ 861   │
│ crowdsecurity/http-crawl-non_statics       │ crowdsec │ ban    │ 1     │
│ crowdsecurity/http-open-proxy              │ CAPI     │ ban    │ 1618  │
│ crowdsecurity/thinkphp-cve-2018-20062      │ CAPI     │ ban    │ 11    │
│ crowdsecurity/CVE-2023-22515               │ CAPI     │ ban    │ 28    │
│ crowdsecurity/CVE-2023-22518               │ CAPI     │ ban    │ 13    │
│ crowdsecurity/apache_log4j2_cve-2021-44228 │ CAPI     │ ban    │ 413   │
│ crowdsecurity/f5-big-ip-cve-2020-5902      │ CAPI     │ ban    │ 28    │
│ crowdsecurity/CVE-2022-42889               │ CAPI     │ ban    │ 5     │
│ crowdsecurity/http-backdoors-attempts      │ CAPI     │ ban    │ 3471  │
╰────────────────────────────────────────────┴──────────┴────────┴───────╯

Local API Alerts:
╭───────────────────────────────────────┬───────╮
│                Reason                 │ Count │
├───────────────────────────────────────┼───────┤
│ crowdsecurity/http-backdoors-attempts │ 1     │
│ crowdsecurity/http-probing            │ 9     │
│ crowdsecurity/http-crawl-non_statics  │ 6     │
│ crowdsecurity/http-sensitive-files    │ 2     │
│ crowdsecurity/jira_cve-2021-26086     │ 1     │
│ crowdsecurity/ssh-bf                  │ 42    │
│ crowdsecurity/ssh-bf_user-enum        │ 19    │
│ crowdsecurity/ssh-slow-bf             │ 27    │
│ crowdsecurity/ssh-slow-bf_user-enum   │ 14    │
╰───────────────────────────────────────┴───────╯

Related custom configs versions (if applicable) : notification plugins, custom scenarios, parsers etc.

The text was updated successfully, but these errors were encountered:

github-actions · 2024-01-25T23:32:09Z

@bufanda: Thanks for opening an issue, it is currently awaiting triage.

In the meantime, you can:

Check Crowdsec Documentation to see if your issue can be self resolved.
You can also join our Discord.
Check Releases to make sure your agent is on the latest version.

Details

I am a bot created to help the crowdsecurity developers manage community feedback and contributions. You can check out my manifest file to understand my behavior and what I can do. If you want to use this for your project, you can check out the BirthdayResearch/oss-governance-bot repository.

LaurenceJJones · 2024-01-25T23:38:58Z

Hi there was a patch pushed to the repository today. You can move over to the :dev tag and it should be working for now.

LaurenceJJones · 2024-01-25T23:39:20Z

Link to closed issue #2779

LaurenceJJones · 2024-01-25T23:47:28Z

Interesting you have custom/tainted the crowdsecurity/whitelists s02-enrich file, however, you still have it defined within your docker environments to install. The behaviour before was to ignore it, are you running within docker compose and can remove that whitelists parser from the environment variables as you dont need to install it anymore because your mounting / persisting your own custom one.

LaurenceJJones · 2024-01-25T23:56:34Z

Just checking the docker start script code, I see the issue and have reported it to the team. Thank you 🦙 we hope to get a fix merged and maybe a subsequent version release by next week depending on scope.

bufanda · 2024-01-26T06:43:18Z

Interesting you have custom/tainted the crowdsecurity/whitelists s02-enrich file, however, you still have it defined within your docker environments to install. The behaviour before was to ignore it, are you running within docker compose and can remove that whitelists parser from the environment variables as you dont need to install it anymore because your mounting / persisting your own custom one.

No Idon't use compose I use ansible to deploy the configuration and yes I had to create my own whitelist to whitelist my own IP as I got constantly block because the Nextcloud Client on my PCs was HTTP probing and that triggered the decision to block me. So whitelisting was my solution for it.

I can try to remove the parser from the env variable though, but as you said there is an issue in the startup script I might hold of for the time being on that. Meanwhile I reverted back to Version 1.5.5 on the host.

mushuthecat · 2024-01-26T12:37:47Z

Hi,

Since the last update using docker i think i facing the same issue:

Local agent already registered
Check if lapi needs to register an additional agent
time="2024-01-26T12:37:01Z" level=info msg="hub index is up to date"
time="2024-01-26T12:37:01Z" level=info msg="Ignoring file /etc/crowdsec/hub/parsers/s02-enrich/crowdsecurity/whitelists.yaml of type parsers"
time="2024-01-26T12:37:01Z" level=info msg="update for collection crowdsecurity/sshd available (currently:0.2, latest:0.3)"
time="2024-01-26T12:37:01Z" level=info msg="crowdsecurity/traefik is tainted by outdated parsers:crowdsecurity/traefik-logs"
time="2024-01-26T12:37:01Z" level=info msg="crowdsecurity/traefik is tainted by outdated collections:crowdsecurity/base-http-scenarios"
time="2024-01-26T12:37:01Z" level=info msg="update for collection crowdsecurity/base-http-scenarios available (currently:0.6, latest:0.8)"
time="2024-01-26T12:37:01Z" level=info msg="update for collection crowdsecurity/http-cve available (currently:2.1, latest:2.5)"
Running: cscli collections upgrade "crowdsecurity/linux"
time="2024-01-26T12:37:01Z" level=info msg="Ignoring file /etc/crowdsec/hub/parsers/s02-enrich/crowdsecurity/whitelists.yaml of type parsers"
time="2024-01-26T12:37:01Z" level=fatal msg="can't upgrade crowdsecurity/linux: downloaded but not installed"

LaurenceJJones · 2024-01-26T12:53:11Z

Hi,

Since the last update using docker i think i facing the same issue:

Local agent already registered Check if lapi needs to register an additional agent time="2024-01-26T12:37:01Z" level=info msg="hub index is up to date" time="2024-01-26T12:37:01Z" level=info msg="Ignoring file /etc/crowdsec/hub/parsers/s02-enrich/crowdsecurity/whitelists.yaml of type parsers" time="2024-01-26T12:37:01Z" level=info msg="update for collection crowdsecurity/sshd available (currently:0.2, latest:0.3)" time="2024-01-26T12:37:01Z" level=info msg="crowdsecurity/traefik is tainted by outdated parsers:crowdsecurity/traefik-logs" time="2024-01-26T12:37:01Z" level=info msg="crowdsecurity/traefik is tainted by outdated collections:crowdsecurity/base-http-scenarios" time="2024-01-26T12:37:01Z" level=info msg="update for collection crowdsecurity/base-http-scenarios available (currently:0.6, latest:0.8)" time="2024-01-26T12:37:01Z" level=info msg="update for collection crowdsecurity/http-cve available (currently:2.1, latest:2.5)" Running: cscli collections upgrade "crowdsecurity/linux" time="2024-01-26T12:37:01Z" level=info msg="Ignoring file /etc/crowdsec/hub/parsers/s02-enrich/crowdsecurity/whitelists.yaml of type parsers" time="2024-01-26T12:37:01Z" level=fatal msg="can't upgrade crowdsecurity/linux: downloaded but not installed"

Hi, did you read the comments above how to resolve?

mushuthecat · 2024-01-26T12:56:41Z

Hello @LaurenceJJones

Thanks for your reply,

im using docker-compose with no env variable:

version: "3.4"

services:
crowdsec:
image: crowdsecurity/crowdsec
container_name: crowdsec
networks:
- proxy
ports:
- 9080:8080
environment:
PGID: "1000"
#COLLECTIONS: "crowdsecurity/traefik crowdsecurity/http-cve"
volumes:
- ./data:/var/lib/crowdsec/data
- ./data/:/etc/crowdsec
- /var/log/auth.log:/var/log/auth.log:ro
- /var/log/crowdsec:/var/log/crowdsec:ro
restart: always

crowdsec-traefik-bouncer:
image: fbonalair/traefik-crowdsec-bouncer
container_name: bouncer-traefik
networks:
- proxy
environment:
CROWDSEC_BOUNCER_API_KEY: MYAPI
CROWDSEC_AGENT_HOST: crowdsec:8080
GIN_MODE: release
depends_on:
- crowdsec
restart: always

networks:
proxy:
external: true

mushuthecat · 2024-01-26T13:02:07Z

Hi,

I rolled back to 1.5.5 and now everything is working properly

m3e-g · 2024-01-26T19:09:49Z

I had the same problem, after I've updated container today to the :latest

Both rollback to the :v1.5.5 and :dev are working properly at the moment.

blotus · 2024-02-02T13:37:37Z

We have released a v1.6.0-1 tag (and updated latest) with a fix for this.

bufanda added the kind/bug Something isn't working label Jan 25, 2024

github-actions bot added the needs/triage label Jan 25, 2024

mmetc mentioned this issue Jan 30, 2024

Decouple docker image from package release #2791

Merged

blotus closed this as completed Feb 2, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

v1.6.0 container fails to start on crowdsec/whiltelist installation #2786

v1.6.0 container fails to start on crowdsec/whiltelist installation #2786

bufanda commented Jan 25, 2024

github-actions bot commented Jan 25, 2024

LaurenceJJones commented Jan 25, 2024

LaurenceJJones commented Jan 25, 2024

LaurenceJJones commented Jan 25, 2024 •

edited

LaurenceJJones commented Jan 25, 2024 •

edited

bufanda commented Jan 26, 2024

mushuthecat commented Jan 26, 2024

LaurenceJJones commented Jan 26, 2024

mushuthecat commented Jan 26, 2024

mushuthecat commented Jan 26, 2024

m3e-g commented Jan 26, 2024

blotus commented Feb 2, 2024

v1.6.0 container fails to start on crowdsec/whiltelist installation #2786

v1.6.0 container fails to start on crowdsec/whiltelist installation #2786

Comments

bufanda commented Jan 25, 2024

What happened?

What did you expect to happen?

How can we reproduce it (as minimally and precisely as possible)?

Anything else we need to know?

Crowdsec version

OS version

Enabled collections and parsers

Config show

Related custom configs versions (if applicable) : notification plugins, custom scenarios, parsers etc.

github-actions bot commented Jan 25, 2024

LaurenceJJones commented Jan 25, 2024

LaurenceJJones commented Jan 25, 2024

LaurenceJJones commented Jan 25, 2024 • edited

LaurenceJJones commented Jan 25, 2024 • edited

bufanda commented Jan 26, 2024

mushuthecat commented Jan 26, 2024

LaurenceJJones commented Jan 26, 2024

mushuthecat commented Jan 26, 2024

mushuthecat commented Jan 26, 2024

m3e-g commented Jan 26, 2024

blotus commented Feb 2, 2024

LaurenceJJones commented Jan 25, 2024 •

edited

LaurenceJJones commented Jan 25, 2024 •

edited