Bouncer can't connect to CrowdSec instance

[SAOPP]

Hi!

Face with strange behaviour, I have an several same install of crowdsec and traefik bouncer in docker, and right now I install this stuff to my vps, but, not understand what is wrong, maybe someone can help and resolve it, traefik bouncer can't connect to the crowdsec instance, 'coz ip address of container in not in bouncers list, and I can't understand whats is wrong in my compose or my settings of current stack.

Here is an screen of bouncers list:

Here is an my containers:

And my stack:

version: '3'

networks:
  proxy:
    external: true

volumes:
  config:
    driver: local
  db:
    driver: local

services:

  crowdsec:
    image: crowdsecurity/crowdsec:latest
    container_name: crowdsec
    networks:
      - proxy
    environment:
      - TZ=Europe/Kiev
      - COLLECTIONS="crowdsecurity/linux crowdsecurity/traefik"
      - GID=${GID-1000}
    volumes:
      - ${DOCKER_HOST_PATH}/crowdsec/config/acquis.yaml:/etc/crowdsec/acquis.yaml
      - ${DOCKER_HOST_PATH}/crowdsec/config/profiles.yaml:/etc/crowdsec/profiles.yaml
      - ${DOCKER_HOST_PATH}/crowdsec/notifications/http.yaml:/etc/crowdsec/notifications/http.yaml
      - db:/var/lib/crowdsec/data/
      - config:/etc/crowdsec/
      # Traefik's logs read-only
      - ${DOCKER_HOST_PATH}/traefik/logs:/var/log/traefik:ro
      # Authelia's logs read-only
      #- ${DOCKER_HOST_PATH}/authelia/authelia.log:/var/log/authelia.log:ro
    restart: unless-stopped
    logging:
      driver: "json-file"
      options:
        max-size: "35m"
      
  traefik-crowdsec-bouncer:
    image: docker.io/fbonalair/traefik-crowdsec-bouncer:latest
    container_name: traefik-crowdsec-bouncer
    networks:
      - proxy
    depends_on:
      - crowdsec
    environment:
      - TZ=Europe/Kiev
      - CROWDSEC_BOUNCER_API_KEY=${BOUNCER_TRAEFIK_API_KEY}
      - CROWDSEC_AGENT_HOST=crowdsec:8080
      - CROWDSEC_BOUNCER_LOG_LEVEL=0
    restart: unless-stopped
    logging:
      driver: "json-file"
      options:
        max-size: "10m"

The only difference from my previous stacks: I added telegram alerts, and my internal docker network is 192.168.135.0/24 - and that's all differents.

[SAOPP]

I tried different scenarios, specified the IP address of CrowdSec container, added another container from the default docker subnet, I don’t understand what the problem is and why the CrowdSec behaves like this.

[tinolin]

As I have it with SWARM, I have this variable: "CROWDSEC_AGENT_HOST=192.168.135.8:8080" in the definition of the service "traefik-crowdsec-bouncer" in this way, since it must be explicit, because of how swarm makes the names of The containers

[SAOPP]

But, I don’t have a swarm, and this option didn't work for me, I tried to add the address of the container natively. :(

[SAOPP]

Damn it, I tried all possible options, I just broke my head, I don’t understand where to dig, nonsense. 😞

[SAOPP]

Is there any chance to get more debug logs from the bouncer?

[tinolin]

jaja take it easy my friend!! what docker command do you run? can you telnet to the port?

I leave my configuration for you to try:

version: "3.8"

services:
  crowd_sec:
    container_name: "crowdsec-example"
    image: crowdsecurity/crowdsec:v1.3.2    
    environment:
      - GID="${GID-1000}"
      - COLLECTIONS="crowdsecurity/linux crowdsecurity/traefik"    
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /etc/timezone:/etc/timezone:ro    
      - /mnt/data/docker/crowdsec/config/acquis.yaml:/etc/crowdsec/acquis.yaml
      - /mnt/data/docker/traefik-logs/:/var/log/traefik/:ro
      - /var/log/auth.log:/var/log/auth.log:ro
      - crowdsec_db:/var/lib/crowdsec/data/
      - crowdsec_config:/etc/crowdsec/
    ports:
     - target: 8080
       published: 8085
       protocol: tcp
       mode: host
     - target: 6060
       published: 8086
       protocol: tcp
       mode: host         
    restart: unless-stopped
    networks:
      - traefik-public

  bouncer:
    image: fbonalair/traefik-crowdsec-bouncer:latest
    container_name: "bouncer"
    environment:
      - CROWDSEC_BOUNCER_API_KEY=
      - CROWDSEC_AGENT_HOST=192.168.135.8:8085
      - PORT=8084
      - GIN_MODE=release
    restart: unless-stopped
    ports:
     - target: 8084
       published: 3303
       protocol: tcp
       mode: host    
    networks:
      - traefik-public      
    depends_on:
      - 'crowdsec-example'
      - crowd_sec
    deploy:
      mode: replicated
      replicas: 1
      labels:
        - traefik.enable=true
        - traefik.http.routers.bouncer.rule=Host(`bouncer.example.com`)
        - traefik.http.services.bouncer.loadbalancer.server.port=8084
        - traefik.http.routers.bouncer.tls.certresolver=letsencryptresolver
        - traefik.http.routers.bouncer.tls=true
        - traefik.http.routers.bouncer.entrypoints=websecure
        #- traefik.http.routers.bouncer.middlewares=middlewares-secure-headers@file    


networks:
  traefik-public:
      external: true

volumes:
        crowdsec_db:
                driver: local
                driver_opts:
                        type: volume
                        o: bind
                        device: /mnt/data/docker/crowdsec/db
        crowdsec_config:
                driver: local
                driver_opts:
                        type: volume
                        o: bind
                        device: /mnt/data/docker/crowdsec/config
        traefik-logs:
                external: true
# mkdir -p /mnt/data/docker/crowdsec/{config,db,data}
# touch /mnt/data/docker/crowdsec/config/acquis.yaml 

you can try on host:

root@myhost:~# telnet 192.168.135.8 8085
Trying 192.168.135.8...
Connected to 192.168.135.8.
Escape character is '^]'.

what does the console show you?

[SAOPP]

Telnet is answered the same as yours, btw I tried and at my working instances, the same telnet result.

[SAOPP]

nmap also:

nmap -sT -p- 192.168.135.8
Starting Nmap 7.80 ( https://nmap.org ) at 2022-04-14 11:24 EEST
Nmap scan report for 192.168.135.8
Host is up (0.00017s latency).
Not shown: 65533 closed ports
PORT     STATE SERVICE
6060/tcp open  x11
8080/tcp open  http-proxy
MAC Address: 02:42:C0:A8:87:08 (Unknown)

Nmap done: 1 IP address (1 host up) scanned in 2.26 seconds

[SAOPP]

Run your stack near my, the same... :( Well, I don't know.

[fbonalair]

I am sorry I am extremely busy those months so I can't extensively help you. Some general thoughts:

• You have put CROWDSEC_BOUNCER_LOG_LEVEL to 0, can you past your logs here?
• To be sure, you are using docker compose to put up your whole stack?

To cross off CrowdSec issue:

• Verify again that the container is healthy
• Create a new Bouncer API key

To cross off Network issue:

• For bough Crowdsec and the bouncer container I see networks: - proxy, just verify it's what your currently using.
• Do you have any additional firewall / network configuration in your stack?
• Instead of the bouncer image, can you put another container, with curl and exacly the same configuration, and call your crowdsec endpoint (see their swagger) with the previously generated key.

To cross off Bouncer issue:

• The bouncer have it's own healthcheck route, can you call the route /api/v1/ping?
• It also have it's own heach checker, can you call it ? It should be at the root of the container /healthchecker

I hope it helps

[SAOPP]

• You have put CROWDSEC_BOUNCER_LOG_LEVEL to 0, can you past your logs here?

Yes, I tried this env, and nothing additional infos in log was added.
This logs is not changes if I added level 0:

• To be sure, you are using docker compose to put up your whole stack?

Yes exactly.

• Verify again that the container is healthy
• Create a new Bouncer API key

Yes, I tried this, and also tried run several bouncers, the same result :(

• For bough Crowdsec and the bouncer container I see networks: - proxy, just verify it's what your currently using.

Sure, it's my network for this docker instance and all my container use it, exclude portainer agent container.

• Do you have any additional firewall / network configuration in your stack?

Nope, I have only several compose near this Crowdsec compose.

• Instead of the bouncer image, can you put another container, with curl and exacly the same configuration, and call your crowdsec endpoint (see their swagger) with the previously generated key.

As I understood u mean call via curl include my api token of bouncer right? Can u help with generate it curl command please?

• The bouncer have it's own healthcheck route, can you call the route /api/v1/ping?

• It also have it's own heach checker, can you call it ? It should be at the root of the container /healthchecker

If I curl this, get 404 page not found, but, if I understood u correctly, it's a container healthcheck? Container is showing Healhy

Sorry if I understood u incorrectly about this question.

[SAOPP]

Very strange, got the same stuff at my local instance, but all work like a magic:

[SAOPP]

Okay guys, I was probably wrong, and I just started panicking myself without understanding some points. But I’ll make a reservation, when I previously connected bouncers, I immediately had their addresses in the list of bouncers, I don’t know, maybe I’m wrong. In general, I decided to add a label with a middleware for one of my applications, and look at the bouncer log, I saw that it was transmitting addresses and trying to get a decision, after which I went to look at the metrics and list of bouncers - and lo and behold :) I saw how my bouncer became connected.

I should probably ask for forgiveness, probably this moment should be described in the readme, because, for example, it was not obvious to me, and again I repeat, earlier I immediately saw the addresses of bouncers without even using them.

Think, the issue can be closed.