Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Port 443 connection refused with gickup, gitea, and traefik #242

Open
Aetherinox opened this issue Jun 1, 2024 · 8 comments
Open

Port 443 connection refused with gickup, gitea, and traefik #242

Aetherinox opened this issue Jun 1, 2024 · 8 comments

Comments

@Aetherinox
Copy link

Aetherinox commented Jun 1, 2024

Hey, looking to see if I can get some direction here.

Recently I moved my system from bare metal over to docker and traefik.
Within the gickup console, I've noticed the following error:

2024-06-01 16:25:35 INF mirroring text-encoding to https://git.mydomain.com stage=gitea url=https://git.mydomain.com

2024-06-01 16:25:35 INF Read token file bytes=41 path=/gickup/tokens/gitea.txt

>>>> 2024-06-01 16:25:35 ERR Get "https://git.mydomain.com/api/v1/version": dial tcp 172.18.0.20:443: connect: connection refused stage=gitea url=https://git.mydomain.com

2024-06-01 16:25:35 INF starting backup for https://github.com/keeweb/xmldom.git stage=backup

2024-06-01 16:25:35 INF cloning github.com/keeweb/xmldom/1717284335 path=/gickup/backup stage=locally

2024-06-01 16:25:35 INF zipping github.com/keeweb/xmldom/1717284335 path=/gickup/backup stage=locally

2024-06-01 16:25:35 INF removing /gickup/backup/github.com/keeweb/xmldom/1717283138.zip path=/gickup/backup stage=locally

2024-06-01 16:25:35 WRN mirrorinterval is deprecated and will be removed in one of the next releases. please move it under the mirror parameter. stage=gitea url=https://git.mydomain.com

2024-06-01 16:25:35 INF mirroring xmldom to https://git.mydomain.com stage=gitea url=https://git.mydomain.com

2024-06-01 16:25:35 INF Read token file bytes=41 path=/gickup/tokens/gitea.txt

The odd thing is that if I look at my Gitea repos, it does appear to still be syncing, but this error gets thrown right before every single repo goes to sync.

I've ensured Traefik has port 443 open, I've set up SSH port 22, I even tried using docker expose: 443 between gickup and gitea, and nothing seems to be working.

I've spent hours looking through other people's repos on Github to see if I'm missing anything, and there's nothing I can find. My configs look identical to other peoples.

I went into Gitea's API, and physically tried to access api/v1/version, and it returned fine

https://git.domain.com/api/v1/version?token=MYAPITOKEN
{"version":"1.22.0"}

Edit: The really weird thing is, if I bring up gickup, gitea, and let the sync start, it will throw the error. Yet if I take down gitea and bring it back up, the error stops.

2024-06-01 18:30:22 INF starting backup for https://github.com/Aetherinox/debian-opengist.git stage=backup
2024-06-01 18:30:22 INF cloning github.com/Aetherinox/debian-opengist/1717291822 path=/gickup/backup stage=locally
2024-06-01 18:30:31 INF zipping github.com/Aetherinox/debian-opengist/1717291822 path=/gickup/backup stage=locally
2024-06-01 18:30:33 INF removing /gickup/backup/github.com/Aetherinox/debian-opengist/1717291222.zip path=/gickup/backup stage=locally
2024-06-01 18:30:33 WRN mirrorinterval is deprecated and will be removed in one of the next releases. please move it under the mirror parameter. stage=gitea url=https://git.mydomain.com
2024-06-01 18:30:33 INF mirroring debian-opengist to https://git.mydomain.com stage=gitea url=https://git.mydomain.com
2024-06-01 18:30:33 INF Read token file bytes=41 path=/gickup/tokens/gitea.txt
2024-06-01 18:30:33 ERR Get "https://git.mydomain.com/api/v1/version": dial tcp 172.18.0.20:443: connect: connection refused stage=gitea url=https://git.mydomain.com

--- This is when I take Gitea down ---

2024-06-01 18:30:38 ERR Unknown API Error: 404
Request: '/api/v1/version' with 'GET' method 'map[Authorization:[token 63811964817ac354db06e6241ee0ecebf391d0d7]]' header and '404 page not found ' body stage=gitea url=https://git.mydomain.com
2024-06-01 18:30:38 INF starting backup for https://github.com/Aetherinox/eslint-config.git stage=backup
2024-06-01 18:30:38 INF cloning github.com/Aetherinox/eslint-config/1717291838 path=/gickup/backup stage=locally
2024-06-01 18:30:38 INF zipping github.com/Aetherinox/eslint-config/1717291838 path=/gickup/backup stage=locally
2024-06-01 18:30:38 INF removing /gickup/backup/github.com/Aetherinox/eslint-config/1717290634.zip path=/gickup/backup stage=locally
2024-06-01 18:30:38 WRN mirrorinterval is deprecated and will be removed in one of the next releases. please move it under the mirror parameter. stage=gitea url=https://git.mydomain.com
2024-06-01 18:30:38 INF mirroring eslint-config to https://git.mydomain.com stage=gitea url=https://git.mydomain.com
2024-06-01 18:30:38 INF Read token file bytes=41 path=/gickup/tokens/gitea.txt
2024-06-01 18:30:38 ERR Unknown API Error: 404

--- This is when Gitea is back up, No error---

2024-06-01 18:32:39 WRN mirrorinterval is deprecated and will be removed in one of the next releases. please move it under the mirror parameter. stage=gitea url=https://git.mydomain.com
2024-06-01 18:32:39 INF mirroring obsidian-gistr to https://git.mydomain.com stage=gitea url=https://git.mydomain.com
2024-06-01 18:32:39 INF Read token file bytes=41 path=/gickup/tokens/gitea.txt
2024-06-01 18:32:39 INF mirror of obsidian-gistr already exists, syncing instead stage=gitea url=https://git.mydomain.com
2024-06-01 18:32:40 INF successfully synced obsidian-gistr. stage=gitea url=https://git.mydomain.com
2024-06-01 18:32:40 INF starting backup for https://github.com/Aetherinox/obsidian-gistr.wiki.git stage=backup
2024-06-01 18:32:40 INF cloning github.com/Aetherinox/obsidian-gistr.wiki/1717291960 path=/gickup/backup stage=locally
2024-06-01 18:32:40 INF zipping github.com/Aetherinox/obsidian-gistr.wiki/1717291960 path=/gickup/backup stage=locally
2024-06-01 18:32:40 INF removing /gickup/backup/github.com/Aetherinox/obsidian-gistr.wiki/1717290158.zip path=/gickup/backup stage=locally
2024-06-01 18:32:40 WRN mirrorinterval is deprecated and will be removed in one of the next releases. please move it under the mirror parameter. stage=gitea url=https://git.mydomain.com
2024-06-01 18:32:40 INF starting backup for https://github.com/Aetherinox/obsidian-gistr.git stage=backup
2024-06-01 18:32:40 INF cloning github.com/Aetherinox/obsidian-gistr/1717291960 path=/gickup/backup stage=locally
2024-06-01 18:32:46 INF zipping github.com/Aetherinox/obsidian-gistr/1717291960 path=/gickup/backup stage=locally
2024-06-01 18:32:48 INF removing /gickup/backup/github.com/Aetherinox/obsidian-gistr/1717290159.zip path=/gickup/backup stage=locally
2024-06-01 18:32:48 WRN mirrorinterval is deprecated and will be removed in one of the next releases. please move it under the mirror parameter. stage=gitea url=https://git.mydomain.com

It however, doesn't matter if I bring Gitea up last on initial startup. The error still occurs. The error only disappears if I take Gitea offline and bring it back up mid sync. Then it doesn't error anymore after that.

@Aetherinox
Copy link
Author

Aetherinox commented Jun 2, 2024

Another update. It appears I get two different results depending on which CURL I use

Works

curl "https://git.domain.com/api/v1/version" \
    -H "accept: application/json" \
    -H "Authorization: token MYTOKEN" -i

Connection Refused

curl "https://172.18.0.20:443/api/v1/version" \
    -H "accept: application/json" \
    -H "Authorization: token MYTOKEN" -i

But I'm not sure where it's pulling the IP from so that I can change it. Or how to open the local IP.

Even though I've defined the domain for all the configs.

The only other way I've found to kill the error, is if I set gickup to run on a cron every 5 minutes. And then bring Gitea + gickup up. By the time Gitea fully gets booted, Gickup has started the sync, and gitea fully starts mid-sync which silences the error

@Aetherinox
Copy link
Author

Another update, in case you have anything to add, it would be helpful. Would like to know why this initially happened.

I went back into my conf.yml and changed the url from my domain, to the localhost ip / port for gitea:

destination:
  gitea:
      url: https://mydomain.com

Changed To:

destination:
  gitea:
      url: http://172.18.0.20:3000/

This seemed to stop the error all together, and it is now syncing without any mention of a connection being refused. Not sure why though. Seeing as I couldn't curl locally earlier.

@cooperspencer
Copy link
Owner

The only thing I can think of is a problem within the DNS settings.

You said that Gitea is still mirroring. That is normal, Gickup uses Giteas' API to create a mirror and Gitea handles this mirror and resyncs it according to your configured interval.

I will try to recreate your setup and see if I can find out what's going on.
So you use docker and run Traefik, Gitea and Gickup?

@cooperspencer
Copy link
Owner

What you could try as well, is to use the name of your gitea container in gickups' config file. Like 'http://gitea:3000'

@Aetherinox
Copy link
Author

Aetherinox commented Jun 2, 2024

Edit: I'll try the gitea container name.

Yeah would be interesting to know. I've triple checked my settings.
I'm running:

  • Docker
  • Traefik
  • Gitea
  • Gickup

All current versions.

I'll provide the composer files below. I took out the stuff like ElasticSearch since it's useless here:

Gickup

docker-compose.yml

services:

  gickup:
    container_name: "gickup"
    image: buddyspencer/gickup:latest
    hostname: gickup.${SERVER_DOMAIN}
    volumes:
      - ${PWD}/conf.yml:/gickup/conf.yml
      - ${PWD}/backup:/gickup/backup
      - ${PWD}/logs:/gickup/logs
      - ${PWD}/tokens/gitea.txt:/gickup/tokens/gitea.txt
      - ${PWD}/tokens/github.txt:/gickup/tokens/github.txt
    command: ["/gickup/conf.yml"]
    environment:
        PUID: ${UID}
        PGID: ${GID}
    networks:
      traefik:
        ipv4_address: "${SERVICE_IP}"
    labels:
      - traefik.enable=true
      - traefik.constraint-label=traefik
      - traefik.docker.network=${SERVER_NETWORK}

      - traefik.http.middlewares.https-redirect.redirectscheme.scheme=https
      - traefik.http.middlewares.https-redirect.redirectscheme.permanent=true

      - traefik.http.routers.gickup-http.rule=Host(`gickup.localhost`) || Host(`gickup.${SERVER_DOMAIN}`) || Host(`www.gickup.${SERVER_DOMAIN}`) || Host(`${SERVICE_IP}`)
      - traefik.http.routers.gickup-http.service=gickup
      - traefik.http.routers.gickup-http.entrypoints=http
      - traefik.http.routers.gickup-http.priority=1
      - traefik.http.routers.gickup-http.middlewares=https-redirect

      - traefik.http.routers.gickup-https.rule=Host(`gickup.localhost`) || Host(`gickup.${SERVER_DOMAIN}`) || Host(`www.gickup.${SERVER_DOMAIN}`) || Host(`${SERVICE_IP}`)
      - traefik.http.routers.gickup-https.service=gickup
      - traefik.http.routers.gickup-https.entrypoints=https
      - traefik.http.routers.gickup-https.priority=1
      - traefik.http.routers.gickup-https.tls=true
      - traefik.http.routers.gickup-https.tls.certresolver=cloudflare
      - traefik.http.routers.gickup-https.tls.domains[0].main=${SERVER_DOMAIN}
      - traefik.http.routers.gickup-https.tls.domains[0].sans=*.${SERVER_DOMAIN}

      # Tried adding this, but it made no difference

      # - traefik.http.services.gickup.loadbalancer.server.port=443
      # - traefik.http.services.gickup.loadbalancer.server.scheme=https

networks:
  traefik:
    name: ${SERVER_NETWORK}
    external: true

Gickup .env

SERVER_DOMAIN=domain.com
SERVER_IP=XX.XX.XX.XX
SERVER_NETWORK=traefik

UID=143
GID=997

SERVICE_IP=172.18.0.9

Gitea

docker-compose.yml:

services:
  gitea:
    container_name: gitea
    image: gitea/gitea
    hostname: git.${SERVER_DOMAIN}
    restart: always
    networks:
      traefik:
        ipv4_address: "${GITEA_IP}"
    environment:
      PUID: ${UID}
      PGID: ${GID}
      USER_UID: ${UID}
      USER_GID: ${GID}
      GITEA__database__DB_TYPE: '${GITEA_DB_TYPE}'
      GITEA__database__HOST: '${GITEA_DB_HOST}'
      GITEA__database__NAME: '${GITEA_DB_NAME}'
      GITEA__database__USER: '${GITEA_DB_USER}'
      GITEA__database__PASSWD: '${GITEA_DB_PASS}'
      GITEA_CUSTOM: '${GITEA_DIR_CUSTOM}'
    volumes:
      - /etc/timezone:/etc/timezone:ro
      - /etc/localtime:/etc/localtime:ro
      - ${PWD}/data:/data
    healthcheck:
      test: ["CMD", "curl", "-f", "http://127.0.0.1:3000"]
      interval: 2s
      timeout: 10s
      retries: 15
    labels:
      - traefik.enable=true
      - traefik.constraint-label=traefik
      - traefik.docker.network=${SERVER_NETWORK}

      - traefik.http.routers.gitea-http.rule=Host(`git.localhost`) || Host(`git.${SERVER_DOMAIN}`) || Host(`www.git.${SERVER_DOMAIN}`) || Host(`${GITEA_IP}`)
      - traefik.http.routers.gitea-http.service=gitea
      - traefik.http.routers.gitea-http.entrypoints=http
      - traefik.http.routers.gitea-http.middlewares=https-redirect@file

      - traefik.http.routers.gitea-https.rule=Host(`git.localhost`) || Host(`git.${SERVER_DOMAIN}`) || Host(`www.git.${SERVER_DOMAIN}`) || Host(`${GITEA_IP}`)
      - traefik.http.routers.gitea-https.service=gitea
      - traefik.http.routers.gitea-https.entrypoints=https
      - traefik.http.routers.gitea-https.tls=true
      - traefik.http.routers.gitea-https.tls.certresolver=cloudflare
      - traefik.http.routers.gitea-https.tls.domains[0].main=${SERVER_DOMAIN}
      - traefik.http.routers.gitea-https.tls.domains[0].sans=*.${SERVER_DOMAIN}

      - traefik.http.services.gitea.loadbalancer.server.port=3000
      - traefik.http.services.gitea.loadbalancer.server.scheme=http

      - traefik.tcp.routers.gitea-ssh.rule=HostSNI(`*`)
      - traefik.tcp.routers.gitea-ssh.entrypoints=gitea-ssh      
      - traefik.tcp.routers.gitea-ssh.tls=true
      - traefik.tcp.routers.gitea-ssh.service=gitea-ssh
      - traefik.tcp.services.gitea-ssh.loadbalancer.server.port=22

networks:
  traefik:
    name: ${SERVER_NETWORK}
    external: true

Gitea .env:

SERVER_DOMAIN=domain.com
SERVER_IP=XX.XX.XX.XX
SERVER_NETWORK=traefik

UID=143
GID=997

GITEA_IP=172.18.0.20
GITEA_DB_TYPE=mysql
GITEA_DB_HOST=mariadb:3306
GITEA_DB_NAME=XXXXX
GITEA_DB_USER=XXXXX
GITEA_DB_PASS=XXXXX
GITEA_DIR_CUSTOM=/data/custom

Traefik Static File:

The Traefik traefik.yml static file is the typical file, just with this added:

entryPoints:
  gitea-ssh:
    address: :22/tcp

Traefik Dynamic.yml file:

http:
  middlewares:
    https-redirect:
      redirectScheme:
        scheme: "https"
        permanent: true

@Aetherinox
Copy link
Author

Aetherinox commented Jun 2, 2024

Thanks. http://gitea:3000 works. I should have done it earlier anyway, but I've worked on this issue since I initially put in the issue report, so my brain is fried for the day. Plus I spent two hours reading through the gickup code to see how things like URL were called and if I could override the CURL url.

I need to change the cron time. It's set to every 5 minutes, so it never stops. As soon as it finishes backing up, it starts again.

@marcuswoy
Copy link

@Aetherinox, Could you share your latest changes to make http://gitea:3000 possible? I appreciate it.

@Aetherinox
Copy link
Author

Sorry, been a month since I wrote this, brain needs refreshing.

In my gickup config.yml I used

destination:
  gitea:
    - token_file: /gickup/tokens/gitea.txt
      user: xxxxxxx
      url: http://gitea:3000/                  <--------
      createorg: true
      mirrorinterval: 1h0m0s
      lfs: false
      visibility:
        repositories: private
        organizations: private

gitea is the name of the docker container, so out of box, you should be able to call any container by its name within other configs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants