Troubleshooting

Troubleshooting

Common Issues

Webhooks Return 401

Problem: GitHub or GitLab webhook deliveries show 401 responses.

Check:

• Verify GITHUB_WEBHOOK_SECRET matches between .env and the GitHub webhook settings
• Verify GITLAB_WEBHOOK_SECRET matches between .env and the GitLab webhook settings
• Regenerate secrets and update both sides if uncertain
• Check hub2lab logs: Invalid GitHub webhook signature or Invalid GitLab webhook token

Webhooks Return 400

Problem: Webhook deliveries return 400 "Unknown event".

Check:

• hub2lab only processes push, create, delete, and repository events from GitHub
• hub2lab only processes push and tag_push events from GitLab
• Events like pull_request, issues, ping are logged as "Unhandled event type" but return 400
• This is harmless — the event is simply ignored

Sync Not Working

Problem: Pushes on one side don't appear on the other.

Checklist:

1. Tokens — Do GITHUB_TOKEN and GITLAB_TOKEN have the correct scopes?

   • GitHub: repo (classic) or Contents read/write (fine-grained)
   • GitLab: api, write_repository

2. Mapping — Is the mapping configured correctly?

   • Does github_org match the org in the webhook payload?
   • Does gitlab_group exist?
   • If using explicit repos, are the repo names spelled correctly?

3. Webhook delivery — Check GitHub/GitLab webhook delivery logs for successful deliveries

4. hub2lab logs — Look for error messages:

   ERROR] GitHub event processing failed { error: '...' }
   WARN] GitLab project not found: my-group/my-repo, creating
   

5. Mirror directory — Check if MIRROR_DIR is writable. hub2lab clones repos to this directory.

6. Loop prevention — If commits contain the SYNC_MARKER (default [hub2lab]), they're intentionally skipped

Mirror Clone Fails

Problem: hub2lab can't clone the source repository.

Causes:

• Invalid or expired GITHUB_TOKEN or GITLAB_TOKEN
• Token doesn't have access to the specific repository
• Network connectivity issues between hub2lab and the git servers
• Repository doesn't exist on the source side (race condition)

Logs to look for:

ERROR] GitHub event processing failed { error: 'Git clone failed: ...' }

Deletion Marker Files Appearing

Problem: I see files like deletion-branch-feature-x.md appearing in my repos.

Explanation: This is by design. hub2lab never automatically deletes branches, tags, or repos. When a delete event occurs on one side, it writes a marker file to the other side. This prevents accidental data loss.

To complete a deletion:

1. Verify the deletion was intentional
2. Delete the branch/tag/repo manually on the target side
3. Remove the marker file (or it can remain as a record)

Port Already in Use

Problem: EADDRINUSE error when starting hub2lab.

Solution: Either:

• Stop the process already using port 3000
• Change PORT in .env to a different value (e.g., 3001)
• Update your webhook URLs to match the new port

Docker: Permission Denied

Problem: Docker container can't write to mounted volumes.

Solution: The container runs as the node user (UID 1000). Ensure mounted directories have correct permissions:

chown -R 1000:1000 /opt/hub2lab/logs
chown -R 1000:1000 /opt/hub2lab/mirrors

Debugging

Enable Debug Logging

Set LOG_LEVEL=debug in .env:

LOG_LEVEL=debug

This logs detailed information about:

• Webhook payload structure
• Mapping resolution attempts
• Git mirror operations
• API call details

Check Mirror Directory

hub2lab maintains bare git mirrors under MIRROR_DIR. You can inspect them:

# List mirrors
ls -la /path/to/mirrors/

# Check the remote of a mirror
cd /path/to/mirrors/<hash>
git remote -v

# View recent git activity in a mirror
git log --oneline -10

Test Webhook with curl

Simulate a GitHub push event:

curl -X POST http://localhost:3000/webhook/github \
  -H "Content-Type: application/json" \
  -H "X-GitHub-Event: push" \
  -H "X-Hub-Signature-256: sha256=$(echo -n '{"ref":"refs/heads/test","repository":{"name":"test-repo","full_name":"test-org/test-repo","owner":{"login":"test-org"}}}' | openssl dgst -sha256 -hmac "$GITHUB_WEBHOOK_SECRET" | cut -d' ' -f2)" \
  -d '{"ref":"refs/heads/test","repository":{"name":"test-repo","full_name":"test-org/test-repo","owner":{"login":"test-org"}}}'

Simulate a GitLab push event:

curl -X POST http://localhost:3000/webhook/gitlab \
  -H "Content-Type: application/json" \
  -H "X-GitLab-Token: $GITLAB_WEBHOOK_SECRET" \
  -d '{"event_name":"push","ref":"refs/heads/test","project":{"name":"test-repo","path_with_namespace":"test-group/test-repo","namespace":"test-group"}}'

Getting Help

If you encounter issues not covered here:

1. Check logs — Review hub2lab logs at LOG_DIR (default: ./logs/)
2. GitHub Issues — Open an issue at https://github.com/wildfirebill-docker/hub2lab/issues
3. Include:
   • hub2lab version
   • Log output at debug level
   • Relevant configuration (with secrets redacted)
   • Steps to reproduce