Operations

Operations

Day-2 operations for a running instance: taking individual pages offline, reading logs, managing the service, refreshing game data, understanding the public API limits, reverting translation overrides, and knowing what to back up. This page assumes the production layout from Deployment: systemd service hailsdotgo running from /opt/hailsdotgo behind Caddy.

Per-page maintenance toggles

Admins can disable individual pages and sections from the admin panel (/admin) without touching the server. A disabled page renders the in-app maintenance template (templates/maintenance.html) with HTTP 503 while the rest of the site stays up; Raid Finder API calls return a JSON 503.

The toggles persist in the site_settings table, so you can also flip them by SQL if the admin panel itself is unreachable:

Setting key	Controls
page_raids_enabled	Raids page
page_dps_enabled	DPS calculator
page_pvp_enabled	PvP IV ranker
page_events_enabled	Events page
page_trainers_enabled	Trainers page
section_trainer_directory_enabled	Directory section within Trainers
section_raid_finder_enabled	Raid Finder section within Trainers
page_shinies_enabled	Personal shiny tracker

'1' is enabled, anything else is disabled:

UPDATE site_settings SET setting_value = '0' WHERE setting_key = 'page_raids_enabled';

The static maintenance fallback page

static/maintenance.html is a self-contained "Updating..." page for the moments when the whole binary is down (deploys, restarts). It needs no backend: it polls /api/data every 5 seconds and reloads itself the instant the server answers with JSON again, so visitors return to the site automatically. It is deployed to /opt/hailsdotgo/static/maintenance.html and served at /static/maintenance.html; to show it automatically while the app is down, point your reverse proxy's error handling at it (for example a Caddy handle_errors block serving the file on 502).

Logs

The app logs to stdout, captured by journald:

# Current status plus the last few lines
systemctl status hailsdotgo

# Follow live
journalctl -u hailsdotgo -f

# Last 100 lines, no pager
journalctl -u hailsdotgo -n 100 --no-pager

# Everything since a point in time
journalctl -u hailsdotgo --since "1 hour ago"
journalctl -u hailsdotgo --since "2026-06-11 00:00:00"

The request log line per HTTP request comes from chi's logger middleware; startup lines show which game data loaded from disk cache versus upstream, and CSRF failures are logged with their reason.

Service management

systemctl start hailsdotgo
systemctl stop hailsdotgo
systemctl restart hailsdotgo
systemctl is-active hailsdotgo
systemctl daemon-reload        # after editing /etc/systemd/system/hailsdotgo.service

Restart=always means a crashed process is back within about 5 seconds; check the journal for the panic or fatal log if that happens repeatedly.

The OCR microservice runs under its own unit (hailsdotgo-ocr); manage and read it the same way (systemctl status hailsdotgo-ocr, journalctl -u hailsdotgo-ocr -f). It is optional: if it is down, only the IV Calculator screenshot scan is affected (it returns empty fields) and the rest of the site is unaffected. See Deployment.

Upgrading the database

After pulling a new version, apply any new migration sections before (re)starting the service. The migrate tool does this for you:

go run ./cmd/migrate -status         # see what is pending
go run ./cmd/migrate                 # apply pending sections

On a database that has never been tracked, baseline it once with go run ./cmd/migrate -from <your version>. See Database-Guide for details and the manual fallback.

Game data refresh

Game data refreshes itself; you rarely need to intervene:

• Pokemon stats, moves, shinies, type chart, CP multipliers (PoGoAPI): every 6 hours
• Raid bosses and Max Battles: every 4 hours on a Mountain Time schedule
• Events feed: every 30 minutes; scraped event detail pages: every 12 hours
• All fetched data is cached in CACHE_DIR and a stale cache (or the embedded snapshot) is used when an upstream fetch fails

To force a refresh, either use the superadmin refresh action in the admin panel, or call the API:

curl -X POST https://your.server.com/api/refresh

POST /api/refresh requires either the superadmin account or an account with the api_access flag (granted by the superadmin in the admin Users tab). It is globally rate limited to 2 calls per 10 minutes across all callers. It kicks off the re-fetch in the background and returns immediately; watch the journal to see the fetches land.

Public API limits

Verified against internal/server/server.go and internal/server/bwlimit.go:

• Per-IP request limits: 10 requests per 2 minutes on each public data endpoint (/api/data, /api/raids, /api/maxbattles, /api/events, /api/pokemon, /api/moves); 30 per 2 minutes on /api/events/{id}.
• Per-IP bandwidth cap: 15 MB per rolling 5-minute window, counted across all public data endpoints combined. Exceeding it blocks that IP for 30 minutes; blocked requests get HTTP 429 with a JSON error.
• The client IP is taken from X-Real-IP, then the first entry of X-Forwarded-For, then the socket address, so the limiter works correctly behind Caddy.
• Logged-in site users go through /api/app/data with no rate limit, and accounts with API access can use the unthrottled /api/private/... mirrors. See API-Reference.

Translation overrides

Approved translator edits are written as sparse override files under LOCALES_DIR (default /opt/hailsdotgo/locales/), one {lang}.json per language, holding only the changed keys. Before any change, the previous state is saved to LOCALES_DIR/backup/{lang}_{timestamp}.json automatically.

• Revert a bad edit: copy the desired backup over the override file and restart the service.
• Revert everything for a language: delete LOCALES_DIR/{lang}.json and restart; the app falls back to the translations embedded in the binary.
• Overrides always win over embedded strings, so after merging a translation sync PR and deploying, a stale override file keeps shadowing the repo version of that key until you remove it. See Translator-Workspace.

Cache directory

CACHE_DIR (default /opt/hailsdotgo/cache/) holds fetched game data JSON and scraped event pages. It is entirely disposable: delete it and restart, and the app reloads from upstream (or serves from the embedded snapshots until upstream answers). Deleting it is a reasonable first move if game data ever looks corrupted.

Backup checklist

Back up:

• The MySQL database (all accounts and community data), e.g. nightly mysqldump; see Database-Guide
• LOCALES_DIR (translation overrides plus their backup/ history)
• Your local .env (it is the source for the server's app.env)

Skip:

• CACHE_DIR (regenerates)
• The binary, templates, static/ (rebuilt and re-uploaded by every deploy)
• deploy-manifest.json (deleting it just makes the next deploy upload everything)