Background Services

Background Services

Some of what HybridCore does can't happen while a visitor waits for a page. Querying your game servers takes seconds; sending an email can fail and need a retry; resizing an avatar is slow. All of it runs in the background instead, which means two processes have to be running on your server.

Without them the site still loads — and quietly stops being current.

On this page: What they do · What breaks without them · Setting them up · Checking they work · After deploying · Troubleshooting

What they do

The scheduler

Runs jobs on a clock — every minute it checks what is due:

Job	How often	What it does
Server queries	Every minute	Asks each game server for its player count, map and ping
Uptime tracking	Every minute	Records whether each server answered, building the uptime history
Bridge command prune	Daily	Deletes delivered and expired bridge commands
Bridge event prune	Daily	Drops telemetry older than 30 days so the table stops growing
Email digest	Weekly	Sends the news and server summary to subscribers

The queue worker

Picks up jobs the moment something creates them:

Job	Created when
Sending email	Someone registers, resets a password, receives a message
Avatar and image processing	A user uploads a picture
Bridge event dispatch	A game server reports telemetry
Extension jobs	Giveaway draws, vote rewards, store deliveries

The scheduler decides when; the worker does the work. They are separate because a job that takes 30 seconds must not delay the minute tick.

What breaks without them

This is the part worth understanding, because nothing announces itself as broken:

No scheduler:

• Player counts freeze at whatever they were — the server browser shows stale data
• Uptime history has a gap for the whole outage
• Bridge tables grow without limit
• No weekly digest goes out

No queue worker:

• No email is ever sent. Registration confirmations, password resets — all queue up and wait. Users see "check your inbox" and nothing arrives.
• Uploaded avatars stay unprocessed
• Bridge telemetry is stored but never handled — events accumulate in the database and no extension ever sees them, so stats and rewards silently stop
• Giveaway draws don't run

The email one catches most people out: everything looks fine until someone tries to reset their password.

Setting them up

HybridCore generates the service files for you, filled in with your real paths and user:

cd /path/to/hybridcore
php artisan hybridcore:systemd

Paste the two units it prints into /etc/systemd/system/, then:

sudo systemctl daemon-reload
sudo systemctl enable --now hybridcore-scheduler hybridcore-worker

enable makes them start on boot; --now starts them immediately.

Run the command as your normal user, not root. It reports the user it sees, and that user goes into the unit file. If the services run as root, everything they write into storage/ becomes root-owned and the web server can no longer write there — which breaks the site in a way that is genuinely hard to trace.

Why systemd and not cron

Older Laravel guides tell you to add schedule:run to a crontab and run the worker under Supervisor. That works, but:

• A crontab entry that fails fails silently — nothing tells you
• It needs two different tools for two similar jobs
• Neither restarts cleanly after a crash or a reboot

The scheduler service runs schedule:work, a foreground process that fires the scheduler every minute itself. systemd restarts it if it dies, starts it at boot, and gives you systemctl status and journalctl for both processes. If you had a crontab entry from an older setup, remove it — otherwise the scheduler runs twice.

Checking they work

Both processes write a heartbeat every minute. Admin → Health shows them:

• Green — heard from within the last minute
• Amber — stale, the process has stopped or is stuck
• Missing — never started

From the shell:

systemctl status hybridcore-scheduler hybridcore-worker
journalctl -u hybridcore-worker -f     # live log

After deploying new code

Restart the worker. A worker holds your code in memory from the moment it started, so it keeps running the old version indefinitely:

sudo systemctl restart hybridcore-worker

The scheduler picks up changes on its own, but restarting both is simpler to remember:

sudo systemctl restart hybridcore-scheduler hybridcore-worker

Forgetting this is the usual reason a fix "didn't work" after a deploy.

Troubleshooting

Service won't start

journalctl -u hybridcore-worker -n 50

Usually the PHP path or the working directory in the unit is wrong. Re-run php artisan hybridcore:systemd and compare.

Health shows a heartbeat but jobs don't run

The worker is running against a different queue connection than the app. Check QUEUE_CONNECTION in .env and restart the worker.

Emails still don't arrive with the worker running

The jobs are being processed but the delivery is failing. Check the failed jobs:

php artisan queue:failed

Jobs run twice

You have both a crontab entry and the scheduler service. Remove the crontab entry.

Worker keeps restarting

Look at the log. A job throwing on every attempt will exhaust --tries and land in failed_jobs; a worker dying repeatedly is usually running out of memory, which the hourly --max-time recycle is there to prevent.

---

See also: Installation · Updating & Maintenance · Game-Server Bridge