Skip to content

Proxy Hot Reload

full-bars edited this page Aug 15, 2026 · 4 revisions

Proxy Hot-Reload

The provider can add and remove proxies from a running instance without restarting or dropping existing connections. Changes take effect within 2 seconds of issuing the reload command.


How It Works

The reload system uses a trigger file at ~/.urnetwork/proxy.reload that contains a sequence number. A background goroutine polls that file every 2 seconds. When it detects the sequence number has changed, it diffs the current proxy list against the live set and applies the delta.

urnet-tools proxy refresh
        โ”‚
        โ–ผ
~/.urnetwork/proxy.reload  (seq incremented)
        โ”‚
        โ–ผ  (within 2 seconds)
ProxyReloader watcher goroutine detects change
        โ”‚
        โ–ผ
reload() diffs source vs running set
        โ”‚
        โ”œโ”€ Added proxies   โ†’ goroutines launched with staggered startup
        โ””โ”€ Removed proxies โ†’ goroutines cancelled, connections cleaned up

The reload is serialized by a mutex โ€” two simultaneous reloads cannot race. If a reload is already in progress when a second trigger fires, the second call returns immediately with an error logged.


Self-Healing: The Hourly Reload Reconciler

Since v3.23.0-fix.26.5 (#309), the provider also runs an hourly reload reconciler (runReloadReconciler). It fires a reload trigger once per hour unconditionally, whether or not anything else requested one.

Why it exists: reload() only ever ran on an explicit trigger (add-source, remove-dead, proxy refresh, URL fetch merge, reaper change). A mass-failure event (e.g. a transient backend outage) could leave a batch of still-desired proxies stuck out of the running set with no future event scheduled to bring them back โ€” observed live on a production node where ~3,300 proxies stayed offline for ~22 hours until an unrelated add-source forced a reload, at which point all of them recovered in one cycle.

The reconciler is deliberately cheap when nothing is wrong: if running already matches desired, reload() just logs +0 added, -0 removed and returns. It is unconditional โ€” not gated behind URNETWORK_SELF_HEAL, same tier as the degraded-proxy reaper.


State Reconciliation: No Ghost Entries

Since v3.23.0-fix.26.5 (#305), reload() also prunes proxy.state to the full desired set (config/file + URL cache), not just the running-diff. Previously, a dead/offline proxy whose goroutine had already exited was never in the "running" set, so its stale proxy.state entry was never deleted โ€” accumulating ghost entries that proxy remove-dead re-reported as "removed" on every run, forever. Now those are pruned on the next reload, and proxy remove-dead reports accurate removal counts.


Triggering a Reload

Native (Linux service)

# Edit your proxy file, then:
urnet-tools proxy refresh

Docker

The ~/.urnetwork directory must be mounted as a volume for the trigger file to be reachable from outside the container:

docker run -v urnetwork_data:/root/.urnetwork ...

Then trigger from the host:

docker exec urfix urnet-tools proxy refresh

Or write the trigger file directly from the host if you have access to the named volume mount point.


What Happens During a Reload

  1. The watcher reads the proxy source file (the --proxy_file flag or the default path)
  2. It computes the diff: which proxy addresses are new, which have been removed
  3. Added proxies โ€” new goroutines are launched. They go through the same jittered startup stagger as initial proxies to avoid thundering-herd on the platform
  4. Removed proxies โ€” their context is cancelled, connections drain and close cleanly
  5. Unchanged proxies โ€” untouched, no interruption to live sessions
  6. The updated proxy set is written to proxy.state

Reload output:

[proxy] reloaded: +12 added, -3 removed

What Counts as a Change

Proxies are identified by address (host:port) and credentials. A proxy that appears in the new file with the same address and credentials as a running proxy is treated as unchanged. Only additions and removals are applied.


Edge Cases

Situation Behavior
Proxy file missing or unreadable at reload time Reload is skipped with a warning log; running proxies are unaffected
Reload already in progress Second reload is skipped; logged as reload already in progress
All proxies removed from file Reload is skipped as a safety guard โ€” removing every proxy is treated as an accidental wipe, not an intent to stop all proxies
Provider restarted Reads the proxy source fresh on startup; proxy.state is used as a secondary consistency check

Environment Variables

Variable Purpose
--proxy_file (flag) Path to the proxy list file (default: ~/.urnetwork/proxy.txt). This is a CLI flag, not an environment variable.

Clone this wiki locally