Low Bandwidth Networks

Low Bandwidth Networks

If your xyOps conductor has a slow connection to the public internet, satellite installs and upgrades may take longer than the default timeouts allow.

This most often shows up during xySat upgrades. xySat asks the xyOps conductor for the latest satellite package, and if the conductor does not already have that package cached locally, xyOps has to download it from GitHub first. During that first uncached request, xySat is still waiting for the conductor to respond.

On a fast connection this usually feels instant. On a slow connection, the conductor may spend a long time downloading the tarball from GitHub before it can start sending anything back to xySat. This can make the upgrade fail or require multiple attempts, especially when the package is large and the public internet link is slow.

There are two good ways to handle this:

• Store xySat packages locally in a xyOps bucket, so installs and upgrades never need to download from GitHub.
• Increase the upgrade and web server timeouts, so the first uncached download has enough time to finish.

Option 1: Store xySat Packages In A Bucket

This is the most reliable option if your xyOps environment has very slow, unreliable, filtered, or air-gapped internet access.

Instead of having xyOps download xySat packages from GitHub on demand, you can download the package yourself, upload it into a xyOps storage bucket, and configure xyOps to use that bucket for satellite installs and upgrades.

When this is set up, xyOps serves the xySat package directly from local storage. Your worker servers download the package from your own xyOps conductor, not from the public internet.

How To Set It Up

1. Download the xySat binary tarball for your worker server's OS and CPU architecture.
2. In xyOps, create a storage bucket.
3. Upload the xySat tarball into that bucket.
4. Set the satellite.bucket property in your conductor configuration to the bucket ID.
5. Install or upgrade your satellite servers normally.

The package filename should follow this format:

satellite-OS-ARCH.tar.gz

For example:

satellite-linux-x64.tar.gz

See the Air-Gapped Satellite Installs section of the hosting guide for the full setup flow.

Tradeoff

The downside is that you need to manually download and upload each new xySat release you want to deploy.

If you prefer controlled upgrades and predictable local installs, this is a good tradeoff. If you prefer fully automatic release fetching, use Option 2 instead.

Option 2: Increase The Upgrade Timeouts

If your conductor can still reach GitHub, but downloads are slow, you can increase two timeout values:

• satellite.config.upgrade_timeout_sec
• WebServer.timeout

For many low-bandwidth networks, 300 seconds is a reasonable starting point. That gives the first uncached upgrade request up to 5 minutes to complete.

Important

The upgrade_timeout_sec setting requires xySat v1.0.31 or later.

If you are trying to upgrade older satellites and the old 60-second limit is what blocks the upgrade, use Option 1 for the first upgrade to v1.0.31 or later. After that, the longer timeout setting can help with future upgrades.

Why Two Settings Matter

The xySat self-upgrade flow has two timeout layers:

1. xySat waits for the upgrade process to complete.
2. The xyOps web server keeps the HTTP request open while xyOps fetches or serves the satellite package.

The first setting, satellite.config.upgrade_timeout_sec, tells xySat how long to wait before it reports that the upgrade did not complete.

The second setting, WebServer.timeout, controls the xyOps web server idle socket timeout. This matters because the first request for a new xySat version may be delayed while xyOps downloads the tarball from GitHub and caches it locally.

In other words, the request looks local from xySat's point of view, but the conductor may be doing a slow upstream fetch in the middle:

xySat -> xyOps conductor -> GitHub

Once xyOps has cached the package, later upgrade attempts should be much faster. The timeout problem is usually worst on the first uncached download.

Example Config

Edit your conductor's config.json and increase both values:

{
	"satellite": {
		"config": {
			"upgrade_timeout_sec": 300
		}
	},
	"WebServer": {
		"timeout": 300
	}
}

If you are using environment variables, the equivalent values are:

XYOPS_satellite__config__upgrade_timeout_sec=300
XYOPS_WebServer__timeout=300

Then restart xyOps so the web server timeout is applied. Connected xySat servers will receive the updated satellite configuration from the conductor.

See these configuration references for more details:

• Satellite Configuration
• satellite.config
• WebServer.timeout

Which Option Should You Choose?

Use the local bucket option when:

• Your conductor has no public internet access.
• Your public internet connection is unreliable.
• You want satellite installs and upgrades to be fully local.
• You are already managing release artifacts manually.

Use the longer timeout option when:

• Your conductor can reach GitHub, but downloads are slow.
• You want xyOps to continue fetching new xySat releases automatically.
• The first upgrade attempt tends to fail, but later attempts usually succeed.

You can also combine both approaches. For example, you can store the packages you care about in a local bucket, and still use longer timeouts as a safety margin for slow internal links.

Troubleshooting

If upgrades still fail after increasing the timeouts, check the xyOps conductor logs for messages about satellite package downloads. These are the most useful clues:

• Cached version has expired, will fetch from upstream
• Fetching satellite core from upstream
• Any network, socket, or timeout error during the GitHub download

Also confirm that:

• The worker server is running xySat v1.0.31 or later.
• satellite.config.upgrade_timeout_sec is high enough for your slowest expected upgrade.
• WebServer.timeout is at least as high as the upgrade timeout.
• The conductor was restarted after changing WebServer.timeout.
• If using a bucket, the uploaded filename matches the expected satellite-OS-ARCH.tar.gz format.

For very slow links, try 600 seconds temporarily. If that works reliably, you can decide whether to keep the larger timeout or switch to the local bucket approach.