Monolithic Game Download Cache Docker Container
This docker container provides a caching proxy server for game download content. For any network with more than one PC gamer in connected this will drastically reduce internet bandwidth consumption.
The primary use case is gaming events, such as LAN parties, which need to be able to cope with hundreds or thousands of computers receiving an unannounced patch - without spending a fortune on internet connectivity. Other uses include smaller networks, such as Internet Cafes and home networks, where the new games are regularly installed on multiple computers; or multiple independent operating systems on the same computer.
This container is designed to support any game that uses HTTP and also supports HTTP range requests (used by Origin). This should make it suitable for:
- Steam (Valve)
- Origin (EA Games)
- Riot Games (League of Legends)
- Battle.net (Hearthstone, Starcraft 2, Overwatch)
- Frontier Launchpad (Elite Dangerous, Planet Coaster)
- Uplay (Ubisoft)
- Windows Updates
This is the best container to use for all game caching and should be used for Steam in preference to the lancachenet/steamcache and lancachenet/generic containers.
You need to be able to redirect HTTP traffic to this container. The easiest way to do this is to replace the DNS entries for the various game services with your cache server.
For the cache files to persist you will need to mount a directory on the host machine into the container. You can do this using
-v <path on host>:/data/cache. You can do the same with a logs directory as well if you want logs to be persistent as well.
Run the container using the following to allow TCP port 80 (HTTP) and to mount
/cache/data directory into the container.
docker run \ --restart unless-stopped \ --name lancache \ -v /cache/data:/data/cache \ -v /cache/logs:/data/logs \ -p 192.168.1.10:80:80 \ lancachenet/monolithic:latest
Unlike lancachenet/generic this service will cache all cdn services defined in the uklans cache-domains repo so multiple instances are not required.
Simple Full Stack startup
To initialise a full caching setup with dns and sni proxy you can use the following script as a starting point:
export HOST_IP=`hostname -I | cut -d' ' -f1` docker run --restart unless-stopped --name lancache-dns --detach -p 53:53/udp -e USE_GENERIC_CACHE=true -e LANCACHE_IP=$HOST_IP lancachenet/lancache-dns:latest docker run --restart unless-stopped --name lancache --detach -v /cache/data:/data/cache -v /cache/logs:/data/logs -p 80:80 lancachenet/monolithic:latest docker run --restart unless-stopped --name sniproxy --detach -p 443:443 lancachenet/sniproxy:latest echo Please configure your router/dhcp server to serve dns as $HOST_IP
Please check that
hostname -I returns the correct IP before running this snippet
Changing from lancachenet/steamcache and lancachenet/generic
This new container is designed to replace an array of steamcache or generic containers with a single monolithic instance. However if you currently run a steamcache or generic setup then there a few things to note.
- Your existing cache files are NOT compatible with lancachenet/monolithic, unfortunately your cache will need repriming
- You do not need multiple containers, a single monolithic container will cache ALL cdns without collision
- lancachenet/monolithic should be compatible with your existing container's env vars so you can use the same run command you currently use, just change to lancachenet/monolithic
Origin and SSL
Some publishers, including Origin, use the same hostnames we're replacing for HTTPS content as well as HTTP content. We can't cache HTTPS traffic, so if you're intercepting DNS, you will need to run an SNI Proxy container on port 443 to forward on any HTTPS traffic.
docker run \ --restart unless-stopped \ --name sniproxy \ -p 443:443 \ lancachenet/sniproxy:latest
Please read the lancachenet/sniproxy project for more information.
You can find a list of domains you will want to use for each service over on uklans/cache-domains. The aim is for this to be a definitive list of all domains you might want to cache.
Regular commodity hardware (a single 2TB WD Black on an HP Microserver) can achieve peak throughputs of 30MB/s+ using this setup (depending on the specific content being served).
Changing Upstream DNS
If you need to change the upstream DNS server the cache uses, these are defined by the
UPSTREAM_DNS environment variable. The defaults are Google DNS (184.108.40.206 and 220.127.116.11).
UPSTREAM_DNS 18.104.22.168 22.214.171.124
You can override these using the
-e argument to docker run and specifying your upstream DNS servers. Multiple upstream dns servers are allowed, separated by whitespace.
-e UPSTREAM_DNS="126.96.36.199 188.8.131.52"
Tweaking Cache sizes
Two environment variables are available to manage both the memory and disk cache for a particular container, and are set to the following defaults.
CACHE_MEM_SIZE 500m CACHE_DISK_SIZE 1000000m
In addition, there is an environment variable to control the max cache age.
You can override these at run time by adding the following to your docker run command. They accept the standard nginx notation for sizes (k/m/g/t) and durations (m/h/d).
-e CACHE_MEM_SIZE=4000m -e CACHE_DISK_SIZE=1000g -e CACHE_MAX_AGE 3560d
Tuning your cache
Steam in particular has some inherrent limitations caused by the adherence to the HTTP spec connection pool. As such steam's download speed is highly dependent on the latency between your server and the steam cdn servers. In the event you find your initial download speed with the default settings is slow this can be resolved by allocating more IP's to your cache. We suggest adding one IP at a time to see how much gain can be had (4 seems to work for a number of people).
Step 1: Adding IP's to your docker host
Consult your OS documentation in order to add additional IP addresses onto your docker cache host machine
Step 2: Adding IP's to your cache container
In order for this to work you need to add the port maps onto the relevant cdn container (for example steam).
- If you are using
-p 80:80should be sufficient as per the documentation.
- If you are using
lancachenet/steamcachethen add multiple
-p <IPadddress>:80:80for each IP you have added. For example
-p 10.10.1.30:80:80 -p 10.10.1.31:80:80
Step 3: Informing lancache-dns of the extra IP's
Finally we need to inform lancache-dns that STEAM is now available on multiple IP addresses. This can be done on the command line using the following command
-e STEAMCACHE_IP="10.10.1.30 10.10.1.31". Note the quotes surrounding the multiple IP addresses.
Step 4: Testing
Choose a game which has not been seen by the cache before (or clear your
/data/cache folder) and start downloading it. Check to see what the maximum speed seen by your steam client is. If necessary repeat steps 1-3 with additional IPs until you see a download equivalent to your uncached steam client or no longer see an improvement vs the previous IP allocation.
Access logs are written to /data/logs. If you don't particularly care about keeping them, you don't need to mount an external volume into the container.
You can tail them using:
docker exec -it lancache tail -f /data/logs/access.log
If you have mounted the volume externally then you can tail it on the host instead.
Advice to Publishers
If you're a games publisher and you'd like LAN parties, gaming centers and other places to be able to easily cache your game updates, we reccomend the following:
- If your content downloads are on HTTPS, you can do what Riot have done - try and resolve a specific hostname. If it resolves to a RFC1918 private address, switch your downloads to use HTTP instead.
- Try to use hostnames specific for your HTTP download traffic.
- Tell us the hostnames that you're using for your game traffic. We're maintaining a list at uklans/cache-domains and we'll accept pull requests!
- Have your client verify the files and ensure the file they've downloaded matches the file they should have downloaded. This cache server acts as a man-in-the-middle so it would be good to ensure the files are correct.
If you need any further advice, please contact us and we'll be glad to help!
Frequently Asked Questions
If you have any questions, please check our FAQs. If this doesn't answer your question, please raise an issue in GitHub.
How to help out
To build just run
docker build --tag lancachenet/monolithic:testing .. To test you can run
If you want to test a new build with a forked uklans repo/branch you can specify
-e CACHE_DOMAIN_REPO="" and even
-e NOFETCH=true if using a locally bind mounted git repo.
- Based on original configs from ansible-lanparty.
- Everyone on /r/lanparty who has provided feedback and helped people with this.
- UK LAN Techs for all the support.
The MIT License (MIT)
Copyright (c) 2019 Jessica Smith, Robin Lewis, Brian Wojtczak, Jason Rivers, James Kinsman
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.