Configuration

SJTUG mirrors are (mostly) configured with a single config config.<site>.yaml.

This configuration is used directly by lug. At the same time, caddy-gen will generate a Caddyfile from this config.

Entry in Configuration

An entry in configuration describes how a repo is synchronized and how it is served through caddy.

  - type: shell_script
    script: /worker-script/rsync.sh
    source: rsync://mirrors.ocf.berkeley.edu/gnu/
    interval: 43200
    path: /mnt/gnu
    name: gnu
    <<: *oneshot_common

Synchronization Configuration

lug will export all properties to environment variable. For example, worker scripts can read LUG_path from its environment.

For different types of repos, different configuration fields are used. Generally, you may use boilerplates from existing configurations.

• shell script

  - type: shell_script
    script: /worker-script/rsync.sh
    source: rsync://mirrors.kernel.org/centos/
    interval: 5800
    path: /srv/disk1/centos
    <<: *oneshot_common

• external

  - type: external
    name: docker-registry
    proxy_to: https://docker.siyuan.internal.sjtug.org/
    subdomain: docker

• repo is served but not displayed in repo list (see below for serving config)

  - type: external
    name: manjarostable
    target: https://mirrors.sjtug.sjtu.edu.cn/manjaro/stable/
    disabled: true

• git

  - type: shell_script
    script: /worker-script/git.sh
    interval: 2000
    name: git/doom-emacs.git
    source: https://github.com/hlissner/doom-emacs.git
    path: /srv/disk2/git/doom-emacs.git
    serve_mode: git
    target: https://git.sjtu.edu.cn/sjtug/doom-emacs.git

• mirror-clone with mirror-intel

  - type: shell_script
    script: /app/mirror-clone --concurrent_resolve 64 --workers 8 crates_io --target http://mirror-intel:8000/crates.io/crates
    serve_mode: mirror_intel
    interval: 7200
    name: crates.io
    <<: *oneshot_common

Serving Configuration

Serving configuration controls how a user will use a repo.

There are current 6 types of serving.

• default, where users can browse and download files in a local disk.
• redir, users will be redirected to {target}$uri.
• redir_force, users will be redirected to {target} with uri stripped.
• git, git-backend will be used to serve request.
• proxy, reverse proxy
• mirror-intel, served by mirror-intel or s3

caddy-gen is responsible for generating Caddyfile from serving configuration.

There are three highlights in caddy-gen implementation:

• As our mirror service is part of the mirrorz protocol, we made mirrorz folder available on CORS.
• For redir repos with git/ prefix, a special case is handled for libgit2. If a user accesses such repo with libgit2, it will be served from disk as libgit2 doesn't support 301/302 redirection. Otherwise, users are redirected to git.sjtu.edu.cn, which yields best performance and low latency.
• caddy-gen supports unified mirror configuration. If a repo is available on mirror.sjtu.edu.cn/xxx, and user accesses it at mirrors.sjtug.sjtu.edu.cn/xxx, we will redirect them to the corresponding server.

You may refer to current configuration to see how to configure this.