tailscale repos provide rsync method

[81pla]

Dose tailscale repos provide rsync method ?
In china ,access repo speed too slow,i want to mirror repo

[bradfitz]

We don't provide rsync currently.

Do you want to mirror the repo for yourself or for others?

[bradfitz]

Our archive of old binaries is currently 34 GiB.

I assume downloading 34 GiB over rsync will take longer than 24 MiB over HTTPS, but I don't understand China's network.

[81pla]

I want to mirror the repo for myself currently.

Under some china‘s isp network, access the repo at first it was fast, then it was slow at 40-50kb/s.
There are unpleasant experiences with each update new version.

I tired to request china's famous open source software mirror website (https://mirrors.tuna.tsinghua.edu.cn/) to mirror the tailscale repo, but they refused my requst because of tailscale provides business services.
So i want to mirror repo by myself, It is easy for me to use rsync method.

but over https, i access url like this (https://pkgs.tailscale.com/stable/centos/8/x86_64/) , notice 404 page not found
not list the directory. i hope like this https://download.docker.com/ docker-ce list the directory. it can be mirror easily.
Do you have any suggestions for me?

[bradfitz]

https://pkgs.tailscale.com/stable/?mode=json lists the latest stable version of everything.

And our server supports HTTP Range requests to download parts of files. So you could automate the mirroring by getting that JSON, seeing if you already have the file, otherwise:

• do a HEAD request on the URL to get its size, like:

$ curl -I https://pkgs.tailscale.com/stable/tailscale_1.10.1_amd64.tgz
HTTP/2 200 
accept-ranges: bytes
content-type: application/x-gtar-compressed
last-modified: Fri, 02 Jul 2021 22:23:57 GMT
content-length: 14396876
date: Sat, 03 Jul 2021 04:09:19 GMT

• then do a bunch of small HTTP range requests (on new TCP connections) to piece the file together, 1MB at a time, or until your ISP gets slow?

• when you have all the parts, rename tailscale_1.10.1_amd64.tgz.partial to tailscale_1.10.1_amd64.tgz so your mirroring doesn't need to do a HEAD request on it later

I'm not sure whether it would make mirrors.tuna.tsinghua.edu.cn change their mind, but some people use the open source control plane server https://github.com/juanfont/headscale in which case Tailscale the company is not involved at all, and all the pieces are open source.

[DentonGentry]

At this point I don't expect to implement rsync for the pkgs repository, and expect it would get throttled like other file transfers do. Retrieving the files 1MByte at a time seems more promising.

[frankli0324]

I would like to bring this up again. I'm maintaining a mirror site, and I intend to mirror this repository for others.
since tailscale is bsd licensed, redistributing the packages with licensing info in the usage of the mirror is fine, I believe?
without rsync:873, this is rather a hard task, since I have to write a script to parse the repo list (a InRelease file for apt, for example) for each distro to retrieve each file.
I also propose to slightly modify tailscale.com/install.sh to accept an option/env to specify a mirror address, similar to what --mirror does in https://get.docker.com

[DentonGentry]

Emailing info@tailscale.com would be a good place to have that conversation. However to set expectations: mirroring implies a great deal of trust-but-verify activity with the organizations providing mirrors which I'm not sure is the right thing for the company to be doing just yet.

[frankli0324]

I'm not sure what trust-but-verify means... but if you are talking about binary integrity...

curl -fsSL https://pkgs.tailscale.com/stable/debian/bullseye.noarmor.gpg | sudo tee /usr/share/keyrings/tailscale-archive-keyring.gpg
echo 'deb [signed-by=/usr/share/keyrings/tailscale-archive-keyring.gpg] https://mirror.site/tailscale/stable/debian bullseye main' | sudo tee /etc/apt/sources.list.d/tailscale.list

this does the trick, package managers could verify the signatures.

I often recommend tailscale to my schoolmates but 17kB/s download speed drives both of us crazy.