Skip to content

v0.0.19 — Pull from the whole fleet

Choose a tag to compare

@Urdemonlord Urdemonlord released this 18 Nov 08:41
· 16 commits to master since this release

v0.0.19 — Pull from the whole fleet

The other half of v0.0.18. send could already fan out; pull could only ask one
machine at a time.

zapsend pull --to gpu-box,edge-1,edge-2 --dir ./collected --json
zapsend pull --to all --dir ./collected          # every host in the hosts file
{
  "ok": true, "command": "pull", "count": 3, "failed": 0,
  "files": 7, "bytes": 4194304, "transferred": 4194304,
  "peers": [
    {"name": "gpu-box", "ok": true, "dir": "./collected/gpu-box", "count": 4, ...},
    {"name": "edge-1",  "ok": true, "dir": "./collected/edge-1",  "count": 2, ...},
    {"name": "edge-2",  "ok": true, "dir": "./collected/edge-2",  "count": 1, ...}
  ]
}

Each peer gets its own directory

--dir is a parent: every peer writes into --dir/<name>/. Two machines that
both have a report.csv must not overwrite each other, and a fleet pull that
silently drops one of them is data loss. --file still filters by name or id
inside each peer.

What it does when a peer cannot serve

--to all is the case that matters, because it sweeps in machines that are
trusted but not sharing anything:

→ a (10.0.2.2)
✗ a: "zapsend" does not offer the Download API — its receiver needs `--share DIR`
→ d (127.0.0.1)
✓ from-linux.txt (18 B)
{"ok":false,"failed":3,"files":2,...}
{"error":"3 of 5 peer(s) failed","ok":false}

Exit 1, the peers that could serve delivered their files, and each failure
carries its own reason. A name that was never trusted is reported as such and
does not stop the rest.

Shape of the JSON

  • One target — unchanged: {"command":"pull","peer":{...},"files":[...], "count":<files>,"bytes":...,"transferred":...}. Nothing that already parses
    pull --json breaks.
  • A list — adds peers[], and count/failed count peers while files
    is a number. Per-peer file arrays are in peers[].files. Exit 0 only when
    every peer delivered every file it had.

Verified

Windows build on WinPE 10.0.26100.1742 pulling from a Linux peer running the same
version, two names into two directories:

pulling 1 file(s) from "zapsend" into C:\pulled\w1
pulling 1 file(s) from "zapsend" into C:\pulled\w2
{"ok":true,"count":2,"failed":0,"files":2,"bytes":36,"transferred":36,...}
exit=0
C:\pulled\w1\from-linux.txt
C:\pulled\w2\from-linux.txt
from-linux-on-019

Linux suite: 43 checks for fan-out (send list, pull list, all, dead peer,
untrusted name, both back-compat JSON shapes), plus 15 unit, 31 compat, 28 json,
25 pull-resume, 29 hosts, 19 pull, 17 resume.

Next

watch DIR --to a,b (push what appears) and sync DIR --to host (send only what
changed) are the two that make this useful without a human typing commands.