tools: add zsync #1905
tools: add zsync #1905
Conversation
Signed-off-by: Paul Spooren <mail@aparcar.org>
Add config option to create zsync index file for created ImageBuilder and SDKs. Signed-off-by: Paul Spooren <mail@aparcar.org>
|
Wouldn't it make more sense to have bsdiff and/or https://github.com/thoughtpolice/minibsdiff ? |
|
I think that's a different approach. Using (mini)bsdiff would require to keep patches of all previous snapshot builds and also keep versions of them, so downloading clients would know where to start. However with zsync it compares the local file with an index (.zsync) provided by the server. |
|
Perhaps do a weekly schedule or something, I'm not sold on using something like zsync. Something like librevault (p2p) would make more sense but that project seems dead... |
chunkeey
added
the
build/scripts/tools
|
@lynxis http://lists.infradead.org/pipermail/openwrt-adm/2019-February/000995.html |
|
I gave it a try and it resulted in bandwidth saving of ~25%. This is somewhat poor but considering that zsync doesn't work with The resulting .zsync file took some ~165KB for the ar71xx/generic ImageBuilder, I think that size is feasible to give it a try. In some future someone (:tm:) could add |
|
I'd also be hesitant use something that's more or less dead upstream. https://svnweb.freebsd.org/base/head/usr.sbin/portsnap/portsnap/portsnap.sh?revision=326276&view=markup First step would be to evaluate how well bspatch (bsdiff) handles xz compressed archives. |
| PKG_RELEASE:=1 | ||
|
|
||
| PKG_SOURCE:=$(PKG_NAME)-$(PKG_VERSION).tar.gz | ||
| PKG_SOURCE_URL:=https://codeload.github.com/ahmedammar/zsync/tar.gz/$(PKG_VERSION)? |
There was a problem hiding this comment.
I would prefer using a git+https:// download with a hash over an external service.
There was a problem hiding this comment.
Sorry I don't understand what you mean, could you please point me to an example where it is done this way?
There was a problem hiding this comment.
I think it means: You should not use codeload.github.com but some other source.
There was a problem hiding this comment.
local tarball basically.
There's no good solution. codeload changes the hashes from time to time (I've only seen it when using submodules) and local tarballs also change hashes in surprising ways. Many of the tarballs currently on the OpenWrt mirrors have wrong hashes. It causes no issues as it falls back to cloning the git repo (not a shallow clone oddly enough).
The latter is smaller as it uses .xz as an archive, usually.
@aparcar Can you respond on this? |
|
Sure let's close this, we got a cdn now anyway and the the space saving is not that great... |
create tar.xz files for ImageBuilder and SDK. This saves bandwidth when daily updating local snapshots.