Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Provide rsync access to the package repo #4020
referenced this issue
Jul 2, 2016
referenced this issue
Jul 16, 2016
Steps to enable on our server:
Re. chroot, no - we should do that. But for the uid/gid, we're creating some files that are not world-readable, and this was the easy fix because I don't have time right now to fix the root problem.
added a commit
Jul 17, 2016
The inconsistent permissions in our
To fix this, we would need to normalise the ownership of all files when we copy them into packages, which could potentially get tricky, because some packages include executable scripts.
Not sure of the best fix, to be honest.
It looks like the modification timestamp of all the files served by rsync is also updated after melpa rebuilds packages
then when I sync with melpa via rsync every day, rsync re-downloads (though I guess rsync only downloads the difference but it still needs time & network resource to compute the difference firstly) all the files since rsync thinks two files are different if they have different modification timestamp.
One possible solution is use the
Yes, that's by design: every time we build packages we overwrite them because if we change our build process, we want the packages to be updated. It's not a perfect system, but that's how it is.
As you say,
Keeping the modification timestamp of the identical files can save lots of resource and time in rsync. to achieve this, melpa can use a separate directory for rsync, for example,
for file in /path/to/melpa/package/*; do dest=/path/to/melpa/rsync/"$(basename $file)" if ! [ -f "$dest" ] || ! cmp "$file" "$dest"; then cp -v "$file" "$dest" chmod 644 "$dest" fi done
During rsync, the content of most files is unchanged. Figuring out which part is modified itself is not a trivial task and also can be avoided for unchanged files. If we sustain the modification timestamp for the identical files, rsync will simply skip these file.
@cpitclaudel I don't have evidence actually, and after doing some testing, it looks you are right. That's my guess of the cause of the following issue.
The command we are using to sync melpa is:
(only rsync can change the content of
and rsync does hang here for a while (though maybe less than