(WIP) Package seeding #554
Conversation
|
Can one of the admins verify this patch? |
|
Any update on this? Any hint appreciated. |
src/share/poudriere/common.sh
Outdated
|
|
||
| # Set up URL prefix | ||
| jail_version=$(jget ${JAILNAME} version) | ||
| # $MARJOR_RELEASE-$SUBRELEASE.RELEASE-px --> $MARJOR_RELEASE |
| [ $# -ne 3 ] && eargs pkg_seeding_download destination url originspec | ||
|
|
||
| msg_debug "Fetching seed from: $1" | ||
| fetch -q -a -w 1 -m -o "$2" "$1" &> /dev/null || \ |
There was a problem hiding this comment.
this could do with a comment; what value does the 2nd retry with different parameters bring here? does it solve an actual problem, and in the event of a failure can we rely on the retrieved package being correct? Should we check anywhere for validity, whether by SHA* or other method?
There was a problem hiding this comment.
This kind of fetch retry is actually an imitation of the fetch_file function defined in the same file. Its purpose is to recover from potential jitterish network that fails the first fetch. However, we cannot verify the validity of fetched package without checking its SHA against pkg.freebsd.org's digests.txz. As this will add quite a bit complexity, I will backlog it for now and finish the most basic functionality first. 😸
|
super idea - this is likely to help a lot of people move to poudriere for general port building. I have one general question, if I read the code right, there is no way to detect if Is there a mechanism elsewhere in poudriere (the code to detect changed ports/options) that can be borrowed to cater for this? |
Sometimes it is not necessary to build ports that has been built elsewhere with default options. Just pull them over and use them if they are just dependencies of the ports we really want to build ourselves. In particular, we can opportunistically pull binary packages from pkg.freebsd.org if they have the exact version we want. Fix freebsd#319
|
It's a good start. It really needs to handle options, arch, versions, rather than blindly fetching and then possibly just deleting everything it just fetched. I'll bring it in and add the extra logic to handle that. |
|
This is probably going to be implemented differently. See #797. |
|
Thank you for this. It's been superseded by #797 which uses |
Right after the first
clean_build_queuecall:${MASTERMNT}/.p/all_pkgs, if it's not marked "listed" or if it's not in${PACKAGES}/All, try fetch its${pkg}-${ver}.txzfrom pkg.freebsd.org.If it succeeds, place the .txz into
${PACKAGES}/All.clean_build_queueto remove fetched packages from build queue.It is still not clear to me whether to place fetched packages in
${PACKAGES}/Allor in${MASTERMNT}/packages/All. It seems that because${MASTERMNT}/packages/Allis in jail, it is read-only, but that is whereclean_build_queuechecks. Need help on this part.Note: the code is still WIP and not working properly ...
Fix #319