etc/builds/funtoo/build.conf etc/builds/funtoo/multi-targets/freshen etc/builds/funtoo/multi-targets/full etc/builds/funtoo/multi-targets/quick etc/builds/funtoo/multi-targets/stage1 etc/builds/gentoo/build.conf etc/builds/gentoo/multi-targets/freshen etc/builds/gentoo/multi-targets/full etc/builds/gentoo/multi-targets/quick etc/builds/gentoo/multi-targets/stage1 etc/builds/~funtoo/build.conf etc/builds/~funtoo/multi-targets/freshen etc/builds/~funtoo/multi-targets/full etc/builds/~funtoo/multi-targets/quick etc/builds/~funtoo/multi-targets/stage1 @@ -1,3 +1,24 @@ +Things to add to the parser: + +Add support for conditional definitions being defined based on conditional +definitions. It's possible to have loops (detect them) but it's also possible +to have unrelated conditions that are perfectly fine. Not supporting +conditional definitions based on other conditional definitions forces people to +work-around perfectly fine situations for no apparent reason. + +We have support for $[foo:zap], which will be $[foo] if defined, and if not, +zap the whole line from output. We also have $[foo?] which will be "yes" if +$[foo] is defined, and "no" otherwise. But we don't have a modifier like +$[foo:opt] that will be $[foo] if $[foo] is defined, otherwise "". This seems +like an obvious and useful addition. + + + + + + + + Fix this: DEBUG: ['/var/tmp/metro'] Traceback (most recent call last): TODO @@ -3,26 +3,94 @@ # # In Metro lingo, a "mirror" is a location on disk that holds all the stuff # that Metro needs as well as all the stuff it creates. So this means that this -# file defines the relative paths and filenames of everything that Metro will -# create, and relative paths and filenames of everything that Metro will -# require for its builds. The only thing that we *don't* define here is the -# actual location of our mirror, which is defined in the $[path/mirror] -# variable in /etc/metro/metro.conf. But this file defines how the mirror -# itself is organized. +# file defines the relative paths and filenames of everything (relative to +# path/mirror, defined in /etc/metro.conf) that Metro will create, and relative +# paths and filenames of everything that Metro will require for its builds. [section path/mirror] -# This is the path where we look for snapshots -- +# This is the path where we look for portage tree snapshots, which are tarballs +# that contain the contents of an entire portage tree. In Metro, portage tree +# snapshots may be either a simple tarball of file or a tarball of an active +# git repository. By default, each build has its own snapshot directory, as +# builds can use different portage trees. + snapshot/subpath: $[]/$[target/build]/snapshots [when target/type is binary-image virtual-image] -# These are the base paths where we look for source binary-images and place built binary-images and virtual-images +# If we are building a binary-image or a virtual-image (stage tarball or +# virtual machine template,) then these subpaths specify the directory +# (relative to $[path/mirror]) where our files are kept. Virtual images are +# typically stored in a subdirectory of this path, while stage tarballs are +# typically stored immediately within this directory. + source/subpath: $[]/$[source/build]/$[source/subarch]/$[source/build]-$[source/subarch]-$[source/version] target/subpath: $[]/$[target/build]/$[target/subarch]/$[target/build]-$[target/subarch]-$[target/version] [when target/subarch] -# This is the location and name of the .control directory +# This is the location and name of the .control directory. A typical control +# directory might be located at /home/mirror/linux/~funtoo/pentium4/.control. +# This directory contains special files that control the behavior of a +# subarch's build. Here are the files that are part of the control directory: +# +# Configured and maintained by user: +# +# ./control/strategy/stage1 contains either "local" or "remote" +# +# If ./control/strategy/stage1 is set to "remote", you will also need to create +# the following files: +# +# ./control/remote/build contains remote build that contains the stage1 we'll use +# ./control/remote/subarch contains remote subarch that contains the stage1 we'll use +# +# Initially configured by user, updated automatically by Metro: +# +# ./control/stage1/version version of most recently-built stage1 for this build/subarch +# ./control/stage3/version version of most recently-built stage3 for this build/subarch + control: $[]/$[target/build]/$[target/subarch]/.control +# When building a stage2, we have the option of using a /remote/ (ie., not in +# this build/subarch directory) stage1 for our build. This is typical when we +# are building an optimized set of stages -- if we're building lots of +# optimized stages, it's more efficient to use a single generic stage1 for all +# our builds. When Metro uses a remote stage1, it needs to look in the remote's +# .control directory to figure out the last-built stage1 version to use. This +# variable points Metro to the right place; you should typically not need to +# change this value: + +source/control: $[]/$[source/build]/$[source/subarch]/.control + +[section strategy] + +# Do we want to build a stage1, and utilize this local stage1 for our stage2 +# build? This is the question that is answered by the setting of the +# .control/strategy/stage1 file. +# +# If set to "local", this means we build our own local stage1 for this subarch, +# and we utilize this local stage1 for building our stage2. This is the +# setting you want to use for generic subarches, such as "x86" and "amd64" +# subarches, for example. +# +# If set to "remote", this means that rather than building and utilizing our +# own stage1, we use a /remote/ (ie., not in this build/subarch directory) +# stage1 tarball to generate our stage2. This is the setting you want to use +# for an optimized build directory, ie. an opteron or pentium4 build. In this +# case, it is more efficient to use an x86 or amd64 stage1 rather than generate +# our own. This can save disk space and compile time. +# +# Note that you may opt to generate an optimized "local" stage1 tarball in some +# circumstances. This can be handy if you only care about creating an optimized +# opteron build. In this case, you can set the stage1 strategy to "local" and +# your opteron build will generate its own (optimized) stage1. And you won't +# need to worry about creating a generic amd64 build just to so you can feed +# updated amd64 stage1's to your opteron stage2 build. +# +# Creating an optimized stage1 can also come in handy when working with +# experimental portage trees. This is useful for temporarily avoiding build +# failures that may occur due to changing CHOST settings during the stage2 +# build process. + +stage1: << $[path/mirror/control]/strategy/stage1 etc/fslayouts/funtoo.conf @@ -3,7 +3,7 @@ # Path configuration: [collect ../targets/$[metro/target]/$[target].spec when target] -[collect ./builds/$[metro/build].conf] +[collect ./builds/$[metro/build]/build.conf] # General Paths etc/metro.conf @@ -55,8 +55,8 @@ class collection: return None truekeys=[] for cond in self.conditionals[varname].keys(): - if self.conditionOnConditional(cond): - raise FlexDataError, "Not Allowed: conditional variable %s depends on condition %s which is itself a conditional variable." % ( varname, cond ) + #if self.conditionOnConditional(cond): + # raise FlexDataError, "Not Allowed: conditional variable %s depends on condition %s which is itself a conditional variable." % ( varname, cond ) if self.conditionTrue(cond): truekeys.append(cond) if len(truekeys) > 1: modules/flexdata.py @@ -13,10 +13,11 @@ source: $[:source/subpath]/$[source/name].tar.bz2 : stage3 name: $[]-$[:subarch]-$[:version] -# When building from a stage3, use the last sucessfully built stage3: +# When building from a stage3, we now simply use the stage3 with matching +# build, subarch and version: -version: << $[path/mirror/control]/lastdate -subarch: << $[path/mirror/control]/subarch -build: << $[path/mirror/control]/build +build: $[target/build] +subarch: $[target/subarch] +version: $[target/version] targets/gentoo/stage/stage3-derivative.spec @@ -20,10 +20,13 @@ name/current: stage3-$[target/subarch]-current ok/run: [ #!/bin/bash -# UPDATE lastdate and subarch for our next build: -echo "$[target/version]" > $[path/mirror/control]/lastdate || exit 1 -echo "$[target/subarch]" > $[path/mirror/control]/subarch || exit 2 -echo "$[target/build]" > $[path/mirror/control]/build || exit 2 +# We completed a successful stage3 build, so record the version of this build in our +# .control/stage3/version file so that other builds can see that this new version is +# available. + +install -d $[path/mirror/control]/stage3 || exit 1 +echo "$[target/version]" > $[path/mirror/control]/stage3/version || exit 1 + # CREATE current symlink for the stage3 rm -f $[path/mirror/link] ln -s $[path/mirror/link/dest] $[path/mirror/link] || exit 3 targets/gentoo/stage/stage3-generator.spec @@ -144,6 +144,7 @@ fi # won't work. This is normally okay. rm -rf $ROOT/var/tmp/* $ROOT/tmp/* $ROOT/root/* $ROOT/usr/portage $ROOT/var/log/* || exit 5 +rm -rf $ROOT/car/cache/edb/dep/* rm -f $ROOT/etc/{passwd,group,shadow}- $ROOT/etc/.pwd.lock rm -f $ROOT/etc/portage/bashrc install -d $ROOT/etc/portage targets/gentoo/stage/steps.spec @@ -1,6 +1,26 @@ [collect ./stage/common.spec] [collect ./stage/capture/tar.spec] -[collect ./stage/stage3-derivative.spec] + +# A stage1 is no longer considered a stage3 derivative, because it may use +# a "remote" (ie. not in the current build/subarch directory) stage3 as a seed. +# True stage3-derivatives use a stage3 that has the same build, subarch and +# version as the target. + +[section source] + +: stage3 +name: $[]-$[:subarch]-$[:version] + +# When we are building a stage1, we are always going to use the last-built +# *local* (this build and subarch) stage3. + +subarch: $[target/subarch] +build: $[target/build] +version: << $[path/mirror/control]/stage3/version + +[section path/mirror] + +source: $[:source/subpath]/$[source/name].tar.bz2 [section target] @@ -83,3 +103,17 @@ emerge $eopts --nodeps baselayout || exit 1 emerge $eopts -p -v --noreplace --oneshot ${buildpkgs} || exit 3 emerge $eopts --noreplace --oneshot ${buildpkgs} || exit 1 ] + +[section trigger] + +ok/run: [ +#!/bin/bash + +# Since we've completed a successful stage1 build, we will update our +# .control/stage1/version file. This file records the version of the +# last successful stage1 build. + +install -d $[path/mirror/control]/stage1 || exit 1 +echo "$[target/version]" > $[path/mirror/control]/stage1/version || exit 1 + +] targets/gentoo/stage1.spec @@ -4,9 +4,18 @@ [section source] : stage1 + +[when strategy/stage1 is remote] + +build: << $[path/mirror/control]/remote/build +subarch: << $[path/mirror/control]/remote/subarch +version: << $[path/mirror/source/control]/stage1/version + +[when strategy/stage1 is local] + version: $[target/version] -subarch: $[target/subarch] build: $[target/build] +subarch: $[target/subarch] [section target] @@ -51,8 +60,8 @@ gcc-config $(gcc-config --get-current-profile) for prof in /etc/env.d/05gcc* do - TESTLDPATH=$(unset LDPATH; source $prof; echo $LDPATH) - if [ ! -e $TESTLDPATH ] + TESTPATH=$(unset PATH; source $prof; echo $PATH) + if [ ! -e $TESTPATH ] then echo echo ">>>" targets/gentoo/stage2.spec etc/builds/funtoo.conf etc/builds/gentoo.conf etc/builds/hollow.conf etc/builds/~funtoo.conf 594e0844ee68e707977f939ecb7f19ecf1a9c961 Daniel Robbins drobbins@funtoo.org http://github.com/funtoo/metro/commit/0a498e8ba035df6b5e7cdbbcf12d64f6a27ebabb 0a498e8ba035df6b5e7cdbbcf12d64f6a27ebabb 2009-06-05T23:45:56-07:00 2009-06-05T23:39:48-07:00 Significant incompatible changes to Metro workflow: Please see etc/fslayouts/funtoo.conf for detailed information. Metro now supports building from and utilizing a local (current version/build/subarch) stage1 as well as building from and utilizing a remote (non-matching, ie. generic) stage1. This has resulted in a change of the .control directory format and conventions. Before, Metro would track the last stage3 built and use this to build new stage1's. With this commit, stage1's are optional, so we generally track the stage1 -- which one to use, either our own, or one from another build. These are the new modular multi-targets Required for new .control file functionality. c4a202a13b38f9b5640163ff2dedcf7f7dbf8e43 Daniel Robbins drobbins@funtoo.org