Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
buildsystem: unset all PKG_* variables before sourcing a new package #2072
This follows on from a discussion on Slack arising out of #2067, which is adding a new package variable
Adding new variables in a package leads to the risk of those variables leaking over into a subsequent package unless they are unset before the new package is sourced. To prevent leakage we currently maintain a list of variables that must be unset in config/path, which is almost always going to be missing one or more variables.
This PR attempts to address this issue by automatically unsetting ALL
I've tested this with clean builds of RPi, RPi2 and Generic, and no problems.
This change does add some overhead - about 10 seconds extra to a Generic build on AMD FX-8350 which isn't excessive in the scheme of things - but does mean we don't have to worry about critical package variables being accidentally leaked and causing unpredictable behaviour.
Potentially all variables set within a package could/should include the
I think for loop in reset_pkg_vars function could be optimized by grepping all package.mk files and extracting PKG_ variables only once? On first grep one flag variable would be set to prevent grepping again.
@vpeter4 grepping all packages sounds far more complicated/error prone, and also slower - there are over 770 packages that would need to be grepped/parsed (at least half of which wouldn't even be needed for any particular build - a Generic build uses only about 360 packages, a create_addon build might only need one package).
Also, this current implementation unsets only those
Well, I don't know what is faster but 10 seconds is a lot. That's why I decided to see how much does it take. But I'm not sure how you even got 10 sec. For me making tar is only few seconds difference (Odroid_C2 project). Which is fine. For Generic it would take few more.
And just for reference
Better having code nicer :)
That is 10 seconds added to a build that might take up to 4 hours.
The 10 seconds I calculated by performing
The problem with parsing package.mk is that you also need to take into consideration those
or that might be indented:
which is why parsing package.mk is inherently more complex.
Some stats, using the latest
I'm using this test script, which can be run in the root of the LE repo:
It sources each package three times as this is a fairly representative test of what happens during a build (build > unpack > install). That said, the script does processes far more packages than any build, eg. Generic processes only 309 packages.
Timings are in seconds, for 557 packages, average of 3 runs (row #1 is the current "traditional" method):
Based on a Generic build with 309 packages, the new method #2 should add about 2.8 (309 * 0.009) seconds to the total build time.
Doesn't really matter to me :)
Oct 7, 2017
My guess is that
We should change the default PKG_VERSION in a separate PR when cleaning up those packages that don't have/need a version.