Remove use of stage symlinks #12072

tldahlgren · 2019-07-20T01:57:45Z

The goal of this work is to simplify stage directory structures by eliminating use of symbolic ilnks. This means, among other things, that $spack/var/spack/stage will no longer be the core staging directory. Instead, the first accessible config:build_stage path will be used.

TODO:

Resolve non-stage.py test failures
Add new test cases
Address coverage issues
Update docs
Integrate use of becker33's
Resolve mpich test issue
mkdirp change for adjusting permissions associated with the first accessible stage path

tgamblin · 2019-07-23T04:50:33Z

@frankwillmore, @carsonwoods: FYI

Failing tests pass when run together but fail when run with the rest of the tests.

The permissions issue arises in tests that perform lock checks that choke on the auto-used mock_stage's creation of a mock-stage subdirectory.

lib/spack/spack/stage.py

lib/spack/spack/test/cmd/common/arguments.py

lib/spack/spack/test/cmd/debug.py

etc/spack/defaults/config.yaml

lib/spack/llnl/util/filesystem.py

lib/spack/spack/compilers/clang.py

lib/spack/spack/config.py

tgamblin

more changes

lib/spack/spack/patch.py

tgamblin · 2019-08-06T22:58:19Z

lib/spack/spack/stage.py

 _stage_prefix = 'spack-stage-'


+def _adjust_stage_access(path):


reconcile this with @becker33's new mkdirp (which he has not pushed yet 😬-- hey @becker33 can you push that?)

Sounds good. I won't put time into ensuring this is tested since I'll be removing it.

Changed to pass the group and default_perms='parents' to Greg's (becker33) mkdirp.

lib/spack/spack/stage.py

tgamblin

More changes

lib/spack/spack/stage.py

lib/spack/spack/test/cmd/print_shell_vars.py

lib/spack/spack/test/conftest.py

adamjstewart · 2019-08-19T19:35:15Z

lib/spack/spack/stage.py

+        """
+        # Emulate file permissions for tempfile.mkdtemp.
+        if not os.path.exists(self.path):
+            print("TLD: %s does not exist, creating it" % self.path)


It looks like you forgot to remove these print statements. Also, I'm not sure how I feel about the logic here. If I install a package where several of the dependencies are already installed, I see:

TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/py-pyqt5-5.13.0-q3dc5mhfrsqg2xzdlwkfjmmszi6vkj7n does not exist, creating it TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/bzip2-1.0.8-llnc6sqphf7yp4drig6njzzkxctjwfse does not exist, creating it ==> bzip2 is already installed in /Users/Adam/spack/opt/spack/darwin-mojave-x86_64/clang-10.0.1-apple/bzip2-1.0.8-llnc6sqphf7yp4drig6njzzkxctjwfse TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/expat-2.2.5-ivx3l77irr7xnft6v2xduzqier7zlct3 does not exist, creating it ==> expat is already installed in /Users/Adam/spack/opt/spack/darwin-mojave-x86_64/clang-10.0.1-apple/expat-2.2.5-ivx3l77irr7xnft6v2xduzqier7zlct3 TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/ncurses-6.1-nhodl3ckldkmn6nfzgtymzlsn7mm3htq does not exist, creating it ==> ncurses is already installed in /Users/Adam/spack/opt/spack/darwin-mojave-x86_64/clang-10.0.1-apple/ncurses-6.1-nhodl3ckldkmn6nfzgtymzlsn7mm3htq TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/readline-7.0-kn3u3qhqwktlvjpr47ism2uqqgstvvf2 does not exist, creating it ==> readline is already installed in /Users/Adam/spack/opt/spack/darwin-mojave-x86_64/clang-10.0.1-apple/readline-7.0-kn3u3qhqwktlvjpr47ism2uqqgstvvf2 TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/gdbm-1.18.1-pvjwideh43qybd2ifi65gjxq2dd6p4nj does not exist, creating it ==> gdbm is already installed in /Users/Adam/spack/opt/spack/darwin-mojave-x86_64/clang-10.0.1-apple/gdbm-1.18.1-pvjwideh43qybd2ifi65gjxq2dd6p4nj TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/libffi-3.2.1-rmioaofogfk7efvt3cpb3d433sevjnaf does not exist, creating it ==> libffi is already installed in /Users/Adam/spack/opt/spack/darwin-mojave-x86_64/clang-10.0.1-apple/libffi-3.2.1-rmioaofogfk7efvt3cpb3d433sevjnaf TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/zlib-1.2.11-nqnlnxddpa4qfz5cnf4grnct57ds25ks does not exist, creating it ==> zlib is already installed in /Users/Adam/spack/opt/spack/darwin-mojave-x86_64/clang-10.0.1-apple/zlib-1.2.11-nqnlnxddpa4qfz5cnf4grnct57ds25ks TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/openssl-1.1.1b-z6erpibfcvzxo5chnvovu3qmet4dzuuv does not exist, creating it ==> openssl is already installed in /Users/Adam/spack/opt/spack/darwin-mojave-x86_64/clang-10.0.1-apple/openssl-1.1.1b-z6erpibfcvzxo5chnvovu3qmet4dzuuv TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/sqlite-3.28.0-mijtevf42dqzctgyaizz44skbyqwxgol does not exist, creating it ==> sqlite is already installed in /Users/Adam/spack/opt/spack/darwin-mojave-x86_64/clang-10.0.1-apple/sqlite-3.28.0-mijtevf42dqzctgyaizz44skbyqwxgol TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/xz-5.2.4-snwzcouby3x3ztkutupqzmqwbbxh2tur does not exist, creating it ==> xz is already installed in /Users/Adam/spack/opt/spack/darwin-mojave-x86_64/clang-10.0.1-apple/xz-5.2.4-snwzcouby3x3ztkutupqzmqwbbxh2tur TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/python-3.7.4-jqlxzxpwzdhmrirodxanluofnce5o3ab does not exist, creating it ==> python is already installed in /Users/Adam/spack/opt/spack/darwin-mojave-x86_64/clang-10.0.1-apple/python-3.7.4-jqlxzxpwzdhmrirodxanluofnce5o3ab TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/py-sip-4.19.18-mqzm6offewfpn4amozlz2gdcntal4jvq does not exist, creating it ==> py-sip is already installed in /Users/Adam/spack/opt/spack/darwin-mojave-x86_64/clang-10.0.1-apple/py-sip-4.19.18-mqzm6offewfpn4amozlz2gdcntal4jvq TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/double-conversion-2.0.1-mf7korxtnck5wcpydk4bwushpzubp7vy does not exist, creating it ==> double-conversion is already installed in /Users/Adam/spack/opt/spack/darwin-mojave-x86_64/clang-10.0.1-apple/double-conversion-2.0.1-mf7korxtnck5wcpydk4bwushpzubp7vy TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/libpng-1.6.34-h24accpckijen4jyzgawyxyoyvqgeb4i does not exist, creating it ==> libpng is already installed in /Users/Adam/spack/opt/spack/darwin-mojave-x86_64/clang-10.0.1-apple/libpng-1.6.34-h24accpckijen4jyzgawyxyoyvqgeb4i TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/freetype-2.9.1-l4ys7nhdhzpv7bgvzh2frzdofkbg3j4l does not exist, creating it ==> freetype is already installed in /Users/Adam/spack/opt/spack/darwin-mojave-x86_64/clang-10.0.1-apple/freetype-2.9.1-l4ys7nhdhzpv7bgvzh2frzdofkbg3j4l TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/libiconv-1.15-6y7tjlqeuu7qmpv3zokgbdowqs7emk7y does not exist, creating it ==> libiconv is already installed in /Users/Adam/spack/opt/spack/darwin-mojave-x86_64/clang-10.0.1-apple/libiconv-1.15-6y7tjlqeuu7qmpv3zokgbdowqs7emk7y TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/libxml2-2.9.9-owmrtw347mtch4jm4cium25nvsjamqkf does not exist, creating it TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/resource-xmlts-owmrtw347mtch4jm4cium25nvsjamqkf does not exist, creating it ==> libxml2 is already installed in /Users/Adam/spack/opt/spack/darwin-mojave-x86_64/clang-10.0.1-apple/libxml2-2.9.9-owmrtw347mtch4jm4cium25nvsjamqkf TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/tar-1.31-vyjf7fdqrr6ybqblhuujonbcguepjbne does not exist, creating it ==> tar is already installed in /Users/Adam/spack/opt/spack/darwin-mojave-x86_64/clang-10.0.1-apple/tar-1.31-vyjf7fdqrr6ybqblhuujonbcguepjbne TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/gettext-0.19.8.1-n3673flplqiagvozwgyeezzs4u22gvrr does not exist, creating it ==> gettext is already installed in /Users/Adam/spack/opt/spack/darwin-mojave-x86_64/clang-10.0.1-apple/gettext-0.19.8.1-n3673flplqiagvozwgyeezzs4u22gvrr TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/pcre-8.42-mgfx4tewjjhhbggofpwqgbto2474uuuk does not exist, creating it ==> pcre is already installed in /Users/Adam/spack/opt/spack/darwin-mojave-x86_64/clang-10.0.1-apple/pcre-8.42-mgfx4tewjjhhbggofpwqgbto2474uuuk TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/perl-5.26.2-agbs3wmvu2qp3hjc3amdhpo4znwkcwxs does not exist, creating it TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/resource-cpanm-agbs3wmvu2qp3hjc3amdhpo4znwkcwxs does not exist, creating it ==> perl is already installed in /Users/Adam/spack/opt/spack/darwin-mojave-x86_64/clang-10.0.1-apple/perl-5.26.2-agbs3wmvu2qp3hjc3amdhpo4znwkcwxs TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/glib-2.56.3-32dbs5w24drcs5rioahldtsfaafxujxt does not exist, creating it ==> glib is already installed in /Users/Adam/spack/opt/spack/darwin-mojave-x86_64/clang-10.0.1-apple/glib-2.56.3-32dbs5w24drcs5rioahldtsfaafxujxt TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/font-util-1.3.1-jkc3lwybt5drczabq3wsxlhovcjhc6oy does not exist, creating it TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/resource-encodings-jkc3lwybt5drczabq3wsxlhovcjhc6oy does not exist, creating it TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/resource-font-alias-jkc3lwybt5drczabq3wsxlhovcjhc6oy does not exist, creating it TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/resource-font-bh-ttf-jkc3lwybt5drczabq3wsxlhovcjhc6oy does not exist, creating it TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/resource-font-bh-75dpi-jkc3lwybt5drczabq3wsxlhovcjhc6oy does not exist, creating it TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/resource-font-bh-type1-jkc3lwybt5drczabq3wsxlhovcjhc6oy does not exist, creating it TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/resource-font-dec-misc-jkc3lwybt5drczabq3wsxlhovcjhc6oy does not exist, creating it TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/resource-font-jis-misc-jkc3lwybt5drczabq3wsxlhovcjhc6oy does not exist, creating it TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/resource-font-sun-misc-jkc3lwybt5drczabq3wsxlhovcjhc6oy does not exist, creating it TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/resource-font-bh-100dpi-jkc3lwybt5drczabq3wsxlhovcjhc6oy does not exist, creating it TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/resource-font-ibm-type1-jkc3lwybt5drczabq3wsxlhovcjhc6oy does not exist, creating it TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/resource-font-isas-misc-jkc3lwybt5drczabq3wsxlhovcjhc6oy does not exist, creating it TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/resource-font-misc-misc-jkc3lwybt5drczabq3wsxlhovcjhc6oy does not exist, creating it TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/resource-font-mutt-misc-jkc3lwybt5drczabq3wsxlhovcjhc6oy does not exist, creating it TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/resource-font-micro-misc-jkc3lwybt5drczabq3wsxlhovcjhc6oy does not exist, creating it TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/resource-font-adobe-75dpi-jkc3lwybt5drczabq3wsxlhovcjhc6oy does not exist, creating it TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/resource-font-arabic-misc-jkc3lwybt5drczabq3wsxlhovcjhc6oy does not exist, creating it TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/resource-font-cursor-misc-jkc3lwybt5drczabq3wsxlhovcjhc6oy does not exist, creating it TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/resource-font-daewoo-misc-jkc3lwybt5drczabq3wsxlhovcjhc6oy does not exist, creating it TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/resource-font-misc-meltho-jkc3lwybt5drczabq3wsxlhovcjhc6oy does not exist, creating it TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/resource-font-adobe-100dpi-jkc3lwybt5drczabq3wsxlhovcjhc6oy does not exist, creating it TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/resource-font-misc-cyrillic-jkc3lwybt5drczabq3wsxlhovcjhc6oy does not exist, creating it TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/resource-font-misc-ethiopic-jkc3lwybt5drczabq3wsxlhovcjhc6oy does not exist, creating it TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/resource-font-xfree86-type1-jkc3lwybt5drczabq3wsxlhovcjhc6oy does not exist, creating it TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/resource-font-bitstream-75dpi-jkc3lwybt5drczabq3wsxlhovcjhc6oy does not exist, creating it TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/resource-font-bitstream-type1-jkc3lwybt5drczabq3wsxlhovcjhc6oy does not exist, creating it TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/resource-font-cronyx-cyrillic-jkc3lwybt5drczabq3wsxlhovcjhc6oy does not exist, creating it TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/resource-font-schumacher-misc-jkc3lwybt5drczabq3wsxlhovcjhc6oy does not exist, creating it TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/resource-font-screen-cyrillic-jkc3lwybt5drczabq3wsxlhovcjhc6oy does not exist, creating it TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/resource-font-bitstream-100dpi-jkc3lwybt5drczabq3wsxlhovcjhc6oy does not exist, creating it TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/resource-font-bitstream-speedo-jkc3lwybt5drczabq3wsxlhovcjhc6oy does not exist, creating it TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/resource-font-winitzki-cyrillic-jkc3lwybt5drczabq3wsxlhovcjhc6oy does not exist, creating it TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/resource-font-adobe-utopia-75dpi-jkc3lwybt5drczabq3wsxlhovcjhc6oy does not exist, creating it TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/resource-font-adobe-utopia-type1-jkc3lwybt5drczabq3wsxlhovcjhc6oy does not exist, creating it TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/resource-font-adobe-utopia-100dpi-jkc3lwybt5drczabq3wsxlhovcjhc6oy does not exist, creating it TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/resource-font-bh-lucidatypewriter-75dpi-jkc3lwybt5drczabq3wsxlhovcjhc6oy does not exist, creating it TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/resource-font-bh-lucidatypewriter-100dpi-jkc3lwybt5drczabq3wsxlhovcjhc6oy does not exist, creating it ==> font-util is already installed in /Users/Adam/spack/opt/spack/darwin-mojave-x86_64/clang-10.0.1-apple/font-util-1.3.1-jkc3lwybt5drczabq3wsxlhovcjhc6oy TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/fontconfig-2.12.3-ktmcn2j2yuyrgu5fz67zubcsutyrjc44 does not exist, creating it ==> fontconfig is already installed in /Users/Adam/spack/opt/spack/darwin-mojave-x86_64/clang-10.0.1-apple/fontconfig-2.12.3-ktmcn2j2yuyrgu5fz67zubcsutyrjc44 TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/pixman-0.38.0-uyvgo633ewb3bqlnpduyl2dviquhlhxk does not exist, creating it ==> pixman is already installed in /Users/Adam/spack/opt/spack/darwin-mojave-x86_64/clang-10.0.1-apple/pixman-0.38.0-uyvgo633ewb3bqlnpduyl2dviquhlhxk TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/cairo-1.16.0-vgqs3ecslekb2kmcntes7hg6sphap4ga does not exist, creating it ==> cairo is already installed in /Users/Adam/spack/opt/spack/darwin-mojave-x86_64/clang-10.0.1-apple/cairo-1.16.0-vgqs3ecslekb2kmcntes7hg6sphap4ga TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/icu4c-60.1-iuyrtmmlyd6dlqs2rxsxuy44v3tfnaku does not exist, creating it ==> icu4c is already installed in /Users/Adam/spack/opt/spack/darwin-mojave-x86_64/clang-10.0.1-apple/icu4c-60.1-iuyrtmmlyd6dlqs2rxsxuy44v3tfnaku TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/harfbuzz-2.3.1-uee73fo27ad66tjhaejukg6qrz3k5cpr does not exist, creating it ==> harfbuzz is already installed in /Users/Adam/spack/opt/spack/darwin-mojave-x86_64/clang-10.0.1-apple/harfbuzz-2.3.1-uee73fo27ad66tjhaejukg6qrz3k5cpr TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/libjpeg-turbo-2.0.2-hzhdmt6d73ina3jzdzxh6n44kvgmyg6o does not exist, creating it ==> libjpeg-turbo is already installed in /Users/Adam/spack/opt/spack/darwin-mojave-x86_64/clang-10.0.1-apple/libjpeg-turbo-2.0.2-hzhdmt6d73ina3jzdzxh6n44kvgmyg6o TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/libtiff-4.0.10-7skodcsbdqpfjtxmsp26t5hzgpzvfvfd does not exist, creating it ==> libtiff is already installed in /Users/Adam/spack/opt/spack/darwin-mojave-x86_64/clang-10.0.1-apple/libtiff-4.0.10-7skodcsbdqpfjtxmsp26t5hzgpzvfvfd TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/lcms-2.9-faanb5fcylzjueoepefp37tkyt75svay does not exist, creating it ==> lcms is already installed in /Users/Adam/spack/opt/spack/darwin-mojave-x86_64/clang-10.0.1-apple/lcms-2.9-faanb5fcylzjueoepefp37tkyt75svay TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/libmng-2.0.3-lz2tuhauvupyf2aeka2uqrdjvyog2qng does not exist, creating it ==> libmng is already installed in /Users/Adam/spack/opt/spack/darwin-mojave-x86_64/clang-10.0.1-apple/libmng-2.0.3-lz2tuhauvupyf2aeka2uqrdjvyog2qng ==> opengl@2.1 : externally installed in /opt/X11 ==> opengl@2.1 : already registered in DB TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/pcre2-10.31-sbr6r5l4ghlt3rabeawheyrih3hpme2y does not exist, creating it ==> pcre2 is already installed in /Users/Adam/spack/opt/spack/darwin-mojave-x86_64/clang-10.0.1-apple/pcre2-10.31-sbr6r5l4ghlt3rabeawheyrih3hpme2y TLD: /var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/spack-stage/Adam/qt-5.11.3-xuifxuxq6rxho2sswa7vpfqhdmq5epif does not exist, creating it

We should not be creating stages for packages that are already installed.

My apologies. The print statements are removed in #12474 .

Removes messages accidentally introduced in #12072 .

This reverts commit 0ea6e0f.

tldahlgren added 2 commits July 19, 2019 18:45

bugfix: disallow copy_tree to copy to child directory

274657d

Initial pass at removing stage symbolic links

f0468cd

tldahlgren added WIP stage build General build capability labels Jul 20, 2019

tldahlgren added 8 commits July 31, 2019 17:12

Cleanup mock_stage/fix test_mirror_with_url_patches

5878f0b

Resolve test_url_patch errors

c0e218e

Cleanup Stage.managed_by_spack.

92c01ba

Fix portability of test_get_stage_root_tmp.

7dcefbe

Create dest directory AFTER check passes

01168c5

Snapshot with testing issues.

8c22638

Failing tests pass when run together but fail when run with the rest of the tests.

Snapshot with all tests passing for me (Py2.7)

3930ff6

Fix default build_stage; fix perms issue.

08ddd7c

The permissions issue arises in tests that perform lock checks that choke on the auto-used mock_stage's creation of a mock-stage subdirectory.

tldahlgren commented Aug 6, 2019

View reviewed changes

lib/spack/spack/stage.py Outdated Show resolved Hide resolved

tldahlgren commented Aug 6, 2019

View reviewed changes

lib/spack/spack/test/cmd/common/arguments.py Outdated Show resolved Hide resolved

tldahlgren commented Aug 6, 2019

View reviewed changes

lib/spack/spack/test/cmd/debug.py Outdated Show resolved Hide resolved

tgamblin requested changes Aug 6, 2019

View reviewed changes

Cleanup plus addressed Todd's feedback.

5dc3ce6

tgamblin requested changes Aug 7, 2019

View reviewed changes

lib/spack/spack/stage.py Outdated Show resolved Hide resolved

lib/spack/spack/stage.py Show resolved Hide resolved

lib/spack/spack/test/cmd/print_shell_vars.py Outdated Show resolved Hide resolved

lib/spack/spack/test/conftest.py Outdated Show resolved Hide resolved

tldahlgren and others added 5 commits August 6, 2019 17:53

Updated comments and build_stage docs.

2474603

Merge branch 'develop' into features/remove-stage-symlinks

b787ee6

Merge branch 'develop' into features/remove-stage-symlinks

73a166e

Resolve conflicts from web merge and flake8 issues.

aefdabe

Fixed YAML config docs.

3e78668

tgamblin added this to In progress in Spack/ECP Facilities Hackathon (Aug 5-9) Aug 9, 2019

Resolve mpich install test issue

9093179

tldahlgren changed the title ~~WIP: Remove use of stage symlinks~~ Remove use of stage symlinks Aug 12, 2019

Added several tests to increase code coverage.

d1e54fe

tldahlgren removed the WIP label Aug 12, 2019

Leverage mkdirp for fixing first accessible path permissions.

28c7678

tldahlgren requested a review from tgamblin August 14, 2019 00:10

tldahlgren self-assigned this Aug 15, 2019

tldahlgren merged commit 0ea6e0f into spack:develop Aug 19, 2019

adamjstewart reviewed Aug 19, 2019

View reviewed changes

tldahlgren mentioned this pull request Aug 19, 2019

cleanup: remove extraneous prints #12474

Merged

sethrj mentioned this pull request Aug 19, 2019

Remove rogue debug messages #12476

Closed

tldahlgren added a commit that referenced this pull request Aug 19, 2019

cleanup: remove extraneous prints (#12474)

aed204e

Removes messages accidentally introduced in #12072 .

tldahlgren mentioned this pull request Aug 19, 2019

build: staging existing packages #12477

Closed

scheibelp added a commit to scheibelp/spack that referenced this pull request Aug 20, 2019

Revert "features: Remove stage symlinks (spack#12072)"

6dac08a

This reverts commit 0ea6e0f.

scheibelp mentioned this pull request Aug 20, 2019

Revert 12072 #12490

Closed

tldahlgren mentioned this pull request Aug 21, 2019

docs: update build_stage examples #12495

Closed

scheibelp moved this from In progress to Done in Spack/ECP Facilities Hackathon (Aug 5-9) Aug 30, 2019

This was referenced Oct 2, 2019

spack test leaves behind config.yaml file #13009

Closed

spack clean -s doesn't clean stage dirs #13006

Closed

BenWeber42 mentioned this pull request Dec 7, 2020

Spack stages file/directory permissions hardcoded to 700 #20266

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Remove use of stage symlinks #12072

Remove use of stage symlinks #12072

tldahlgren commented Jul 20, 2019 •

edited

tgamblin commented Jul 23, 2019

tgamblin left a comment

tgamblin Aug 6, 2019

tldahlgren Aug 6, 2019

tldahlgren Aug 13, 2019 •

edited

tgamblin left a comment

adamjstewart Aug 19, 2019

tldahlgren Aug 19, 2019

		_stage_prefix = 'spack-stage-'


		def _adjust_stage_access(path):

Remove use of stage symlinks #12072

Remove use of stage symlinks #12072

Conversation

tldahlgren commented Jul 20, 2019 • edited

tgamblin commented Jul 23, 2019

tgamblin left a comment

Choose a reason for hiding this comment

tgamblin Aug 6, 2019

Choose a reason for hiding this comment

tldahlgren Aug 6, 2019

Choose a reason for hiding this comment

tldahlgren Aug 13, 2019 • edited

Choose a reason for hiding this comment

tgamblin left a comment

Choose a reason for hiding this comment

adamjstewart Aug 19, 2019

Choose a reason for hiding this comment

tldahlgren Aug 19, 2019

Choose a reason for hiding this comment

tldahlgren commented Jul 20, 2019 •

edited

tldahlgren Aug 13, 2019 •

edited