Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix module loads #5599

Merged
merged 5 commits into from
Oct 5, 2017
Merged

Fix module loads #5599

merged 5 commits into from
Oct 5, 2017

Conversation

becker33
Copy link
Member

@becker33 becker33 commented Oct 4, 2017

Fixes #5455

All methods within setup_package use an EnvironmentModifications object to control the environment. Those modifications are applied at the end of setup_package. Module loads for the build environment need to be done after the rest of the environment modifications are applied, as otherwise Spack will unset variables set by those modules.

This is a quick fix, but I don't see a more elegant solution to this problem.

@@ -511,6 +502,19 @@ def setup_package(pkg, dirty):
validate(spack_env, tty.warn)
spack_env.apply_modifications()

# All module loads that otherwise would belong in previous functions
# have to occur after previous line. They are now here
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO this should explicitly refer to the call to spack_env.apply_modifications and it may be work adding to the explanation like:

spack_env includes modifications like unsetting LD_LIBRARY path, which required modules may update. Modules are therefore loaded after the call to spack_env.apply_modifications.

@becker33
Copy link
Member Author

becker33 commented Oct 5, 2017

@scheibelp I've updated the comment as you asked

@scheibelp scheibelp merged commit 328ab32 into develop Oct 5, 2017
@junghans junghans deleted the fix-module-loads branch October 5, 2017 03:47
@pramodskumbhar
Copy link
Contributor

@becker33 : I have tested this on our x86 cluster and it's working fine. BUT, with this change, our Jenkins build is failing for TAU profiler build with error like:

     363   gcc -c -fPIC  -O2 -Wl,-rpath,/jenkins/01/workspace/hpc.spack.github/platform/cscsviz/opt/spack/linux-rhel6-x86_64/intel-17.0.0/tau-2.26.3-odsr7ym4v3uvfiartkswa7leqgi3quz6/lib -Wl,-rpath,/jenkins/01/workspace/hpc.spack.github/platform/cscsviz/opt/spack/linux-rhel6-x86_64/intel-17.0.0/tau-2.26.3-odsr7ym4v3uvfiartkswa7leqgi3quz6/lib64 -Wl,-rpath,/jenkins/01/workspace/hpc.spack.github/platform/cscsviz/opt/spack/linux-rhel6-x86_64/intel-17.0.0/pdt-3.24-itnh5oukvpbgmrwyilmfnnft4mdb65n5/lib -Wl,-rpath,/gpfs/bbp.cscs.ch/apps/viz/intel/2017/compilers_and_libraries_2017.0.098/linux/mpi/intel64/lib TauLinuxTimers.c
     367   In file included from TauLinuxTimers.c:5:
  >> 368   /usr/include/stdio.h:34:21: error: stddef.h: No such file or directory
     369   In file included from /usr/include/stdio.h:75,
     370                    from TauLinuxTimers.c:5:
  >> 371   /usr/include/libio.h:53:21: error: stdarg.h: No such file or directory
     372   In file included from /usr/include/stdio.h:75,
     373                    from TauLinuxTimers.c:5:
  >> 374   /usr/include/libio.h:332: error: expected specifier-qualifier-list before ‘size_t’
  >> 375   /usr/include/libio.h:364: error: expected declaration specifiers or ‘...’ before ‘size_t’
  >> 376   /usr/include/libio.h:373: error: expected declaration specifiers or ‘...’ before ‘size_t’
  >> 377   /usr/include/libio.h:491: error: expected declaration specifiers or ‘...’ before ‘__gnuc_va_list’
  >> 378   /usr/include/libio.h:493: error: expected declaration specifiers or ‘...’ before ‘__gnuc_va_list’
  >> 379   /usr/include/libio.h:495: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘_IO_sgetn’
     380   In file included from TauLinuxTimers.c:5:
  >> 381   /usr/include/stdio.h:80: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘va_list’
     382   In file included from TauLinuxTimers.c:5:

In above case, I am building tau with intel (or XL on bg-q) compiler but TAU does use GCC compiler to build some tools. Above error happens where those files get compiled with GCC. Its strange that above error is related to not finding standards headers from /usr/include.

Unfortunately I am not able to reproduce this in interactive session on our cluster. Do you have any idea what might be going wrong after this PR?

@scheibelp
Copy link
Member

@pramodskumbhar could you post the full compiler entry? I was going to open a new issue for this but I think that detail would be useful in a better description for the issue.

@pramodskumbhar
Copy link
Contributor

@scheibelp : I was debugging this and it seems like it was our Jenkins CI glitch. Now I am able to build everything fine. I will open an issue if I see this again.

Sorry for noise!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Use of --dirty with externals packages and compilers (always required?)
3 participants