Telcon: 2022 01 26

Jump to bottom Edit New page

Peter Scheibel edited this page Jan 26, 2022 · 19 revisions

Attendees

Peter Scheibel
Greg Becker
Massimiliano Culpo
Andrew Elble
Mark Krentel
Richarda Butler
Phil Sakievich
Tammy Dahlgren
Nick Sly
Asher Mancinelli
Todd Gamblin
Brian Van Essen

Agenda

(Andrew) nvhpc installs CUDA, so which CUDA is being used if I install nvhpc with Spack?
- Pre-meeting notes:
  - We also have this problem with mvapich2 (it installs its own hwloc)
  - This came up while discussing use_variants on 1/19/22 (https://github.com/spack/spack/issues/28442)
    - (Peter) I don't think this will be sufficient (or in other words, I think use_variants solve a separate problem)
  - Making it virtual was a suggestion
    - (Peter) I'm concerned about that since I think the idea was to make CUDA virtual, which imposes additional complexity on the CUDA package
  - (Peter) IMO there should be a way for a package to claim it "supplies" additional packages
    - i.e. nvhpc can say it supplies CUDA, and could add CUDA to the DAG
      - It would specify the versions etc. that come with it
- Notes from meeting:
  - (Peter) Is vendoring a problem?
    - (Greg) potential issue with symbol clashes
      - (Greg) this issue is resolved if the vendored library uses symbol name mangling; also only one library can be loaded at a time, so if there are two libraries with the same name but different implementations
  - (Greg) the solution up to now in Spack is to disable the vendoring whenever possible
    - e.g. if a X depends on Y, the configure script of X might have a --system-Y option
  - Similar problem for externals: https://github.com/spack/spack/pull/24894
  - Note that even detecting this can be a problem
    - E.g. a package might silently vendor a dependency if it isn't available on the system
    - (Greg) after building a package, we can look at the libraries in its directories and compare them to libraries of dependencies, if there are duplicate library names in dependencies then we could say we detected.
      - We would also need to look for duplicate symbols
      - There may be false positives there
  - (Andrew) will create an issue on this and the details here will be added
Externals cut off dependencies: https://github.com/spack/spack/issues/9149#issuecomment-1020740273
(Massimiliano) archspec currently assumes that, for a given microarchitecture, the set of instructions emitted by compilers stays the same over time. This might not be the case (see https://github.com/archspec/archspec-json/issues/38). Should we model and record "optional features" in microarchitectures?
- (Todd) archspec is not about the hardware but rather what will run on the hardware
- (Massimiliano) we aren't as interested in modeling optional features on target, but rather defining new targets
(Mark) status of Cray detection on Perlmutter
- (Greg) Will check in with Cray about arch detection on Permutter (which doesn't seem to behave like other Crays)
- Issue: 28315

Possible agenda items

Multiple providers of same virtual package
- Case 1: we want blas from X and lapack from Y
- Case 2: we actually want several instances of the same virtual (e.g. to run different instances of MPI for the same root)
Possibly revisit: https://github.com/spack/spack/discussions/24966
- Concerning improvements to spack develop
Harmen (not sure if I can join): there are packages like libblastrampoline / libmpitrampoline which provide a blas / lapack / mpi interface to link to, and forwards calls to an actual blas / lapack / mpi provider lib. E.g. julia uses it to link to a blas interface for its binary deps, allowing the user to switch blas provider at runtime, avoiding abi issues. The problem is Spack only allows one provider per dag, but these packages both provides the virtual and depends on the virtual. How do we deal with this "composition" type of pattern? I was thinking: maybe we can relax unique provider per dag to unique provider per subgraph connected through

Clone this wiki locally