spack edit -f => spack create

[adamjstewart]

In my mind, the following distinction should be made:

• spack create: create a new package
• spack edit: edit an existing package

However, spack edit is doing double duty, because if spack create fails or your software cannot be downloaded, then you need to run spack edit -f to create a new package. This is kind of confusing, but more importantly both lib/spack/spack/cmd/create.py and lib/spack/spack/cmd/edit.py have completely different default templates. The template in edit.py doesn't have license info, doesn't contain FIXME info, isn't PEP8 compliant, and assumes Autotools instead of saying Unknown build system.

[tgamblin]

I agree -- want to submit a PR? I think there should probably be edit -f and create -f, where one is an alias for the other.

[adamjstewart]

I can start working on this. I disagree about the alias though. What reason would there be to keep edit -f around? When you run spack edit and the package doesn't exist, you should run spack create instead. Seems like a logical step.

[tgamblin]

If you type $EDITOR <file> most editors will edit a new, empty file with that name. That was the intuition behind edit -f. Of course, it requires -f, so that is already different from $EDITOR, and most editors will create a buffer, and only create the file once you type something and save it. So the metaphor is kind of strained.

spack edit and spack create take very different arguments (edit takes a package, create takes a URL), so you'll have to handle the fact that when you run spack create -f it wants a package name instead of the usual URL. There is also the backward-compatibility argument, but edit -f is interactive, so I don't think people would actually script it anyway.

I think in the end, your way is more intuitive. How about at least a warning saying that it's been changed to create -f?

[adamjstewart]

Ah, now I see the logic behind the original design. But I think spack create --force or maybe spack create --empty makes more sense, at least to me. $EDITOR <file> creates a new file, but it doesn't create a boilerplate like spack create does. It would be nice to get more opinions on this, particularly from new users.

There is also the backward-compatibility argument, but edit -f is interactive, so I don't think people would actually script it anyway.

spack edit -f needs to be run so infrequently that I'm not super worried about backward-compatibility. And I definitely can't see anyone scripting it.

How about at least a warning saying that it's been changed to create -f?

Deal. I'll put in a deprecation warning for now. If you really want, I can put in a deprecation warning and a temporary alias, but you'll have to show me how to do that.

I'll get started on this as soon as #1112 gets merged.

[tgamblin]

Pinging some users for opinions.

@alalazo @mwilliammyers @trws @robertdfrench @DavidPoliakoff @mamelara @eschnett @citibeth @gartung @glennpj @mathstuf

[glennpj]

There already is a -f flag to spack create so I do not think you can do anything there unless you want the functionality to change.

spack create -h
usage: spack create [-h] [--keep-stage] [-n NAME] [-r REPO] [-N NAMESPACE]
                    [-f]
                    [url]

positional arguments:
  url                   url of package archive

optional arguments:
  -h, --help            show this help message and exit
  --keep-stage          Don't clean up staging area when command completes.
  -n NAME, --name NAME  Override the autodetected name for the created
                        package.
  -r REPO, --repo REPO  Path to a repository where the package should be
                        created.
  -N NAMESPACE, --namespace NAMESPACE
                        Specify a namespace for the package. Must be the
                        namespace of a repository registered with Spack.
  -f, --force           Overwrite any existing package file with the same
                        name.

What about having the spack create command accept either a URL or a non-URL string. If a non-URL string then that is the package name, if a URL then the package name is ascertained from it. If a URL is given then run the command as it is done now. If not, then create a new package file, assuming no name conflicts, and populate the file with appropriate content from a template. Then the -f flag would work as now, forcing an overwrite of an existing package with the same name, whether the name is explicitly given or determined from a URL. I think in the above model the spack edit -f option would fall away.

[trws]

I've never actually used edit -f, but it occurs to me that the one reason I would have used it had I known about it would be to create packages where the link doesn't work or doesn't parse correctly. Can we have a -F or --just-create-the-darn-package-with-this-name flag or something to make the url optional?

Maybe make spack create -n <package> valid, sans url?

[mwilliammyers]

I am a new user and I have already used the -f option a few times, so I think we should keep it, but agree it makes much more sense to put it in create. I think create should be smart enough to handle a package name or URL but either way if you pass an -f it will overwrite any existing package with the new one from the URL or with basic boilerplate if it is a package name. I think edit should fail if a package with that name doesn't exist and there should't be any options to create one, just use create for that.

[adamjstewart]

@mwilliammyers What packages have you had to use spack edit -f for?

[mwilliammyers]

@adamjstewart I messed up some URLs or something when creating new packages and wanted to overwrite the incorrect existing packages. It was confusing to me that they were part of edit, my first reaction was to look for a -f flag in create...

[adamjstewart]

Ah, you were using spack create -f, not spack edit -f. Nvm

[adamjstewart]

So we have one vote for spack create -n working without a URL, and 2 votes for spack create accepting URLs or package names with no flag necessary.

[trws]

I'm fine with it just taking either, as long as the -n option is a
reasonable fallback for when someone decides to name their package
my://package-is_31337& or some such fool thing that will always
parse wrong.

On 27 Jun 2016, at 14:25, Adam J. Stewart wrote:

So we have one vote for spack create -n working without a URL, and 2
votes for spack create accepting URLs or package names with no flag
necessary.

---

You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
https://github.com/LLNL/spack/issues/1108#issuecomment-228880350

[citibeth]

So far I don't use spack create or spack edit. Reasons for not using spack create:

1. It chokes on many URLs, unpredictably
2. Getting an MD5 of the latest version is useful. Getting it for the past 5 versions is not so useful (and often the reason it chokes, leaving me with 0 MD5s).
3. The one-size-fits-all template isn't so useful for me. I prefer to copy from a package that's similar to the one I'm creating --- that usually mean it's based on on Python setup.py, Autotools, CMake, etc.
4. It runs spack edit when it's done --- see below.

Reasons for not using spack edit:

1. I'm an Emacs user, and the emacsclient stuff always seems to be more effort than it's worth to set up and get working across all my different machines, Emacs versions and ways of logging in. One common way I work is using Emacs on my local desktop, editing files on the "big" computer via Tramp/SSH. My command prompt is on the "big" computer. Good luck getting emacsclient working with that...

That said, I do think that some Spack commands could be useful in helping build Spack packages:

1. `spack create [-t template] [-u URL]
   1. The unnamed argument should be <package-name> because that is how other Spack commands work. Having it be a URL is a little unusual.
   2. By adding the -t/--template flag, we can have multiple templates that would be useful for different types of projects. Start with python, autotools, cmake, cmakeclass, python, r. (cmakeclass is for PR [RtM] spack setup, Developer Support for CMake Projects #543). We probably build a directory full of templates.
   3. The optional -u/--url flag will add URL/version stuff to the package. Otherwise, you fill it in yourself.
2. spack addversion <package-name> [-v version] [-u URL]. This would add a new version() line to the package. It would be smart and use some heuristics. For example:
   1. If there aren't yet any url or version lines in the package, it would add them in at an appropriate place. "Appropriate" would be defined by comments in the templates --- or of those comments can no longer be found, just put them in at the beginning and make the user move them to the right place.
   2. If the URL is similar to existing URLs, it would just add a new version. Otherwise, add a url= as part of the version() line.
   3. If users have a way to specify which parts of the URL are a version number, then in many more-complex cases, Spack could write the url_for_version() function.
   4. Other command lines should allow the addition of Git-based versions.
   5. Even if none of these thing are possible, inserting the correct MD5 checksum is the most useful thing Spack can do here. All the rest is icing.

[DavidPoliakoff]

+1 for spack create -f accepting URL or package name

Maybe in a separate PR, but perhaps having a -t (template) and having a set of template packages. I certainly wouldn't mind typing

spack create -f my_package_name -t cmake

and seeing a package with that name and some boilerplate cmake put together. Dragging this back on topic, +1 for create -f accepting URL or package name

[DavidPoliakoff]

Also, while this is Spack-Christmas, I want to be able to store a package.py in the root of my git repo and have spack (something) (my-github-url) do a clone, look at package.py, and magically make the right thing happen through magic

[adamjstewart]

As for the spack create -t template idea, spack create already does this. It peeks inside of the tarball and determines the build system based on its contents:

file found	build system
configure	Autotools
CMakeLists.txt	CMake
SConstruct	SCons
setup.py	Python
NAMESPACE	R

@citibeth If you haven't used spack create in a while, things have been significantly improved. See #1112 for the latest of my additions. Here are some of the default templates:

CMake

# FIXME: Add additional dependencies if required.
depends_on('cmake')

def install(self, spec, prefix):
    with working_dir('spack-build', create=True):
        # FIXME: Modify the cmake line to suit your build system here.
        cmake('..', *std_cmake_args)

        # FIXME: Add logic to build and install here.
        make()
        make('install')

R

# FIXME: Add additional dependencies if required.
extends('R')

def install(self, spec, prefix):
    # FIXME: Add logic to build and install here.
    R('CMD', 'INSTALL', '--library={0}'.format(self.module.r_lib_dir),
      self.stage.source_path)

Python

# FIXME: Add additional dependencies if required.
extends('python')

def install(self, spec, prefix):
    # FIXME: Add logic to build and install here.
    python('setup.py', 'install', '--prefix={0}'.format(prefix))

If there are any other build systems you would like to add, or default installation instructions you would like to modify, please let me know.

[trws]

Quite so, but this is assuming 1) that there is a tarball, and 2) that
it gets it right. The first of these is by far the more frequent
problem, but either way, having some control would be helpful.

On 27 Jun 2016, at 19:10, Adam J. Stewart wrote:

As for the spack create -t template idea, spack create already
does this. It peeks inside of the tarball and determines the build
system based on its contents:

file found	build system
configure	Autotools
CMakeLists.txt	CMake
SConstruct	SCons
setup.py	Python
NAMESPACE	R

@citibeth If you haven't used spack create in a while, things have
been significantly improved. See #1112 for the latest of my additions.
Here are some of the default templates:

CMake

# FIXME: Add additional dependencies if required.
depends_on('cmake')

def install(self, spec, prefix):
    with working_dir('spack-build', create=True):
        # FIXME: Modify the cmake line to suit your build system here.
        cmake('..', *std_cmake_args)

        # FIXME: Add logic to build and install here.
        make()
        make('install')

R

# FIXME: Add additional dependencies if required.
extends('R')

def install(self, spec, prefix):
    # FIXME: Add logic to build and install here.
    R('CMD', 'INSTALL', '--library={0}'.format(self.module.r_lib_dir),
      self.stage.source_path)

Python

# FIXME: Add additional dependencies if required.
extends('python')

def install(self, spec, prefix):
    # FIXME: Add logic to build and install here.
    python('setup.py', 'install', '--prefix={0}'.format(prefix))

If there are any other build systems you would like to add, or default
installation instructions you would like to modify, please let me
know.

---

You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
https://github.com/LLNL/spack/issues/1108#issuecomment-228929443

[adamjstewart]

I guess we could add a -t template flag for use when Spack incorrectly guesses the build system, multiple build systems are possible, or no URL is specified. I'll add that to my list of ideas.

[DavidPoliakoff]

Yeah, less crassly on my second comment, better git interactions (including just doing some of the awesome stuff it does with tarballs) would be great. Otherwise I'll write an extension that downloads a github repository, tars it up, then creates a spack package from that, and Todd will cry

Out for the day, but thanks for your effort and engagement on this!

[adamjstewart]

Out of the packages currently in Spack, 99% have a downloadable URL (1% are licensed). spack create <url> doesn't crash very often (I fixed the crash when using a .zip file), and if it does you should report it and someone will correct it. Therefore, I think the default should still be to accept a URL, but make it optional. spack edit -f isn't used very frequently (some of you didn't even know it existed), so creating a blank template from a name isn't that common. I think specifying the name should be done through the current -n <name> flag. I'll add a -t <template> flag due to popular demand. Any other wishes/bugs specifically relating to spack create?

[trws]

As @DavidPoliakoff mentioned, doing something useful with a repository URL where the protocol can be determined easily would be great.

[citibeth]

If you're using GitHub, you can always get a tarball from it for any Git
revision.

On Mon, Jun 27, 2016 at 10:14 PM, David Poliakoff notifications@github.com
wrote:

Yeah, less crassly on my second comment, better git interactions
(including just doing some of the awesome stuff it does with tarballs)
would be great. Otherwise I'll write an extension that downloads a github
repository, tars it up, then creates a spack package from that, and Todd
will cry

Out for the day, but thanks for your effort and engagement on this!

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/LLNL/spack/issues/1108#issuecomment-228929866, or mute
the thread
https://github.com/notifications/unsubscribe/AB1cd0FQM7P7dJhROon9CUEUUhUJVyWEks5qQIN7gaJpZM4I963P
.

[DavidPoliakoff]

I guess my push on this is that I want a lot more out of Spack+git (ideally with the hub package enabled). I want

"When a spack install fails on a package associated with a git repository, if the repo owner allows it and the spack install allows it, raise an issue" and "spack install [github repo containing a valid package.py, as in Bower]" and for that "spack install github repo" to allow for dependencies on other github repo links. I want "spack request [github URL]" to raise an issue saying that a person wants a spack package to be made for a given URL, hopefully not on the main spack issues list, but somewhere. I think there's a lot of fun stuff to be done with git and github and hub.

I haven't really put in the time/effort on Spack to get too belligerent about how things should work, but think there's a really fun space to explore in "spack [words] [github URL]," the other day I did a "bower install [github URL]," used the resulting extension in my web page, and went from having the idea that a project was cool to seeing it live on my website in under five minutes, that has me pretty excited for package managers being aware of github

Sorry, I can't tell whether the tone of this is aggressive, it's really not meant that way if it's coming off that way, more "whoa look at what's cool in this part of the idea space"

[citibeth]

Pardon my long rant below, I hope it is uesful.

Out of the packages currently in Spack, 99% have a downloadable URL (1%
are licensed). spack create doesn't crash very often (I fixed the
crash when using a .zip file), and if it does you should report it and
someone will correct it. Therefore, I think the default should still be to
accept a URL.

I don't understand the reasoning here. "URL's usually work, therefore this
is the right UI." I don't buy that. I actually think spack reate <url>
is the wrong UI.

Spack commands have a grammar, one that can now be deduced by looking at
all the othe Spack commands. Consider spack find, spack load, spack install, spack uninstall, etc. The grammar here is always spack <verb> <direct object>, where the direct object is a package or spec. People are
intuitive about grammar, and will instinctively deduce these rules from
numerous examples. That's why 4-year-old children say "this table is
mines" and "I goed to the store."

Therefore, the correct form for spack create needs to be spack create <package> -u <url> (or --url). This is conforms to the Spack grammar
set up by other Spack commands, and is therefore easy to remember.

But you say that (in at least some cases), the direct object (package)
isn't needed, because it can be deduced from the indirect object (URL). In
that case, the correct form would be spack create -u <url>. But this is
still a "sentence" with a direct object, albeit one that is implied: spack create is still creating something, after all!

By setting up a consistent grammar like this, other variants become
intuitively obvious, all of them useful in this case:

• spack create <package-name> -u <url>: Create a package with a name
  potentially different from the one Spack would deduce by looking at the URL.
• spack create <package-name>: Create a package but don't bother with
  adding any URLs to it.
• spack create -g <git-url> <git-version/branch>: Create a package, where
  the main version is a git version, not a URL tarball, and the package name
  is the same as the Git repo name.

Rather than arguing about what percentage of users need these variants, it
becomes easy to just implement them. You now have a spack create command
that is more intuitive, more flexible and more functional. It recognizes
that users have many different use cases. Sometimes they need to specify
the package name explicitly. Sometimes they don't have a URL. Sometimes
they need to specify the template. Sometimes they don't have a tarball and
need to use Git. And on and on... a good grammar allows a natural
combination of elements to produce all these effects. Even if you don't
implement all possibilities to day, you've accounted for them already in
the grammar --- and therefore, allowed people to imagine ways the command
could/should be extended in the future.

The added clarity, consistency and flexibility is well worth the extra
three characters (, '-', 'u') that one must type in this grammar for
the "common" case of automagically creating a package from a URL. spack create is a seldom used command; therefore, it's more important that it is
easy to remember, than saving a few keystrokes.

Is this a trivial matter? Maybe. But add it up over (now) dozens of Spack
commands, plus all the other programs with command line args I have to work
with daily, and it is no longer trivial. Lack of consistency has a
negative effect on usability. I hope I've convinced somebody that thinking
globally about UI design is a worthwhile endeavor.

Any other wishes/bugs specifically relating to spack create?

Spack currently has 132 open Issues and 49 open pull requests. This number
has only grown since I started using Spack in February --- indicating that
the project is growing faster than the development team. Some outstanding
PRs are important features we've been waiting for months to see merged.
Other important core features we're only beginning to tackle
(constraints-based concretization algorithm and module-based compilers on
Cray, for example). Developer time is limited, the needs are great, and we
must prioritize.

Automagic package creation is a minor issue: nobody is stuck if it doesn't
work. Most Spack users understand they can create new packages by copying
old packages and changing things to suit --- that is a fundamental practice
in computer programming. A broken/dysfunctional spack create is at worst
a minor inconvenience.

Therefore, after my long rant on grammar, I would suggest that we don't get
stuck on spack create while we have so many larger problems to deal with.

[citibeth]

David,

I replied to you on the general Spack mailing list:

https://groups.google.com/forum/?hl=en#!topic/spack/eD5BDi3j7vA

-- Elizabeth

On Mon, Jun 27, 2016 at 10:59 PM, David Poliakoff notifications@github.com
wrote:

I guess my push on this is that I want a lot more out of Spack+git
(ideally with the hub package enabled). I want

"When a spack install fails on a package associated with a git repository,
if the repo owner allows it and the spack install allows it, raise an
issue" and "spack install [github repo containing a valid package.py, as in
Bower]" and for that "spack install github repo" to allow for dependencies
on other github repo links. I want "spack request [github URL]" to raise an
issue saying that a person wants a spack package to be made for a given
URL, hopefully not on the main spack issues list, but somewhere. I think
there's a lot of fun stuff to be done with git and github and hub.

I haven't really put in the time/effort on Spack to get too belligerent
about how things should work, but think there's a really fun space to
explore in "spack [words] [github URL]," the other day I did a "bower
install [github URL]," used the resulting extension in my web page, and
went from having the idea that a project was cool to seeing it live on my
website in under five minutes, that has me pretty excited for package
managers being aware of github

Sorry, I can't tell whether the tone of this is aggressive, it's really
not meant that way if it's coming off that way, more "whoa look at what's
cool in this part of the idea space"

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/LLNL/spack/issues/1108#issuecomment-228935394, or mute
the thread
https://github.com/notifications/unsubscribe/AB1cd4Ll7XxON0MpHwDD_I1sU9e7nd9Mks5qQI3_gaJpZM4I963P
.

[alalazo]

Just my two cents : I think the interface @citibeth is proposing is a good one. Having a url may give you additional information (you may be able to infer a name, you can checksum a package, etc.) but is not an essential information you need to create a skeleton for a package.py that in any case needs to be refined afterwards.

I won't comment on issues or prioritization : I think the topic can very well be a (long) thread on its own ... 😄

[adamjstewart]

@DavidPoliakoff I'm not entirely sure I understand what functionality you would like, but it sounds like it's outside of the scope of this issue (spack create). You can open a new issue and let people comment on it.

@citibeth I see your point. I was arguing in terms of usability and you were arguing in terms of sensibility (grammar and consistency). I'm not opposed to your naming scheme. @tgamblin Thoughts on the following scheme:

spack create [package] [-u url] [-t template]

It certainly won't be backwards compatible, but I've never cared much for backwards compatibility anyway. We would have to deprecate spack create -n since it would become the new default. I could check the [package] argument and give a warning if someone tries to give a URL instead of a package name.

[citibeth]

If someone says ' Spack create url', why not gently warn them and go ahead
with 'spack create -u url'. You are right, it is easy to tell difference
between a url and pkg name.

Spack create should never overwrite an existing pkg, unless -f is used.

[citibeth]

I believe we should think about the UI in terms of a consistent "correct" grammar that we've defined, and what people might actually type. The UI shown above (spack help create) shows the correct grammar, and I think it's just about perfect. I don't think the names of the flags, or the documentation, should be changed.

If users type something else --- most likely spack create <url> -n <name> --- then Spack spack should certainly point out to the user that this usage is deprecated, and suggest a modern alternative. That's how git works. If Spack can figure out what the user meant, it should do it; that's nicer than quitting and forcing the user to "do it right." However, I don't think that deprecated usages should be an official part of the UI, or should be documented.

[tgamblin]

I was sort of proposing that the spack create URL usage, since it is easy to keep around, could stay without deprecation. I don't think we need to be too rigid in that case. I don't think it needs to be documented as I do think it clutters the text above.

[adamjstewart]

Can someone explain how this line works? I'm confused as to why this is a class instead of a function. Is that the only way to access stage? Is self.build_system propagated outside of create.py? I was thinking about turning ConfigureGuesser into a function that just returned the build_system.

[mwilliammyers]

@adamjstewart That allows you to call the class's instance like a function. It allows you to use it like, ConfigGuesser() - like a function - this calls __call__,just like foo = ConfigGuesser() calls __init__. As to why it is a class, I have't looked at it a lot but maybe just because it has data members...

[tgamblin]

@adamjstewart: it's hacky. It's a callable object because it sets some attributes about what it discovered at the end of __call__. Look at all the self.* lines at the end. You can't just return those things because the ConfigureGuesser is called indirectly within get_checksums, and it would need to know about build systems if you required this stuff to just be returned.

[adamjstewart]

Oh wow, this is hacky. I guess I always assumed spack create just downloaded one version and determined the build system, then did all of the checksum stuff independently. It could probably be rewritten to do the checksum stuff first and then independently run the configure guesser stuff on the last downloaded checksum. Not sure if it's worth the time to do that or not. I hate doing something like this half way...

[adamjstewart]

Another design question for you. Currently, if I run spack create on a PyPi repo, it will say something like:

==> This looks like a URL for restview version 1.2.3
==> Creating template for package restview

but then it will create a package called py-restview. Should it say this looks like a URL for restview or py-restview?

[citibeth]

I like this feature. Probably best to say "py-restview" throughout, to let
it sink in that "py-" has been prepended.

On Thu, Jun 30, 2016 at 10:30 AM, Adam J. Stewart notifications@github.com
wrote:

Another design question for you. Currently, if I run spack create on a
PyPi repo, it will say something like:

==> This looks like a URL for restview version 1.2.3
==> Creating template for package restview

but then it will create a package called py-restview. Should it say this
looks like a URL for restview or py-restview?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/LLNL/spack/issues/1108#issuecomment-229675699, or mute
the thread
https://github.com/notifications/unsubscribe/AB1cd48eDQJ4GjX5r-0uHdHNwPfgCHqhks5qQ9MDgaJpZM4I963P
.

[trws]

Agreed, it should use the actual package name if at all possible.

[adamjstewart]

What exactly is a namespace?

[tgamblin]

Every repo has a namespace that gets prepended to its packages' names so we can tell where they came from. e.g., the builtin repo has namespace builtin. So when you do:

    $ spack install libelf

On a vanilla Spack install, after concretization, the full name is actually set to builtin.libelf. That gets stored in spec.yaml and in the DB, and you can query by namespace with spack find too. If you had registered your own repo (via spack repo add) it might also have a libelf in it.

So, suppose your repo had the namespace anl.lcrc. The concretized spec would have a full name of anl.lcrc.libelf. If you just do spack find, they all show up as libelf. If you do spack find --namespace, yours will show up as anl.lcrc.libelf and the builtin one will be builtin.libelf.

You can also use fully qualified names in Specs. So if you really want to build anl.lcrc.libdwarf that depends on the builtin libelf, but the anl.lcrc repo contains a libelf that shadows the builtin one, you can do this:

    $ spack install anl.lcrc.libdwarf ^builtin.libelf

Basically it's a mechanism for referring to potentially conflicting packages in different repositories.

[adamjstewart]

Oh, that's really useful. So let's say I have two clusters (ours are called Blues and Fusion), both with identical platform/os/target and a single shared GPFS /soft directory. But, each cluster has a different InfiniBand network, so things that depend on MPI need to be built for that particular cluster. Can I build the things that don't depend on MPI in builtin and build the MPI specific software under the blues or fusion namespace? I was wondering what the best way to do that was. We don't use modules yet, but we're looking to switch from SoftEnv (a homegrown thing). Is there a way to tell modules to automatically use Spack-generated modules from the blues namespace when on Blues or the builtin namespace when it doesn't matter?

[tgamblin]

You could use it for that, although this particular thing actually sounds like something @alalazo and I have talked about w.r.t. how the architecture should be extended. namespaces might be useful as a stopgap.

I do not know if @alalazo actually supports any classification by namespace in his module stuff, but I imagine that would be easy to add.

[citibeth]

Are we confusing package with built installations? Shouldn't you be able to build multiple versions of your MPI things on Blues vs. Fusion, providing different MPI dependency resolution?

[alalazo]

@adamjstewart For module generation I think that repo namespaces won't be the way to go to discern among different architectures. What you should use imho is either arch (in a pre #561 branch like ours) or the triplet platform/os/target in more recent branches.

At EPFL in fact we share your same situation : what we did was to set up an automated way (i.e. we use Jenkins) to build software for 5 different arch types. For modules we use lmod ( see #107 ) and have further extensions that permit to map an arbitrary number of virtual dependencies to an lmod hierarchy.

I may be more specific if you would like more information, but maybe we should move this discussion elsewhere (to the google group?)

[adamjstewart]

@citibeth Yeah, I can. I was just wondering what namespaces are used for and if they can be useful to my situation. Ideally, I could set something in packages.yaml that would say "If I'm on Blues, always use mvapich2+psm, if I'm on Fusion, always use mvapich2+mrail". Similarly, I would like to be able to tell modules to load the correct one, and have Spack automatically generate module files that differentiate in a way that modules can understand.

@alalazo But once our new machine lands, we'll have two linux-centos7-x86_64 machines, so I don't think platform/os/target is sufficient for us to distinguish them.

Anyway, I was just wondering because it's an option for spack create and I'm trying to wrap my head around everything before I break it.

[alalazo]

@adamjstewart Ok, but you can either define a custom target or wait for spack to be extended to account for the network type. If your targets were, say x86_IntelIB vs x86_Mellanox the problem would be solved, right?

[adamjstewart]

@alalazo Probably? Although things that don't depend on MPI should theoretically work across both. Right now, our two machines aren't actually on the same version of CentOS, and I'm finding that things are linking to system libraries. In particular, glibc is a different version on each machine, and that's making almost everything incompatible if it was installed on the other machine. Anyway, this tangent has gone on long enough. I'll ask you for more details once we actually switch to modules.

[alalazo]

Probably? Although things that don't depend on MPI should theoretically work across both.

Yes, but what's the harm in duplicating a build ? spack is happy to build things for you 😄

[citibeth]

I agree, I would just have two separate spacks cnd be done with the issue.
On Jun 30, 2016 3:13 PM, "Massimiliano Culpo" notifications@github.com
wrote:

Probably? Although things that don't depend on MPI should theoretically
work across both.

Yes, but what's the harm in duplicating a build ? spack is happy to build
things for you 😄

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/LLNL/spack/issues/1108#issuecomment-229759465, or mute
the thread
https://github.com/notifications/unsubscribe/AB1cdzul0Ggx-FXiu9OazRfJg9p4XLzJks5qRBVCgaJpZM4I963P
.

[adamjstewart]

Time, space. Mostly it's annoying that I can't use Spack while I'm waiting for the package to finish installing. But I think we have a couple PRs in progress now that should alleviate that 😃

[adamjstewart]

Anyway, if this isn't a great use case for using different namespaces, what is?

[alalazo]

@adamjstewart Having site specific packages that you need to install for any reason, but that are not worth to be put / cannot be put in spack builtin. In that case you can set up a repo of those packages somewhere else.

In the long run the feature may also be useful to set up, e.g. independent community specific repos (though this needs spack to be a bit more stable than it currently is)

[adamjstewart]

spack edit --force has now been replaced by spack create as of #2707! Thanks to everyone for such a lively discussion on something I thought was only of annoyance to me.

@citibeth's proposal to change the UI from spack create --name <name> <url> to spack create --url <url> <name> has been separated out into #2843. I'll let others comment on that PR and close this issue as the original intent has been satisfied.