Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

spack does not handle multiple repos any more #5754

Closed
luigi-calori opened this issue Oct 13, 2017 · 7 comments
Closed

spack does not handle multiple repos any more #5754

luigi-calori opened this issue Oct 13, 2017 · 7 comments
Assignees

Comments

@luigi-calori
Copy link
Contributor

I had recently updated develop. I was using an additional repo.
Now this seem to prevent spack to work:

To reproduce:
cd /tmp
git clone https://github.com/LLNL/spack.git
cd spack
source share/spack/setup-env.sh
spack spec zlib
---------- WORK as expected ----------
spack repo create /tmp/goofy
spack repo add /tmp/goofy
spack spec zlib
==> Error: Invalid namespace for goofy repo: builtin

@luigi-calori
Copy link
Contributor Author

I' ve used git bisect to find the problem ( beware I' m a newbie on bisect.... first time ) and seems the problem shows up after commit
65b3876
That is the concretizer speedup
Am I doing something wrong?

@luigi-calori
Copy link
Contributor Author

@tgamblin Sorry, It seems to me the problem persist: Am I alone in using multiple repos or am I doing something basically wrong? Thanks in advance

@luigi-calori
Copy link
Contributor Author

By trying and testing I found that this problem goes away by changing fullname to name in line 1198 of spec.py
RemoteConnectionManager@adaa478

As I' ve not the slightest idea of what I' m doing, hope that someone shade some light on this.
For now, continue testing as I need the repo functionality

In case still work, will submit a PR

@scheibelp scheibelp self-assigned this Oct 16, 2017
@mwkrentel
Copy link
Member

I was just about to report the same problem.
I get the same 'Invalid namespace' error.

Reverting to commit db14987 seems to solve the problem,
at least for the few examples I've tried so far.

@scheibelp
Copy link
Member

I think I see the issue here, although I'm not sure how it was triggered only recently. I'll open a PR in the next hour or so.

scheibelp added a commit to scheibelp/spack that referenced this issue Oct 16, 2017
Previously when RepoPath.repo_for_pkg was invoked with a string,
it did not check if the string included a namespace. Any
namespace-qualified package provided as a string would not be found
(at which point the behavior was to return the highest-precedence
repository).
@pramodskumbhar
Copy link
Contributor

Same issue here!

@luigi-calori
Copy link
Contributor Author

Confirm that #5787 fix the issue, Thanks a lot @scheibelp

scheibelp added a commit to scheibelp/spack that referenced this issue Oct 17, 2017
davydden referenced this issue Oct 18, 2017
This isn't a rework of the concretizer but it speeds things up a LOT.

The main culprits were:
  1. Variant code, `provider_index`, and `concretize.py` were calling
     `spec.package` when they could use `spec.package_class`
    - `spec.package` looks up a package instance by `Spec`, which requires a
      (fast-ish but not that fast) DAG compare.
    - `spec.package_class` just looks up the package's class by name, and you
        should use this when all you need is metadata (most of the time).
    - not really clear that the current way packages are looked up is
      necessary -- we can consider refactoring that in the future.

  2. `Repository.repo_for_pkg` parses a `str` argument into a `Spec` when
     called with one, via `@_autospec`, but this is not needed.
     - Add some faster code to handle strings directly and avoid parsing

This speeds up concretization 3-9x in my limited tests.  Still not super
fast but much more bearable:

Before:
  - `spack spec xsdk` took 33.6s
  - `spack spec dealii` took 1m39s

After:
  - `spack spec xsdk` takes 6.8s
  - `spack spec dealii` takes 10.8s
tgamblin pushed a commit that referenced this issue Oct 25, 2017
* Fixes #5754

Previously when RepoPath.repo_for_pkg was invoked with a string,
it did not check if the string included a namespace. Any
namespace-qualified package provided as a string would not be found
(at which point the behavior was to return the highest-precedence
repository).

* handle nested namespaces for packages specified as strings in repo_for_pkg
* add preliminary repository tests
* add test which replicates #5754
* refactor repo tests with fixtures
* define repo_path equivalent at test-level scope for repo tests
* add tests for unknown namespace/package
* rename fixture function (no longer prefixed with 'test_')
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants