-
Notifications
You must be signed in to change notification settings - Fork 2.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
new dependency types merge broke R dependency resolution #1337
Comments
Ah, this is probably because R dependencies should also be |
I have a Python package that depends on numpy, scipy, and matplotlib. I haven't specified the dependency types because I'm not exactly sure what they should be. Anyway, when I activate my Python package, it activates numpy but not scipy or matplotlib. When I try to import my package, it complains that scipy doesn't exist. Is this a bug? Am I just missing the proper dependency types? What should they be? |
@mathstuf @adamjstewart |
Adding the nolink deptype to my python package fixed the problem. Sorry about the confusion. I think Python is handled correctly. Seems like the problem may be restricted to R. |
Yeah, adding |
So every r- package, and presumably, every python package, would need to have that specified for each dependency? |
Yeah, the default is |
I guess I am not entirely clear on how the deptype searching works but would changing the deptype_query setting in the R package itself be an option? |
I should note that I am not opposed to making the change in the existing package files. What concerns me though going forward is that specifying the deptype may be forgotten when developing extension packages and then lead to odd problems. That may lead to "fixing" the problem by over specifying dependencies in the respective package files. That in turn will lead to diverging views between spack and CRAN about what the dependencies are for R packages. A similar problem could arise with python and pip. |
It would work, but the thing is that then R packages would be considered for things like rpath by default which could make the paths overly long. |
Spack originally had a single deptype that was build, link, and run all in one. With the switch to deptypes we're trying to avoid pulling unnecessary things into the runtime environment, particularly build dependencies. We also made the default deptype build/link since, as @mathstuf said, that's what a typical library wants. It seems like we didn't make things intuitive enough. We actually punted on
I think packages should ideally be explicit, but I think it's clear this isn't intuitive enough right now, and it might be hard to make this intuitive for beginners. It's a barrier for new packagers. Would making the default deptype build/link/run make sense? It might reduce confusion for packagers if they do NOT yet understand dep types. This addresses Glenn's concern above. |
@mathstuf |
If Rcpp is found using R's module loading logic, then it's a |
@tgamblin What about adding a comment to the boilerplate template reminding packagers to set the deptype for extension packages like R? That is probably sufficient to address the concern that I raised, at least for packages created with |
I agree that if the concern is mostly about misuse, documentation (and specifically a reminder in the template package) is the way to handle it. From: Glenn Johnson <notifications@github.commailto:notifications@github.com> @tgamblinhttps://github.com/tgamblin What about adding a comment to the boilerplate template reminding packagers to set the deptype for extension packages like R? That is probably sufficient to address the concern that I raised, at least for packages created with spack create. — |
I can do that! |
I will add the
For the non-r dependencies, the ones that are Thanks. |
Leave them blank if you just want the default. |
The following change from #378 broke dependency resolution in R packages.
After a package is built for R it loads it in a session. It is required that the dependency chain be on the R_LIB path for that to succeed. That broke with the above change and so all second level dependencies would have to be specified as direct dependencies. That then makes dependency specification out of sync with what is on CRAN.
I have another change that I need to make to the R package and I can revert the above change in that PR. However, the above change was also made for the python and lua packages and I do not know if there is a similar issue with those.
The text was updated successfully, but these errors were encountered: