Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

allow for cmake installation #160

Closed
wants to merge 3 commits into from
Closed

Conversation

marcelo-alvarez
Copy link
Contributor

This modification changes desiInstall such that if there is a CMakeLists.txt file in the top-level directory of a package, then it will use the commands cmake .; make -j 8 to install. This currently is for specex but should work generally and can be modified as necessary in the future.

@sbailey
Copy link
Contributor

sbailey commented Feb 3, 2021

@weaverba137 this PR appears to have trigged a bunch of MaskedArrayWithLimits-related test failures completely unrelated to the changes in this PR. Please investigate.

Copy link
Contributor

@sbailey sbailey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks ok, with minor cleanup requests.

Let's get unreleated test failures sorted out first, perhaps in a separate PR and then re-merged into this one.

@@ -516,12 +516,16 @@ def build_type(self):
self.log.debug("Forcing build type: make")
build_type.add('make')
else:
print('cwd: ',self.working_dir);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove bare print or switch to useing self.log.debug.

proc = Popen(command, universal_newlines=True,
if 'cmake' in self.build_type:
proc = Popen(command, universal_newlines=True,
stdout=PIPE, stderr=PIPE, shell=True)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems messy / fragile where "cmake" requires two-commands-in-one and thus a string command with shell=True, while "make" uses a single-command-passed-as-a-list with default shell=False. The normal admonishment against shell=True [link] is due to security concerns about shell injection vulnerabilities from unsanitized user input strings, but in this case we have completely control over the commands created without any user-provided options, and thus I think both "make" and "cmake" could use string (not list) commands with shell=True to avoid this if/else. Thoughts?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To further @sbailey's comment, did you actually test the commands with shell=False and come to the conclusion that it was impossible to run with shell=False?

I don't think anyone would object if we ran two commands, both with shell=False, if the one and only reason for shell=True is a ;.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did a test and found it would require two running two commands and the shell=True was chosen. Will try the setup.py solution @tskisner mentioned below and if so these changes will be unnecessary.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That sounds promising. I was about to point out that you have if 'cmake' in self.build_type:, but you have not modified the code in any way to detect 'cmake' as the build_type.

@tskisner
Copy link
Member

tskisner commented Feb 3, 2021

Just throwing out another option- it is possible to have a setup.py which calls CMake to do the actual building of extensions. See for example: https://github.com/hpc4cmb/toast/blob/master/setup.py

This even supports setting environment variables to pass options to the underlying cmake call.

This way the desiInstall tools can just run setup.py and not worry about what it is doing under the hood. It is just a question of where you want to put the extra logic- in the package or in the external tools.

@marcelo-alvarez
Copy link
Contributor Author

Just throwing out another option- it is possible to have a setup.py which calls CMake to do the actual building of extensions. See for example: https://github.com/hpc4cmb/toast/blob/master/setup.py

This even supports setting environment variables to pass options to the underlying cmake call.

This way the desiInstall tools can just run setup.py and not worry about what it is doing under the hood. It is just a question of where you want to put the extra logic- in the package or in the external tools.

I think having cmake called by setup.py is preferable and will look into it, thanks for pointing this out.

@weaverba137
Copy link
Member

Requiring numpy<1.20 fixed the masked array problems reported. I'll spin that to a separate issue.

@marcelo-alvarez
Copy link
Contributor Author

The specex build now uses python setup.py instead of cmake, so this change is no longer necessary. Closing this pull request and deleting the branch.

@marcelo-alvarez marcelo-alvarez deleted the spx_io_refactor branch February 5, 2021 20:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants