Add more container dependency checks in run_sorter#3024
Merged
samuelgarcia merged 14 commits intoSpikeInterface:mainfrom Jun 28, 2024
Merged
Add more container dependency checks in run_sorter#3024samuelgarcia merged 14 commits intoSpikeInterface:mainfrom
run_sorter#3024samuelgarcia merged 14 commits intoSpikeInterface:mainfrom
Conversation
22e1268 to
78ccc27
Compare
0040b2a to
f1438c4
Compare
run_sorter
alejoe91
reviewed
Jun 13, 2024
alejoe91
reviewed
Jun 13, 2024
zm711
reviewed
Jun 17, 2024
Co-authored-by: Zach McKenzie <92116279+zm711@users.noreply.github.com>
Co-authored-by: Alessio Buccino <alejoe9187@gmail.com>
for more information, see https://pre-commit.ci
Collaborator
Author
|
Thanks @alejoe91 and @zm711! @alejoe91 thanks for the apptainer addition, I wanted to test this locally but I have completely KO-d my linux parition by messing around with GPU drivers and it no longer boots 😆. So I can't physically check it with singularity / apptainer installed / not installed but checked the code carefully and command looks good and I'm happy to proceed with merging. Cheers! |
samuelgarcia
approved these changes
Jun 28, 2024
Member
|
This looks perfect to me. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR closes #2855 by adding in additional checks on container and GPU dependencies when running
run_sorter().It adds checks that
dockerorsingularityare installed (if running sorter withdockerorsingularityrespectively) and also that the corresponding python packagesdockerandspythonare installed. It also show a warning if, on Linux, some nvidia-docker tools are not available, which are usually necessary. See #2855 for a disucssion, but these nvidia-docker dependencies are a little complex so instead of raising an error, only a warning is shown.I tested all locally and they work okay. It is not easy to test these nvidia-docker checks in code and so have left these, they raise only a warning anyway. The tests are a little bit 'extra', in particulary testing
has_docker_python()andhas_spython(). This patchessys.moduleto pretend like these modules are not available, then checks thesehas_<dependency>functions returnFalse. However, these tests are a little bit complex and add a reasonable amount of code to check what is basically a try/except function so although it's nice to test as much as possible, would be happy to remove if the maintenance burden is too high.The other tests check that at runtime,
run_sorterraises the appropriate error when thehas_<dependency>functions returnFalse. This is achieved by 'monkeypatching' thesehas_<dependency>functions so they returnFalseat runtime. Then it can be checked thatrun_sorteris raising the correct error. I think these tests are more valuable, are less complex that the tests discussed above.