New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
CUDA 9.2 has been released #2084
Comments
I think we'll be able to test out multi-CUDA builds this week, starting with the @Lnaden has a docker image that includes texlive-2018 with CUDA 7.5, 8.0, 9.0, 9.1, and 9.2. We may need to split this up into different images due to image size, but we can start building variants that are paired to each of these CUDA versions. How should we test these aside from the quick test that runs in the docker image? Does AWS have an easy way to spin up instances using AMIs with different CUDA versions installed? We have CUDA 9.0 and 7.5 clusters we can test on physically. |
I haven't done anything with AWS in years. That sounds like the sort of thing that ought to be easy, but I'm not sure. You might also be able to use a single image with all the versions available. That's how a lot of clusters handle it: you put |
Ideally, we'd test with different versions of the driver that comes with the CUDA install. |
@peastman : What's the earliest version of CUDA that the current OpenMM code will work with? |
I'm not sure. It's been quite a while since we intentionally dropped support for a version, but we also haven't tested with old versions in a long time. Going back to 7.5 is probably plenty early enough. |
OK with me. It should be easier to add earlier versions if needed. We've almost got all the docker images built. The next step is to automate the builds for the |
@peastman: Do we want to stick with clang-3.8.1, or upgrade to a more recent release? The releases available to us without great pain are in the |
This is probably a good time for upgrading. Let's try using the most recent release and see if any problems come up. |
Thanks! Moving the discussion of the test builds to here: |
CUDA 9.2 builds are available for testing: https://anaconda.org/omnia/openmm/files conda install -c omnia/label/dev openmm==7.3.2 I'm still awaiting access to CUDA 9.2 upgraded nodes (should have those Monday) for local testing. If this process seems to work, I'll proceed to automate all of the other CUDA builds. |
@peastman : The CUDA 9.2 build ( Could you give it a try too? If it works OK for you, then we can automate the builds for CUDA 7.5-9.2 on |
May I ask how to i compile from source for CUDA9.2? |
Compilation instructions are here: http://docs.openmm.org/latest/userguide/library.html#compiling-openmm-from-source-code Just make sure you have the CUDA 9.2 toolkit installed! |
Thank you for the instructions! However, I have been stuck at: |
There should be an error message somewhere earlier in the output describing what the problem was. Can you find it and post it? Or if that isn't possible, post a link to the full output. |
@peastman : Multi-CUDA Can you try out the following? conda install --yes -c omnia -c omnia/label/dev cuda92 openmm==7.3.0
conda install --yes -c omnia -c omnia/label/dev cuda91 openmm==7.3.0
conda install --yes -c omnia -c omnia/label/dev cuda90 openmm==7.3.0
conda install --yes -c omnia -c omnia/label/dev cuda80 openmm==7.3.0
conda install --yes -c omnia -c omnia/label/dev cuda75 openmm==7.3.0 You may need to remove the packages in between with conda remove --yes openmm cudaXY
conda clean -tipsy where |
@peastman I am able to compile if i only build for Cuda. If I build library with OpenCL or PME for CPU, the make will fail. |
@linusyukwong: We can't help you unless you can post the error messages indicating why the build is failing. Better yet, can you ZIP up and post the whole build output, along with you or CMakeLists.txt file? Also, any chance you can use the conda installs instead since you seem to be having trouble compiling OpenMM? |
I'll give it a try. What happens if the user has multiple cudaXX packages installed? |
Right now, you can accidentally install multiple For now, just making sure that the package works as expected (since we upgraded to clang 6 and are auto-building CUDA and OpenCL versions in a different way) is the focus. |
It looks like I wonder if we could instead add these as dependencies and dispense with the |
Perhaps we can find out who maintains it and see if we can get them to add 9.1 and 9.2? |
I've asked here: ContinuumIO/anaconda-recipes#140 |
If one doesn't specify a CUDA feature package (e.g. |
It should install the CUDA 9.2 version. If CUDA 9.3 comes out and we then add a build for that, it should continue to install the CUDA 9.2 version by default. I see this as being a purely optional feature that should only make installation easier, never harder. Suppose we weren't adding this feature, and we were building this release the same way we've done past releases. In that case we would compile it against CUDA 9.2, and we would document that it required that specific version. Users would then install it with a simple The one case where the old method failed was if someone needed to use a different CUDA version. In that case, they had to compile from source. So the goal is to make that case easier without making the normal case harder. Another possibility we could consider is doing this with labels. If someone wanted the CUDA 8.0 build, they could install it with |
Unfortunately, I don't think this would work. When I tested this earlier, it was clear that a specific package build could have several labels (e.g. |
Surprisingly, this seems to be possible now! Check this out: Let's give this a try and see if it satisfies your desiderata. For now, you'll add Can you try this? Install default CUDA 9.2 version from channel
Install specific CUDA 9.0 version:
etc. |
The command
gives this error:
|
Sorry, had assumed you already had Try
|
I still get the same error message. |
I also get the same error if I leave off the label, so that doesn't seem to be the problem. I'm not sure what's going on. Conda can see the packages on your channel:
But for some reason it can't install them. Here's another strange behavior that might or might not be a clue.
Why does it list |
This is super weird. I cleaned up some of those other unnecessary things (like Can you try temporarily wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh
bash Miniconda3-latest-Linux-x86_64.sh -b -p $HOME/miniconda
export PATH=$HOME/miniconda/bin:$PATH Then you should be able to grab OpenMM with
which should see just the package under the |
If that doesn't work, I wonder if the order is important here?
|
Oh, I wonder if the issue is that I have the |
Have you done that? What channel should I try to install it from? |
Still sorting this out. So far, we're still using the |
There's also a PR open to add the actual CUDA toolkits to conda-forge/conda-forge.github.io#63 (comment) That PR would just install the libraries that NVIDIA allows redistribution of. This is presumably not sufficient for building OpenMM, but could be sufficient for delivering OpenMM without CUDA toolkit installation if I understand correctly. |
@peastman : The If you want a dev build for a specific version of CUDA---say CUDA 7.5---you can use conda install --yes -c omnia/label/cuda75 openmm If you want the default dev build for the latest CUDA (CUDA 9.2), you can use conda install --yes -c omnia/label/dev openmm Later, when we do this for the OpenMM release, this default would just be conda install --yes -c omnia openmm and we can make the dev builds conda install --yes -c omnia/label/cuda75-dev openmm Can you test this out? |
It works now! |
@peastman: Great! I think we should still do some benchmarking to make sure the CUDA and OpenCL performance is as expected, and that we haven't accidentally degraded CPU performance with the compiler toolchain we're using now. If all looks good, I propose we use this approach for an OpenMM 7.3 release "soon-ish". We can push release (candidate) builds of 7.3.0 to different CUDA channels, test them, and then add the @Lnaden is currently looking into upgrading our conda build infrastructure to the new There are also interesting things afoot with |
@peastman : We could simplify the conda installation process a great deal if we can use the anaconda supplied Currently, only toolkit versions up to 9.0 are provided. This issue comment notes that this is awaiting a workable recipe for packaging 9.2, and a pull request to this repository that houses the packaging scripts and recipes would be welcomed. For OpenMM 7.3, we could either contribute to this and encourage the release of updated |
We now have a scheme for deploying different CUDA builds to different labels, and are monitoring the potential for using the |
https://developer.nvidia.com/cuda-toolkit/whatsnew
We should update our
dev
builds to 9.2 and think about what might go into a new OpenMM release.I'm pretty sure we can restructure our build framework to use something similar to the pytorch system to allow the user to select their CUDA version by installing a stub feature package:
pytorch
appears to use a single docker image where multiple versions of CUDA are installed in different paths, and a symlink can then be used to select among paths. We may instead be able to use a different docker container for each CUDA version if that proves problematic.It will probably take us about a week to get this working.
The text was updated successfully, but these errors were encountered: