Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor RedisAI Build #669

Merged
merged 80 commits into from
Sep 19, 2024
Merged

Refactor RedisAI Build #669

merged 80 commits into from
Sep 19, 2024

Conversation

ashao
Copy link
Member

@ashao ashao commented Aug 15, 2024

  • The RedisAIBuilder class was completely overhauled to allow users to
    express a wider range of support for hardware/software stacks. This
    will be extended to support ROCm, CUDA-11, and CUDA-12.
  • Versions for each of these packages are no longer specified in an
    internal class. Instead a default set of JSON files specifies the
    sources and versions. Users can specify their own custom specifications
    at smart build time

MattToast and others added 20 commits June 10, 2024 18:35
Updates to the SmartSim build process:
- SmartSim can now be built using Cuda version 11.8 or Cuda 12.1 by
specify `smart build --device=cuda118` or `smart build
--device=cuda121`. The original `smart build --device=gpu` will default
to using Cuda 11.8.
- As a result of the previous change, SmartSim now requires C++17 and a
minimum Cuda version of 11.8 in order to build Torch 2.1.0.
- In addition this PR now uses Torch 2.1 and TF 2.14/2.15 (for CUDA 11.8/12.1 respectively)

[ committed by @MattToast ]
[ reviewed by @ashao ]
The existing dockerfiles did not have the necessary environments to
enable GPU support for users wanting to run the tutorials. Two
Dockerfiles (for CUDA 11.8 and 12.1) are provided. Note that this has
only been tested on a workstation and no HPC support should be expected
at this time.

[ committed by @ashao ]
[ reviewed by @MattToast ]
Add 'f' to intended f-strings for better error messages

[ committed by @MattToast ]
[ reviewed by @ashao ]
Copy link
Member

@MattToast MattToast left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great! Such a nicer build system than was there previously!!

I just left some initial feedback while we resort out the CI and get everything building and passing. As always, feel free to lmk what you think!!

setup.py Outdated Show resolved Hide resolved
smartsim/_core/_cli/build.py Outdated Show resolved Hide resolved
smartsim/_core/_install/utils.py Outdated Show resolved Hide resolved
smartsim/_core/_install/utils.py Outdated Show resolved Hide resolved
smartsim/_core/utils/helpers.py Outdated Show resolved Hide resolved
smartsim/_core/_install/utils.py Outdated Show resolved Hide resolved
smartsim/_core/_install/utils.py Outdated Show resolved Hide resolved
tests/install/test_package_retriever.py Outdated Show resolved Hide resolved
tests/test_builder.py Outdated Show resolved Hide resolved
@ashao ashao changed the base branch from cuda-12-support to develop September 19, 2024 19:05
@ashao ashao merged commit 5fb8eb4 into CrayLabs:develop Sep 19, 2024
38 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants