Your current environment
conda now solves this without choking (it never used to work, and I presume that's why you added the nvidia channel at the top):
With this (flexible channel priority, didn't try strict):
name: aphrodite
channels:
- pytorch
- nvidia/label/cuda-12.4.1
- conda-forge
- defaults
dependencies:
- python=3.11.*
- pytorch=2.4.0
- pytorch-cuda=12.4.*
- cuda-nvcc=12.4.*
- cuda-libraries-dev=12.4.*
- cuda=12.4
- cuda-version=12.4.*
- nccl
- cxx-compiler
- gxx=11.3
- pip
- git=2.35.1
- ninja
I get:
cuda 12.4.1 0 nvidia/label/cuda-12.4.1
cuda-cccl 12.4.127 0 nvidia/label/cuda-12.4.1
cuda-command-line-tools 12.4.1 0 nvidia/label/cuda-12.4.1
cuda-compiler 12.4.1 0 nvidia/label/cuda-12.4.1
cuda-cudart 12.4.127 0 nvidia/label/cuda-12.4.1
cuda-cudart-dev 12.4.127 0 nvidia/label/cuda-12.4.1
cuda-cudart-static 12.4.127 0 nvidia/label/cuda-12.4.1
cuda-cuobjdump 12.4.127 0 nvidia/label/cuda-12.4.1
cuda-cupti 12.4.127 0 nvidia/label/cuda-12.4.1
cuda-cupti-static 12.4.127 0 nvidia/label/cuda-12.4.1
cuda-cuxxfilt 12.4.127 0 nvidia/label/cuda-12.4.1
cuda-demo-suite 12.4.127 0 nvidia/label/cuda-12.4.1
cuda-documentation 12.4.127 0 nvidia/label/cuda-12.4.1
cuda-driver-dev 12.4.127 0 nvidia/label/cuda-12.4.1
cuda-gdb 12.4.127 0 nvidia/label/cuda-12.4.1
cuda-libraries 12.4.1 0 nvidia/label/cuda-12.4.1
cuda-libraries-dev 12.4.1 0 nvidia/label/cuda-12.4.1
cuda-libraries-static 12.4.1 0 nvidia/label/cuda-12.4.1
cuda-nsight 12.4.127 0 nvidia/label/cuda-12.4.1
cuda-nsight-compute 12.4.1 0 nvidia/label/cuda-12.4.1
cuda-nvcc 12.4.131 0 nvidia/label/cuda-12.4.1
cuda-nvdisasm 12.4.127 0 nvidia/label/cuda-12.4.1
cuda-nvml-dev 12.4.127 0 nvidia/label/cuda-12.4.1
cuda-nvprof 12.4.127 0 nvidia/label/cuda-12.4.1
cuda-nvprune 12.4.127 0 nvidia/label/cuda-12.4.1
cuda-nvrtc 12.4.127 0 nvidia/label/cuda-12.4.1
cuda-nvrtc-dev 12.4.127 0 nvidia/label/cuda-12.4.1
cuda-nvrtc-static 12.4.127 0 nvidia/label/cuda-12.4.1
cuda-nvtx 12.4.127 0 nvidia/label/cuda-12.4.1
cuda-nvvp 12.4.127 0 nvidia/label/cuda-12.4.1
cuda-opencl 12.4.127 0 nvidia/label/cuda-12.4.1
cuda-opencl-dev 12.4.127 0 nvidia/label/cuda-12.4.1
cuda-profiler-api 12.4.127 0 nvidia/label/cuda-12.4.1
cuda-runtime 12.4.1 0 nvidia/label/cuda-12.4.1
cuda-sanitizer-api 12.4.127 0 nvidia/label/cuda-12.4.1
cuda-toolkit 12.4.1 0 nvidia/label/cuda-12.4.1
cuda-tools 12.4.1 0 nvidia/label/cuda-12.4.1
cuda-version 12.4 h3060b56_3 conda-forge
cuda-visual-tools 12.4.1 0 nvidia/label/cuda-12.4.1
gds-tools 1.9.1.3 0 nvidia/label/cuda-12.4.1
libcublas 12.4.5.8 0 nvidia/label/cuda-12.4.1
libcublas-dev 12.4.5.8 0 nvidia/label/cuda-12.4.1
libcublas-static 12.4.5.8 0 nvidia/label/cuda-12.4.1
libcufft 11.2.1.3 0 nvidia/label/cuda-12.4.1
libcufft-dev 11.2.1.3 0 nvidia/label/cuda-12.4.1
libcufft-static 11.2.1.3 0 nvidia/label/cuda-12.4.1
libcufile 1.9.1.3 0 nvidia/label/cuda-12.4.1
libcufile-dev 1.9.1.3 0 nvidia/label/cuda-12.4.1
libcufile-static 1.9.1.3 0 nvidia/label/cuda-12.4.1
libcurand 10.3.5.147 0 nvidia/label/cuda-12.4.1
libcurand-dev 10.3.5.147 0 nvidia/label/cuda-12.4.1
libcurand-static 10.3.5.147 0 nvidia/label/cuda-12.4.1
libcusolver 11.6.1.9 0 nvidia/label/cuda-12.4.1
libcusolver-dev 11.6.1.9 0 nvidia/label/cuda-12.4.1
libcusolver-static 11.6.1.9 0 nvidia/label/cuda-12.4.1
libcusparse 12.3.1.170 0 nvidia/label/cuda-12.4.1
libcusparse-dev 12.3.1.170 0 nvidia/label/cuda-12.4.1
libcusparse-static 12.3.1.170 0 nvidia/label/cuda-12.4.1
libnpp 12.2.5.30 0 nvidia/label/cuda-12.4.1
libnpp-dev 12.2.5.30 0 nvidia/label/cuda-12.4.1
libnpp-static 12.2.5.30 0 nvidia/label/cuda-12.4.1
libnvfatbin 12.4.127 0 nvidia/label/cuda-12.4.1
libnvfatbin-dev 12.4.127 0 nvidia/label/cuda-12.4.1
libnvjitlink 12.4.127 0 nvidia/label/cuda-12.4.1
libnvjitlink-dev 12.4.127 0 nvidia/label/cuda-12.4.1
libnvjpeg 12.3.1.117 0 nvidia/label/cuda-12.4.1
libnvjpeg-dev 12.3.1.117 0 nvidia/label/cuda-12.4.1
libnvjpeg-static 12.3.1.117 0 nvidia/label/cuda-12.4.1
nsight-compute 2024.1.1.4 0 nvidia/label/cuda-12.4.1
pytorch 2.4.0 py3.11_cuda12.4_cudnn9.1.0_0 pytorch
pytorch-cuda 12.4 hc786d27_7 pytorch
pytorch-mutex 1.0 cuda pytorch
Ain't life grand?
Collecting environment information...
PyTorch version: 2.4.0
Is debug build: False
CUDA used to build PyTorch: 12.4
ROCM used to build PyTorch: N/A
OS: Ubuntu 22.04.4 LTS (x86_64)
GCC version: (conda-forge gcc 11.3.0-19) 11.3.0
Clang version: Could not collect
CMake version: version 3.26.4
Libc version: glibc-2.35
Python version: 3.11.9 | packaged by conda-forge | (main, Apr 19 2024, 18:36:13) [GCC 12.3.0] (64-bit runtime)
Python platform: Linux-6.8.0-45-generic-x86_64-with-glibc2.35
Is CUDA available: True
CUDA runtime version: 12.4.131
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration:
GPU 0: NVIDIA H100 PCIe
GPU 1: NVIDIA H100 PCIe
Nvidia driver version: 560.35.03
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True
CPU:
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Address sizes: 52 bits physical, 57 bits virtual
Byte Order: Little Endian
CPU(s): 256
On-line CPU(s) list: 0-255
Vendor ID: AuthenticAMD
Model name: AMD EPYC 9554 64-Core Processor
CPU family: 25
Model: 17
Thread(s) per core: 2
Core(s) per socket: 64
Socket(s): 2
Stepping: 1
Frequency boost: enabled
CPU max MHz: 3762.9880
CPU min MHz: 1500.0000
BogoMIPS: 6191.12
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good amd_lbr_v2 nopl nonstop_tsc cpuid extd_apicid aperfmperf rapl pni pclmulqdq monitor ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb cat_l3 cdp_l3 hw_pstate ssbd mba perfmon_v2 ibrs ibpb stibp ibrs_enhanced vmmcall fsgsbase bmi1 avx2 smep bmi2 erms invpcid cqm rdt_a avx512f avx512dq rdseed adx smap avx512ifma clflushopt clwb avx512cd sha_ni avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local user_shstk avx512_bf16 clzero irperf xsaveerptr rdpru wbnoinvd amd_ppin cppc arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif x2avic v_spec_ctrl vnmi avx512vbmi umip pku ospke avx512_vbmi2 gfni vaes vpclmulqdq avx512_vnni avx512_bitalg avx512_vpopcntdq la57 rdpid overflow_recov succor smca fsrm flush_l1d debug_swap
Virtualization: AMD-V
L1d cache: 4 MiB (128 instances)
L1i cache: 4 MiB (128 instances)
L2 cache: 128 MiB (128 instances)
L3 cache: 512 MiB (16 instances)
NUMA node(s): 2
NUMA node0 CPU(s): 0-63,128-191
NUMA node1 CPU(s): 64-127,192-255
Vulnerability Gather data sampling: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Mitigation; Safe RET
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2: Mitigation; Enhanced / Automatic IBRS; IBPB conditional; STIBP always-on; RSB filling; PBRSB-eIBRS Not affected; BHI Not affected
Vulnerability Srbds: Not affected
Vulnerability Tsx async abort: Not affected
Versions of relevant libraries:
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2: Mitigation; Enhanced / Automatic IBRS; IBPB conditional; STIBP always-on; RSB filling; PBRSB-eIBRS Not affected; BHI Not affected
Vulnerability Srbds: Not affected
Vulnerability Tsx async abort: Not affected
Versions of relevant libraries:
[pip3] torch==2.4.0
[pip3] triton==3.0.0
[conda] blas 2.16 mkl conda-forge
[conda] libblas 3.8.0 16_mkl conda-forge
[conda] libcblas 3.8.0 16_mkl conda-forge
[conda] liblapack 3.8.0 16_mkl conda-forge
[conda] liblapacke 3.8.0 16_mkl conda-forge
[conda] mkl 2020.2 256
[conda] nccl 2.23.4.1 h52f6c39_0 conda-forge
[conda] pytorch 2.4.0 py3.11_cuda12.4_cudnn9.1.0_0 pytorch
[conda] pytorch-cuda 12.4 hc786d27_7 pytorch
[conda] pytorch-mutex 1.0 cuda pytorch
[conda] torchtriton 3.0.0 py311 pytorch
ROCM Version: Could not collect
Neuron SDK Version: N/A
Aphrodite Version: N/A
Aphrodite Build Flags:
CUDA Archs: Not Set; ROCm: Disabled; Neuron: Disabled
GPU Topology:
ESC[4mGPU0 GPU1 CPU Affinity NUMA Affinity GPU NUMA IDESC[0m
GPU0 X NODE 64-127,192-255 1 N/A
GPU1 NODE X 64-127,192-255 1 N/A
Legend:
X = Self
SYS = Connection traversing PCIe as well as the SMP interconnect between NUMA nodes (e.g., QPI/UPI)
NODE = Connection traversing PCIe as well as the interconnect between PCIe Host Bridges within a NUMA node
PHB = Connection traversing PCIe as well as a PCIe Host Bridge (typically the CPU)
PXB = Connection traversing multiple PCIe bridges (without traversing the PCIe Host Bridge)
PIX = Connection traversing at most a single PCIe bridge
NV# = Connection traversing a bonded set of # NVLinks
How did you install Aphrodite?
N/A
Your current environment
conda now solves this without choking (it never used to work, and I presume that's why you added the nvidia channel at the top):
With this (flexible channel priority, didn't try strict):
I get:
Ain't life grand?
How did you install Aphrodite?
N/A