Skip to content

Conversation

@matt23654
Copy link
Contributor

Fixes #17536

Submitted as a draft for now as I do not have a Spark to hand to confirm cudaGetDeviceProperties. I wrote a small CUDA to confirm that prop.unifiedAddressing == 1 on my discrete GPUs. This means it cannot be used to check if the device is a DGX Spark.

https://developer.download.nvidia.com/compute/DevZone/docs/html/C/doc/html/group__CUDART__DEVICE_g5aa4f47938af8276f08074d09b7d520c.html -- these docs seem to suggest that this should be prop.integrated instead? This flag is 0 on my discrete GPUs.

@github-actions github-actions bot added Nvidia GPU Issues specific to Nvidia GPUs ggml changes relating to the ggml tensor library for machine learning labels Nov 27, 2025
@ggerganov
Copy link
Member

On DGX Spark, prop.integrated is equal to 1.

@matt23654
Copy link
Contributor Author

thanks for checking that. looks like this straightforward fix works.

@matt23654 matt23654 marked this pull request as ready for review November 27, 2025 11:23
@ggerganov ggerganov merged commit 909072a into ggml-org:master Nov 27, 2025
71 of 74 checks passed
Copy link
Contributor

@ORippler ORippler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, just spotted this on our side as well.

@yeahdongcn
Copy link
Collaborator

yeahdongcn commented Nov 28, 2025

https://manpages.ubuntu.com/manpages/focal/en/man3/CUDART_UNIFIED.3.html

Whether or not a device supports unified addressing may be queried by calling cudaGetDeviceProperties()
with the device property cudaDeviceProp::unifiedAddressing.

Unified addressing is automatically enabled in 64-bit processes .

I’ve also encountered this issue on MTGPU. llama-cli is incorrectly reading /proc/meminfo to determine the free VRAM. Thanks for fixing this!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ggml changes relating to the ggml tensor library for machine learning Nvidia GPU Issues specific to Nvidia GPUs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Misc. bug: 5090 incorrectly recognized as unified memory.

4 participants