Skip to content

Conversation

@CISC
Copy link
Collaborator

@CISC CISC commented Dec 8, 2025

Allows ggml_fill to effectively be made in place if possible.

@CISC CISC requested a review from ggerganov December 8, 2025 20:18
@CISC
Copy link
Collaborator Author

CISC commented Dec 8, 2025

So, it looks like this can only inplace a node's parent once:

// if the node's data is external, then we cannot re-use it
if (!ggml_gallocr_is_own(galloc, parent)) {
AT_PRINTF("not reusing parent %s for %s as %p is external\n", parent->name, node->name, parent->data);
continue;
}

static bool ggml_gallocr_is_own(ggml_gallocr_t galloc, struct ggml_tensor * t) {
return ggml_gallocr_hash_get(galloc, t)->allocated;
}

p_hn->allocated = false; // avoid freeing the parent
view_src_hn->allocated = false;

@github-actions github-actions bot added Nvidia GPU Issues specific to Nvidia GPUs ggml changes relating to the ggml tensor library for machine learning labels Dec 8, 2025
@CISC CISC merged commit 86a3f0f into master Dec 9, 2025
77 of 78 checks passed
@CISC CISC deleted the cisc/allow-fill-alloc-inplace branch December 9, 2025 11:23
Ethan-a2 pushed a commit to Ethan-a2/llama.cpp that referenced this pull request Dec 12, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ggml changes relating to the ggml tensor library for machine learning Nvidia GPU Issues specific to Nvidia GPUs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants