A quick question: how do I calculate overhead for a model?

[znsoftm]

for BERT mode, its overhead is calculated as :

model_mem_req += (5 + 16 * n_layer) * 256; // object overhead

Can anyone explain the meaning 5, 16 , 256 for me ?

[znsoftm]

for GPT-2, it is:

ctx_size += (6 + 12*n_layer)*512; // object overhead

[znsoftm]

For the model BERT, it has n_layer, each layer has 16 tensors and 5 extra parameters (tensor), but what the 512 is for? I guess it is the size of "struct ggml_tensor", but it is not...
so what is the 256 for? It does be the size of something...

the sizeof(struct ggml_tensor) is 208, so 256 is rounded to n times 16?

[ggerganov]

The object overhead is calculated as:

size_t overhead_bytes = n_tensors * ggml_tensor_overhead();