Validations
Problem
I'm using ollama as the model provider, using the official Deepseek-R1 distilled 32B model on ollama.
The model config of continue is:
"model": "deepseek-r1:32b",
"title": "DeepSeek R1-distilled 32B",
"provider": "ollama",
"contextLength": 128000
However, when I'm asking the model to edit a long piece of code, the response would cutoff suddenly.
I suspect that it is related to the ollama's limitation of the length of the model response, which is controlled by num_predict (see https://github.com/ollama/ollama/blob/main/docs/modelfile.md#parameter).
although the ollama's default num_predict is infinite, I found that when I'm chatting with the model, the log shows the continue gives a maxTokens of 4096:
==========================================================================
==========================================================================
##### Completion options #####
{
"contextLength": 128000,
"model": "deepseek-r1:32b",
"maxTokens": 4096,
"stop": [
"<｜begin▁of▁sentence｜>",
"<｜end▁of▁sentence｜>",
"<｜User｜>",
"<｜Assistant｜>"
]
}
##### Request options #####
{}
##### Prompt #####
<user>
How much is 40eV of single-photon energy converted into atomic units?
<assistant>
To convert 40 eV into atomic units (Hartrees), we use the conversion factor where 1 Hartree ≈ 27.2114 eV.
The calculation is:
Energy in atomic units = 40 eV / 27.2114 ≈ 1.47
Therefore, 40 eV of single-photon energy is approximately **1.47 Hartrees** when converted into atomic units.
I think the maxTokens should also become a tunable parameter in the configuration of continue to allow for longer response. I really need to ask AI to help me to revise my LaTeX document!!
Solution
The solution is just add maxTokens as a tunable parameter in the configuration of continue to allow for longer response :) Thank you for attention.
Validations
Problem
I'm using ollama as the model provider, using the official Deepseek-R1 distilled 32B model on ollama.
The model config of continue is:
However, when I'm asking the model to edit a long piece of code, the response would cutoff suddenly.
I suspect that it is related to the ollama's limitation of the length of the model response, which is controlled by
num_predict(see https://github.com/ollama/ollama/blob/main/docs/modelfile.md#parameter).although the ollama's default
num_predictis infinite, I found that when I'm chatting with the model, the log shows the continue gives amaxTokensof 4096:I think the
maxTokensshould also become a tunable parameter in the configuration of continue to allow for longer response. I really need to ask AI to help me to revise my LaTeX document!!Solution
The solution is just add
maxTokensas a tunable parameter in the configuration of continue to allow for longer response :) Thank you for attention.