Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Receiving error from ollama server after first successful prompt and response #72

Closed
jmdaly opened this issue Mar 5, 2024 · 3 comments
Closed

Receiving error from ollama server after first successful prompt and response #72

jmdaly opened this issue Mar 5, 2024 · 3 comments

Comments

@jmdaly
Copy link
Contributor

jmdaly commented Mar 5, 2024

Hi David,

I've started seeing an issue when using gen.nvim in the last couple of days. When I make a first request to the ollama server using the plugin, I get a response as expected. However, when I make follow-up requests, I'm not able to get any further responses from the ollama server. If I close the gen.nvim buffer and ask something again, I will get another response back, but the context of the conversation has been lost. These follow-up responses contain an error from what I think is the ollama server. Here is a sample error response:

Response data:
{ '{"error":"json: cannot unmarshal string into Go struct field ChatRequest.messages of type api.Message"}' }

I'll paste an entire example set of requests and responses here, in the hope that it's helpful. In the example, I ask codellama to generate a small C++ function, which is does. I then ask it to explain the code to me, and that's when the error occurs. Here is the debug output,

curl --silent --no-buffer -X POST http://localhost:11434/api/chat -d '{"messages": [{"role": "user", "content": "Please write a C++ function to print the prime numbers from 1 to 100. Do not explain it to me"}], "model": "codella
ma:34b-instruct", "stream": true}'
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:35.009173699Z","message":{"role":"assistant","content":"```"},"done":false}', '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:35.047441015Z","message"
:{"role":"assistant","content":"\\n"},"done":false}', '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:35.085628159Z","message":{"role":"assistant","content":"#"},"done":false}', '{"model":"codellama:34b-instruct","crea
ted_at":"2024-03-05T17:05:35.123695404Z","message":{"role":"assistant","content":"include"},"done":false}', '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:35.198421919Z","message":{"role":"assistant","content":" \\u00
3ciostream"},"done":false}', '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:35.235195017Z","message":{"role":"assistant","content":"\\u003e"},"done":false}', '{"model":"codellama:34b-instruct","created_at":"2024-03-05
T17:05:35.27201077Z","message":{"role":"assistant","content":"\\n"},"done":false}', '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:35.308843027Z","message":{"role":"assistant","content":"\\n"},"done":false}', '{"model
":"codellama:34b-instruct","created_at":"2024-03-05T17:05:35.345662982Z","message":{"role":"assistant","content":"void"},"done":false}', '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:35.382433902Z","message":{"role":
"assistant","content":" print"},"done":false}', '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:35.419245195Z","message":{"role":"assistant","content":"Pr"},"done":false}', '{"model":"codellama:34b-instruct","created_a
t":"2024-03-05T17:05:35.456065413Z","message":{"role":"assistant","content":"ime"},"done":false}', '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:35.492872028Z","message":{"role":"assistant","content":"Numbers"},"done
":false}', '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:35.529735857Z","message":{"role":"assistant","content":"("},"done":false}', '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:35.566470304Z","m
essage":{"role":"assistant","content":"int"},"done":false}', '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:35.603336974Z","message":{"role":"assistant","content":" start"},"done":false}', '{"model":"codellama:34b-ins
truct","created_at":"2024-03-05T17:05:35.640141819Z","message":{"role":"assistant","content":","},"done":false}', '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:35.676956973Z","message":{"role":"assistant","content":"
 int"},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:35.713909021Z","message":{"role":"assistant","content":" end"},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:35.750783326Z","message":{"role":"assistant","content":")"},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:35.787617617Z","message":{"role":"assistant","content":" {"},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:35.82453608Z","message":{"role":"assistant","content":"\\n"},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:35.86146572Z","message":{"role":"assistant","content":"   "},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:35.898424389Z","message":{"role":"assistant","content":" for"},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:35.935363189Z","message":{"role":"assistant","content":" ("},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:35.972173152Z","message":{"role":"assistant","content":"int"},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:36.009080314Z","message":{"role":"assistant","content":" i"},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:36.045978564Z","message":{"role":"assistant","content":" ="},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:36.08294064Z","message":{"role":"assistant","content":" start"},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:36.119884015Z","message":{"role":"assistant","content":";"},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:36.156846022Z","message":{"role":"assistant","content":" i"},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:36.193699726Z","message":{"role":"assistant","content":" \\u003c="},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:36.232027162Z","message":{"role":"assistant","content":" end"},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:36.272238082Z","message":{"role":"assistant","content":";"},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:36.309580377Z","message":{"role":"assistant","content":" i"},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:36.34643344Z","message":{"role":"assistant","content":"++)"},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:36.383336973Z","message":{"role":"assistant","content":" {"},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:36.420216072Z","message":{"role":"assistant","content":"\\n"},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:36.457033493Z","message":{"role":"assistant","content":"       "},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:36.493881685Z","message":{"role":"assistant","content":" if"},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:36.530788352Z","message":{"role":"assistant","content":" ("},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:36.567643942Z","message":{"role":"assistant","content":"i"},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:36.604589178Z","message":{"role":"assistant","content":" %"},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:36.641386678Z","message":{"role":"assistant","content":" "},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:36.678248218Z","message":{"role":"assistant","content":"2"},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:36.71516269Z","message":{"role":"assistant","content":" =="},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:36.75203016Z","message":{"role":"assistant","content":" "},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:36.788871802Z","message":{"role":"assistant","content":"0"},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:36.825754667Z","message":{"role":"assistant","content":" ||"},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:36.862601768Z","message":{"role":"assistant","content":" i"},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:36.89951774Z","message":{"role":"assistant","content":" %"},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:36.936406116Z","message":{"role":"assistant","content":" "},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:36.973298694Z","message":{"role":"assistant","content":"3"},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:37.010291791Z","message":{"role":"assistant","content":" =="},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:37.047143467Z","message":{"role":"assistant","content":" "},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:37.08410792Z","message":{"role":"assistant","content":"0"},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:37.121002733Z","message":{"role":"assistant","content":" ||"},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:37.157942703Z","message":{"role":"assistant","content":" i"},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:37.194845011Z","message":{"role":"assistant","content":" %"},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:37.231778067Z","message":{"role":"assistant","content":" "},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:37.268652855Z","message":{"role":"assistant","content":"5"},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:37.305495306Z","message":{"role":"assistant","content":" =="},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:37.342430314Z","message":{"role":"assistant","content":" "},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:37.379275735Z","message":{"role":"assistant","content":"0"},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:37.416151905Z","message":{"role":"assistant","content":" ||"},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:37.453024455Z","message":{"role":"assistant","content":" i"},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:37.489919007Z","message":{"role":"assistant","content":" %"},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:37.526748953Z","message":{"role":"assistant","content":" "},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:37.563510857Z","message":{"role":"assistant","content":"7"},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:37.600391822Z","message":{"role":"assistant","content":" =="},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:37.637302744Z","message":{"role":"assistant","content":" "},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:37.67418099Z","message":{"role":"assistant","content":"0"},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:37.711052939Z","message":{"role":"assistant","content":")"},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:37.747898345Z","message":{"role":"assistant","content":" {"},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:37.784810681Z","message":{"role":"assistant","content":"\\n"},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:37.821686724Z","message":{"role":"assistant","content":"           "},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:37.858459102Z","message":{"role":"assistant","content":" std"},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:37.895354448Z","message":{"role":"assistant","content":"::"},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:37.932200666Z","message":{"role":"assistant","content":"cout"},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:38.005820321Z","message":{"role":"assistant","content":" \\u003c\\u003c i"},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:38.07975631Z","message":{"role":"assistant","content":" \\u003c\\u003c \\""},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:38.116880041Z","message":{"role":"assistant","content":" \\";"},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:38.154014465Z","message":{"role":"assistant","content":"\\n"},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:38.191079665Z","message":{"role":"assistant","content":"       "},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:38.228262956Z","message":{"role":"assistant","content":" }"},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:38.265302891Z","message":{"role":"assistant","content":"\\n"},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:38.302430778Z","message":{"role":"assistant","content":"   "},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:38.339522085Z","message":{"role":"assistant","content":" }"},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:38.376677551Z","message":{"role":"assistant","content":"\\n"},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:38.413720326Z","message":{"role":"assistant","content":"}"},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:38.450877984Z","message":{"role":"assistant","content":"\\n"},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:38.487964601Z","message":{"role":"assistant","content":"```"},"done":false}', "" }
Response data:
{ '{"model":"codellama:34b-instruct","created_at":"2024-03-05T17:05:38.525275496Z","message":{"role":"assistant","content":""},"done":true,"total_duration":3900933048,"load_duration":545483,"prompt_eval_count":28,"prompt_eval_duration":
383055000,"eval_count":96,"eval_duration":3515957000}', "" }
Response data:
{ "" }
curl --silent --no-buffer -X POST http://localhost:11434/api/chat -d '{"messages": ["```", "\n", "#", "include", " <iostream", ">", "\n", "\n", "void", " print", "Pr", "ime", "Numbers", "(", "int", " start", ",", " int", " end",
 ")", " {", "\n", "   ", " for", " (", "int", " i", " =", " start", ";", " i", " <=", " end", ";", " i", "++)", " {", "\n", "       ", " if", " (", "i", " ", " ", "2", " ==", " ", "0", " ||", " i", " ", " ", "3", " ==", " ", "0", " ||",
 " i", " ", " ", "5", " ==", " ", "0", " ||", " i", " ", " ", "7", " ==", " ", "0", ")", " {", "\n", "           ", " std", "::", "cout", " << i", " << \"", " \";", "\n", "       ", " }", "\n", "   ", " }", "\n", "}", "\n", "```", "", {
"role": "user", "content": "Now provide the explanation"}], "model": "codellama:34b-instruct", "stream": true}'
Response data:
{ '{"error":"json: cannot unmarshal string into Go struct field ChatRequest.messages of type api.Message"}' }
Response data:
{ "" }

The logs from the ollama server for those two requests are as follows:

[GIN] 2024/03/05 - 17:05:38 | 200 |  3.901055037s |     127.0.0.1 | POST     "/api/chat"
[GIN] 2024/03/05 - 17:05:59 | 400 |     213.546µs |     127.0.0.1 | POST     "/api/chat"

When I run that last curl command from my shell, I see the same cannot unmarshal... error as a response. I'm unfortunately not a JSON, Go, or Web expert, but I wonder if part of the message isn't getting escaped properly or something like that?
@abayomi185
Copy link
Contributor

I think my change may have caused this issue. I’ll make a PR to fix

@abayomi185 abayomi185 mentioned this issue Mar 5, 2024
Merged
@abayomi185
Copy link
Contributor

This should be fixed now

@jmdaly
Copy link
Contributor Author

jmdaly commented Mar 6, 2024

This is great, the fix works for me! Thanks so much for your quick work on this :)

@jmdaly jmdaly closed this as completed Mar 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@jmdaly @abayomi185