Repository navigation
Feature Request: Allow API Token Authentication for chat_ollama() #501
Description
Activity
chat_ollama()uses the OpenAI API (with a few modifications). Can you try the following to see if it works (with appropriate values for all the args)?chat <- chat_openai( base_url = "https://path.ollama.internal/v1:11434", api_key = "your_key", model = "llama3" ) chat$chat("Make up a riddle.")
If that works, great! If not, can you include the output?
I'm getting this error. I updated the ollama prodiver in my fork, and that works though. https://github.com/scjohannes/ellmer/blob/main/R/provider-ollama.R
`Error in `req_perform_connection()`: ! HTTP 405 Method Not `Allowed. • { "detail": "Method Not Allowed" } Hide Traceback ▆ 1. └─chat$chat("Make up a riddle.") 2. └─coro::collect(...) 3. └─coro:::reduce_steps(x, steps, along_builder(list())) 4. └─coro:::reduce(x, reducer, .init = identity) 5. └─coro:::reduce_impl(.x, .f, ..., .init = .init) 6. └─coro:::iter_reduce_impl(.x, .f, ..., .left = .left) 7. ├─coro::is_exhausted(new <- .x()) 8. └─coro (local) .x() 9. └─base::evalq(...) 10. └─base::evalq(...) 11. └─base::evalq(...) 12. └─base::evalq(...) 13. ├─coro::is_exhausted(elt <- iterator()) 14. └─coro (local) iterator() 15. └─base::evalq(...) 16. └─base::evalq(...) 17. └─base::evalq(...) 18. └─base::evalq(...) 19. ├─coro::is_exhausted(elt <- iterator()) 20. └─coro (local) iterator() 21. └─base::evalq(...) 22. └─base::evalq(...) 23. ├─base::evalq(...) 24. │ └─base::evalq(...) 25. │ └─coro (local) user(...) 26. │ ├─.last_value <<- eval_bare(substitute(expr), user_env) 27. │ │ └─rlang::env_poke(env, lhs, value, inherit = TRUE, create = FALSE) 28. │ └─rlang::eval_bare(substitute(expr), user_env) 29. └─httr2::req_perform_connection(req) 30. └─httr2:::handle_resp(req, resp) 31. └─httr2:::resp_failure_cnd(req, resp, error_call = error_call) 32. ├─rlang::catch_cnd(...) 33. │ ├─rlang::eval_bare(...) 34. │ ├─base::tryCatch(...) 35. │ │ └─base (local) tryCatchList(expr, classes, parentenv, handlers) 36. │ │ └─base (local) tryCatchOne(expr, names, parentenv, handlers[[1L]]) 37. │ │ └─base (local) doTryCatch(return(expr), name, parentenv, handler) 38. │ └─base::force(expr) 39. └─rlang::abort(...)`It looks to me like you might need to fiddle with the
base_url. I was just guessing at what it might be for you and I realize now that the port would definitely not be at the end of the url, I should have written something like:base_url = "https://path.ollama.internal:11434/v1",Can you try a few more variations? Use whatever you were giving to
chat_ollama()but add/v1at the end.Reacted by Johannes SchwenkeAh my last edit didn't save, had already discovered that I had to add /v1. That did fix it! Thanks.
Thanks @scjohannes! #506 added
api_keytochat_ollama()so the upcoming ellmer release will include this feature. You can try it out withpak::pak("tidyverse/ellmer").
Is your feature request related to a problem? Please describe.
Currently,
chat_ollama()assumes that the Ollama instance is running locally and does not require authentication. However, in some IT-infrastructures (meaning where I am working) Ollama might run behind a reverse proxy that requires bearer token authentication. The current implementation does not provide a direct way to pass an API token for such scenarios.Describe the solution you'd like
The
chat_ollama()function should be updated to accept anapi_keyparameter.