You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I noticed that an embedding vector from llm.el and Python using llama2 from ollama don't match. It is possible there is some small setting I have overlooked, and maybe even they don't use the same libraries, but I thought they would. They are least the same length. I only show the first and last elements of the embedding below for brevity.
elisp:
#+BEGIN_SRC emacs-lisp
(require 'llm)
(require 'llm-ollama)
(let* ((model (make-llm-ollama :embedding-model "llama2"))
(v (llm-embedding model "test")))
(list (length v) (elt v 0) (elt v 4095)))
#+END_SRC
#+RESULTS:
| 4096 | -0.9325962662696838 | -0.05561749264597893 |
According to a conversation I had with @s-kostyaev, there has been some issues with ollama embeddings, so he may have some ideas on what might be going on here.
Thanks @s-kostyaev for the update. I think this is likely to be upstream as well (there's nothing in llm that would cause this). However, it would be good to add some tests to make sure the embedding is giving sane results. That would be a good thing to add, so we can check this in the future. But for now I'm going to close this, since the issue is (as far as we can tell) not in the llm library. Thank you @jkitchin for your investigation, and for reporting this!
I noticed that an embedding vector from llm.el and Python using llama2 from ollama don't match. It is possible there is some small setting I have overlooked, and maybe even they don't use the same libraries, but I thought they would. They are least the same length. I only show the first and last elements of the embedding below for brevity.
elisp:
Python:
any tips on tracking down why the are different?
The text was updated successfully, but these errors were encountered: