Error loading embedding model nomic-embed-text-v1.5.f16.gguf

[taksan]

I'm running the following code:

ModelParameters modelParams = new ModelParameters()
		.setModel("models/nomic-embed-text-v1.5.f16.gguf")
                .setBatchSize(8192)
                .setUbatchSize(8192);

LlamaModel model = new LlamaModel(modelParams)

But it's failing to load the model. I noticed the following issue in the above output, which seems to indicate the error:

/work/build/_deps/llama.cpp-src/src/llama.cpp:8666: GGML_ASSERT(strcmp(res->name, "result_output") == 0 && "missing result_output tensor") failed

A similar command line execution with latest llamacpp's llama-embedding works:

llama-embedding  -m nomic-embed-text-v1.5.f16.gguf -b 8192   -p 'search_query: What is TSNE?'

[kherud]

Hey, can you please try to set ModelParameters#enableEmbedding() and see if it works? It's a little bit tricky, but llama.cpp requires to set that separately (which of course automatically works if you use a dedicated binary llama-embedding).

[taksan]

Awesome, it worked! Thank you very much

[taksan]

Ok, I'll keep it open because the example in the README is outdate and doesn't reflect this

[bernardladenthin]

I noticed the repository hasn't seen recent activity, so I've taken the liberty of forking it and continuing development at: https://github.com/bernardladenthin/java-llama.cpp

See also #125

[bernardladenthin]

Resolved in the downstream fork at https://github.com/bernardladenthin/java-llama.cpp.
The original failure came from the bindings not forwarding --embedding at
all; calling ModelParameters.enableEmbedding() (sets ModelFlag.EMBEDDING,
ModelParameters.java:1040) plus setPoolingType(...) now initialises the
embedding pipeline correctly. Regression-tested by
LlamaEmbeddingsTest#testNomicEmbedLoads (PR #185, commit cba693c), which
reproduces the reporter's setBatchSize(8192).setUbatchSize(8192) config and
asserts the returned vector length is 768; CI downloads
nomic-embed-text-v1.5.f16.gguf via NOMIC_EMBED_MODEL_URL in
.github/workflows/publish.yml.