Skip to content
This repository was archived by the owner on Jul 4, 2025. It is now read-only.

Conversation

@hiro-v
Copy link
Contributor

@hiro-v hiro-v commented Jan 23, 2024

Fixes for #371

  • Load model
curl --location 'http://127.0.0.1:3928/inferences/llamacpp/loadmodel' \
--header 'Content-Type: application/json' \
--data '{
   "llama_model_path": "/Users/hiro/Downloads/ggml-model-q4_k.gguf",
   "ctx_len": 2048,
   "ngl": 100,
   "cont_batching": false,
   "embedding": true,
   "system_prompt": "",
   "user_prompt": "\n### Instruction:\n",
   "ai_prompt": "\n### Response:\n"
 }'
  • Embedding with input as string
curl --location 'http://localhost:3928/v1/embeddings' \
--header 'Content-Type: application/json' \
--header 'Accept: text/event-stream' \
--header 'Access-Control-Allow-Origin: *' \
--data '{
    "input": "Hello",
    "model": "embedding",
    "encoding_format": "float"
}'
  • Embedding with input as vector
curl --location 'http://localhost:3928/v1/embeddings' \
--header 'Content-Type: application/json' \
--header 'Accept: text/event-stream' \
--header 'Access-Control-Allow-Origin: *' \
--data '{
    "input": ["Hello", "Nam", "here"],
    "model": "embedding",
    "encoding_format": "float"
}'

@hiro-v hiro-v added the P1: important Important feature / fix label Jan 23, 2024
@hiro-v hiro-v requested a review from tikikun January 23, 2024 17:00
@hiro-v hiro-v self-assigned this Jan 23, 2024
@hiro-v hiro-v marked this pull request as draft January 23, 2024 17:00
@hiro-v hiro-v marked this pull request as ready for review January 24, 2024 01:21
Copy link
Contributor

@tikikun tikikun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@hiro-v hiro-v merged commit 92b5a5c into main Jan 24, 2024
@hiro-v hiro-v deleted the feat/batch_embedding branch January 24, 2024 02:20
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

P1: important Important feature / fix

Projects

No open projects
Archived in project

Development

Successfully merging this pull request may close these issues.

3 participants