Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pass the lora dir into start req #261

Merged
merged 5 commits into from
Apr 13, 2024
Merged

Conversation

rusenask
Copy link
Contributor

@rusenask rusenask commented Apr 12, 2024

Enables the new sessions API to specify lora dir. Needs to be matching the lora dir within users account.

Example article https://www.theguardian.com/uk-news/2024/apr/13/squatters-york-albany-hotel-pub-gordon-ramsay-regents-park-london

Usage:

curl --request POST \
  --url http://localhost/api/v1/sessions/chat \
  --header 'Authorization: Bearer hl-ETHduXUj6-kzjUBNpXtkejxvU25m-R6ciuTxHnZqvEU=' \
  --header 'Content-Type: application/json' \
  --data '{
  "session_id": "", "lora_dir": "dev/users/d5680e7e-73a2-4a92-adc7-2d8d0c56b43a/sessions/c9b8d9b8-66e6-4537-9849-50d769c8e090/lora/8f223431-e2fe-40b6-a4f0-dceb0972f35d",
  "system": "You are an intelligent chatbot named Helix that has been fine-tuned on document(s) f3a0a7dd7d in document group c9b8d9b866.\nThe document group contains 1 document(s).\nThe user will ask you questions about these documents: you must ONLY answer with context from the documents listed. Do NOT refer to background knowledge.",
  "messages": [
    {
      "role": "user",
      "content": { "content_type": "text", "parts": ["who are living in this restaurant now?"] }
    }
  ]
}'
{"created":1713021064,"object":"chat.completion","id":"ses_01hvc05xtbmf58s2kc6wbjh8yz","model":"mistralai/Mistral-7B-Instruct-v0.1","choices":[{"index":0,"finish_reason":"stop","message":{"role":"assistant","content":"\n[DOC_ID:f3a0a7dd7d] [DOC_GROUP:c9b8d9b866]\n\nSquatters have taken over a pub in London leased by Gordon Ramsay. The group locked themselves inside the Grade II-listed York and Albany hotel and gastropub, next to Regent’s Park, boarding up the windows and putting up a notice defending their takeover. Ramsay called the police on Wednesday but was unable to have the people removed. The police consider it a civil matter. The property was previously owned by a film director and leased to Ramsay for 25 years. Ramsay tried to free himself from the lease in 2015 but failed. The venue went on sale at the end of last year with a guide price of £13m.\n "}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}

@rusenask rusenask merged commit 50a0dc3 into main Apr 13, 2024
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant