Skip to content

Add Llama support to Inference Plugin #130092

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 26 commits into
base: main
Choose a base branch
from

Conversation

Jan-Kazlouski-elastic
Copy link
Contributor

@Jan-Kazlouski-elastic Jan-Kazlouski-elastic commented Jun 26, 2025

Creation of new Llama inference provider integration allowing text_embedding, completion (both streaming and non-streaming) and chat_completion (only streaming) to be executed as part of inference API.

Changes were tested locally against next models:

  • all-MiniLM-L6-v2 (text embedding)
  • llama3.2:3b (completion & chat_completion)

For testing ollama service was used.

  • - Have you signed the contributor license agreement?
  • - Have you followed the contributor guidelines?
  • - If submitting code, have you built your formula locally prior to submission with gradle check?
  • - If submitting code, is your pull request against main? Unless there is a good reason otherwise, we prefer pull requests against main and will backport as needed.
  • - If submitting code, have you checked that your submission is for an OS and architecture that we support?
  • - If you are submitting this code for a class then read our policy for that.

@elasticsearchmachine elasticsearchmachine added v9.2.0 external-contributor Pull request authored by a developer outside the Elasticsearch team labels Jun 26, 2025
Jan-Kazlouski-elastic and others added 24 commits June 26, 2025 21:14
…r handling and improve error response parsing
…g-completion

# Conflicts:
#	server/src/main/java/org/elasticsearch/TransportVersions.java
@Jan-Kazlouski-elastic Jan-Kazlouski-elastic marked this pull request as ready for review July 4, 2025 14:02
@Jan-Kazlouski-elastic Jan-Kazlouski-elastic requested a review from a team as a code owner July 4, 2025 14:02
@elasticsearchmachine elasticsearchmachine added the needs:triage Requires assignment of a team area label label Jul 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
external-contributor Pull request authored by a developer outside the Elasticsearch team needs:triage Requires assignment of a team area label v9.2.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants