Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

community: Bedrock add support for mistral models #18756

Merged
merged 16 commits into from
Mar 9, 2024
Merged

community: Bedrock add support for mistral models #18756

merged 16 commits into from
Mar 9, 2024

Conversation

AnisZakari
Copy link
Contributor

Description*: My previous PR was mistakenly closed, so I am reopening this one. Context: AWS released two Mistral models on Bedrock last Friday (March 1, 2024). This PR includes some code adjustments to ensure their compatibility with the Bedrock class.

@dosubot dosubot bot added the size:S This PR changes 10-29 lines, ignoring generated files. label Mar 7, 2024
@vercel Vercel
Copy link

vercel bot commented Mar 7, 2024 •
edited

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment
Name Status Preview Comments Updated (UTC)
langchain ⬜️ Ignored (Inspect) Visit Preview Mar 9, 2024 1:18am

@dosubot dosubot bot added Ɑ: models Related to LLMs or chat model modules 🔌: aws Primarily related to Amazon Web Services (AWS) integrations 🤖:improvement Medium size change to existing code to handle new use-cases labels Mar 7, 2024
@AnisZakari
Copy link
Contributor Author

@3coins @efriis

@3coins
Copy link
Contributor

3coins commented Mar 7, 2024

@AnisZakari
Thanks for working on these changes. Similar comment as PR #18687 about support for BedrockChat class.

@dosubot dosubot bot added size:M This PR changes 30-99 lines, ignoring generated files. and removed size:S This PR changes 10-29 lines, ignoring generated files. labels Mar 7, 2024
@AnisZakari
Copy link
Contributor Author

AnisZakari commented Mar 7, 2024 •
edited

@AnisZakari Thanks for working on these changes. Similar comment as PR #18687 about support for BedrockChat class.

Thank you @3coins for your comment, I have just pushed some code to adapt mistral to the BedrockChat class

@AnisZakari AnisZakari mentioned this pull request Mar 8, 2024
Open
5 tasks
@AnisZakari AnisZakari changed the title mistral: Bedrock add support for mistral models community: Bedrock add support for mistral models Mar 8, 2024
3coins
3coins suggested changes Mar 8, 2024
View reviewed changes
Copy link
Contributor

@3coins 3coins left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@AnisZakari
See my comment about moving code out of mistral.py, plz delete that file. Also, it seems like the async streaming support is missing for mistral here, can you bring in code from PR #18687 and see if that works.

libs/community/langchain_community/chat_models/mistral.py Outdated Show resolved Hide resolved
@3coins 3coins mentioned this pull request Mar 8, 2024
Closed
3coins
3coins approved these changes Mar 8, 2024
View reviewed changes
Copy link
Contributor

@3coins 3coins left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@AnisZakari
Awesome job! Thanks for working on this.

@3coins
Copy link
Contributor

3coins commented Mar 8, 2024

@efriis
Can you please help merge this change. Thanks.

@dosubot dosubot bot added the lgtm PR looks good. Use to confirm that a PR is ready for merging. label Mar 9, 2024
@efriis efriis enabled auto-merge (squash) March 9, 2024 01:19
@efriis efriis merged commit 37e89ba into langchain-ai:master Mar 9, 2024
59 checks passed
@str-aws str-aws mentioned this pull request Mar 16, 2024
Closed
5 tasks
@Maxinho96
Copy link

Hi @AnisZakari, by looking at the code looks like the output format will be the following:

<<SYS>> System message <</SYS>>
[INST] Human message [/INST]
AI message

But to my understanding, the format should be the following instead:

<s>[INST] System/Human message [/INST] AI message</s>[INST] Human message [/INST]

The <<SYS>> tag is not used by Mistral, just [INST] for both System and Human messages. And </s> closing tag should be used after the AI message. Also, looks like newlines are not needed.

References:

bechbd pushed a commit to bechbd/langchain that referenced this pull request Mar 29, 2024
@AnisZakari @efriis
community[patch]: Bedrock add support for mistral models (langchain-a…
93025d1 
…i#18756)

*Description**: My previous
[PR](langchain-ai#18521) was
mistakenly closed, so I am reopening this one. Context: AWS released two
Mistral models on Bedrock last Friday (March 1, 2024). This PR includes
some code adjustments to ensure their compatibility with the Bedrock
class.

---------

Co-authored-by: Anis ZAKARI <anis.zakari@hymaia.com>
Co-authored-by: Erick Friis <erick@langchain.dev>
gkorland pushed a commit to FalkorDB/langchain that referenced this pull request Mar 30, 2024
@AnisZakari @efriis @gkorland
community[patch]: Bedrock add support for mistral models (langchain-a…
2581981 
…i#18756)

*Description**: My previous
[PR](langchain-ai#18521) was
mistakenly closed, so I am reopening this one. Context: AWS released two
Mistral models on Bedrock last Friday (March 1, 2024). This PR includes
some code adjustments to ensure their compatibility with the Bedrock
class.

---------

Co-authored-by: Anis ZAKARI <anis.zakari@hymaia.com>
Co-authored-by: Erick Friis <erick@langchain.dev>
hinthornw pushed a commit that referenced this pull request Apr 26, 2024
@AnisZakari @efriis @hinthornw
community[patch]: Bedrock add support for mistral models (#18756)
6ebdbb9 
*Description**: My previous
[PR](#18521) was
mistakenly closed, so I am reopening this one. Context: AWS released two
Mistral models on Bedrock last Friday (March 1, 2024). This PR includes
some code adjustments to ensure their compatibility with the Bedrock
class.

---------

Co-authored-by: Anis ZAKARI <anis.zakari@hymaia.com>
Co-authored-by: Erick Friis <erick@langchain.dev>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@3coins 3coins 3coins approved these changes

@efriis efriis efriis approved these changes

Assignees
No one assigned
Labels
🔌: aws Primarily related to Amazon Web Services (AWS) integrations 🤖:improvement Medium size change to existing code to handle new use-cases lgtm PR looks good. Use to confirm that a PR is ready for merging. Ɑ: models Related to LLMs or chat model modules size:M This PR changes 30-99 lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@AnisZakari @3coins @Maxinho96 @efriis