community[minor]: Integration for `Friendli` LLM and `ChatFriendli` ChatModel. #5004

seuha516 · 2024-04-08T06:04:10Z

Description

Add Friendli integration for Friendli LLM and ChatFriendli chat model.
- Similar python langchain integration is merged in community: Integration for Friendli LLM and ChatFriendli ChatModel. langchain#17913.
Documentations corresponding to this change are added.

Twitter handle

https://twitter.com/friendliai

vercel · 2024-04-08T06:04:14Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
langchainjs-api-refs	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Apr 13, 2024 1:57am
langchainjs-docs	✅ Ready (Inspect)	Visit Preview		Apr 13, 2024 1:57am

dosubot · 2024-04-08T06:06:25Z

examples/src/models/chat/friendli.ts

@@ -0,0 +1,47 @@
+import { ChatFriendli } from "@langchain/community/chat_models/friendli";


Hey there! I've reviewed the code and flagged a change for you to review. The added lines in the diff explicitly access environment variables using process.env, so it's important to ensure that this is handled appropriately. Let me know if you need further assistance with this.

dosubot · 2024-04-08T06:06:27Z

examples/src/models/llm/friendli.ts

@@ -0,0 +1,42 @@
+import { Friendli } from "@langchain/community/llms/friendli";


Hey there! 👋 This is a friendly flag to bring attention to the addition of explicit environment variable access using process.env in the friendli.ts file. It's important for maintainers to review this change for potential security and configuration considerations. Thank you! 🚀

dosubot · 2024-04-08T06:06:28Z

langchain/package.json

@@ -3212,6 +3229,15 @@
      "import": "./chat_models/fireworks.js",


Hey there! 👋 I noticed the addition of a new entry for "./chat_models/friendli" in the package.json file. This change seems to be related to internal module dependencies, and I've flagged it for the maintainers to review. Keep up the great work! 🚀

dosubot · 2024-04-08T06:06:29Z

libs/langchain-community/src/chat_models/friendli.ts

@@ -0,0 +1,378 @@
+import { CallbackManagerForLLMRun } from "@langchain/core/callbacks/manager";


Hey team, I've reviewed the code and noticed that the new changes introduce a net-new HTTP request using the fetch API to the Friendli inference endpoint. This comment is to flag the change for maintainers to review. Let me know if you have any questions or need further clarification.

dosubot · 2024-04-08T06:06:30Z

libs/langchain-community/src/chat_models/friendli.ts

@@ -0,0 +1,378 @@
+import { CallbackManagerForLLMRun } from "@langchain/core/callbacks/manager";


Hey team, I've reviewed the code and flagged a specific change related to environment variable usage for your attention. The code appears to explicitly access environment variables using the getEnvironmentVariable function, so it's important to ensure that the handling of these variables aligns with best practices.

dosubot · 2024-04-08T06:06:31Z

libs/langchain-community/src/llms/friendli.ts

@@ -0,0 +1,236 @@
+import { CallbackManagerForLLMRun } from "@langchain/core/callbacks/manager";


Hey team, I've reviewed the code and noticed that the new changes introduce net-new HTTP requests using the fetch function. This comment is to flag the change for maintainers to review and ensure it aligns with the project's requirements and standards. Great work on the implementation!

dosubot · 2024-04-08T06:06:33Z

libs/langchain-community/src/llms/friendli.ts

@@ -0,0 +1,236 @@
+import { CallbackManagerForLLMRun } from "@langchain/core/callbacks/manager";


Hey there! I noticed that the recent code changes explicitly access environment variables using the getEnvironmentVariable function. I've flagged this for your review to ensure that the handling of environment variables aligns with best practices. Let me know if you need further assistance with this.

andrewnguonly

Nice job @seuha516. I have a few comments and questions before I approve. Please take a look. Thank you!

My suggestions aren't blocking, but just wanted to share them with you.

andrewnguonly · 2024-04-11T20:57:26Z

langchain/src/chat_models/friendli.ts

+/* #__PURE__ */ logVersion010MigrationWarning({
+  oldEntrypointName: "chat_models/friendli",
+});
+export * from "@langchain/community/chat_models/friendli";


@jacoblee93, do we need to include this for newly added models moving forward? Why not exclude?

Yes, we should remove as we are planning to completely remove @langchain/community from langchain in the near future.

I removed langchain/src/llms/friendli.ts and langchain/src/chat_models/friendli.ts files.

libs/langchain-community/src/load/import_constants.ts

libs/langchain-community/src/load/import_map.ts

libs/langchain-community/src/load/import_type.ts

libs/langchain-community/src/llms/friendli.ts

andrewnguonly · 2024-04-11T21:58:52Z

libs/langchain-community/src/llms/friendli.ts

+    for await (const chunk of stream) {
+      const parsedChunk = JSON.parse(chunk) as FriendliResponse;
+
+      if (parsedChunk.event !== "complete") {


Suggestion: The complete event can also be processed so that generation metadata is returned.

Example (something like this):

if (parsedChunk.event !== "complete") { ... } else { yield new GenerationChunk({ text: "", generationInfo: { promptTokens: parsedChunk.prompt_tokens, completionTokens: parsedChunk.completion_tokens, totalTokens: parsedChunk.total_tokens, }, }); }

I'm looking at this documentation: https://docs.friendli.ai/openapi/create-completions

The same can be implemented in ChatFriendli.

Updated Friendli to return generation metadata when event is completed.

libs/langchain-community/src/llms/friendli.ts

andrewnguonly · 2024-04-11T22:21:52Z

libs/langchain-community/src/chat_models/friendli.ts

+        method: "POST",
+        headers: {
+          "Content-Type": "application/json",
+          Accept: "application/json",


This needs to be Accept: "text/event-stream".

Fixed to Accept: "text/event-stream".

andrewnguonly · 2024-04-11T22:30:29Z

libs/langchain-community/src/chat_models/friendli.ts

+  // eslint-disable-next-line @typescript-eslint/no-explicit-any
+  delta: Record<string, any>
+) {
+  const { role } = delta;


This is probably a bug in the Friendli API docs, but the example response shows that role isn't always returned in the streaming response. If this is true, then this will cause an error.

I'm looking at this documentation: https://docs.friendli.ai/openapi/create-chat-completions#responses

I added defaultValue "assistant" for role.

jacoblee93 · 2024-04-11T23:18:50Z

libs/langchain-community/langchain.config.js

@@ -311,6 +314,7 @@ export const config = {
    "vectorstores/zep",
    "chat_models/bedrock",
    "chat_models/bedrock/web",
+    "chat_models/friendli",


You don't need to add your models here since they don't use any peer dependencies.

I removed "llms/friendli" and "chat_models/friendli" from requiresOptionalDependency.

seuha516 · 2024-04-12T07:59:02Z

Thank you for the detailed and kind comments! I made the requested changes.

andrewnguonly

Nice job with all the of updates! @seuha516

@jacoblee93, are any of the changes under /langchain needed? The original chat_model and llm classes are removed now. Besides that, everything looks good to me.

jacoblee93 · 2024-04-12T22:28:24Z

Ah yes can you revert the changes there @seuha516? You fork doesn't seem to allow maintainer edits. Otherwise LGTM!

seuha516 · 2024-04-13T01:27:00Z

@jacoblee93 I reverted changes under /langchain, please let me know if there is anything wrong. Thank you!

jacoblee93 · 2024-04-15T22:51:12Z

Thank you!

seuha516 added 4 commits April 8, 2024 14:55

add friendli llm and chat model

ea1de97

add docs and examples

c0ad6df

add tests

d9d24b9

add configurations

0ef9b5c

dosubot bot reviewed Apr 8, 2024

View reviewed changes

vercel bot deployed to Preview – langchainjs-api-refs April 8, 2024 06:10 View deployment

vercel bot deployed to Preview – langchainjs-docs April 8, 2024 06:10 View deployment

andrewnguonly self-requested a review April 11, 2024 17:00

andrewnguonly requested changes Apr 11, 2024

View reviewed changes

jacoblee93 reviewed Apr 11, 2024

View reviewed changes

seuha516 added 3 commits April 12, 2024 12:53

remove unnecessary files

d15084f

remove unnecessary config

49cb97b

remove unnecessary config

ac1d5bf

vercel bot deployed to Preview – langchainjs-api-refs April 12, 2024 04:10 View deployment

vercel bot deployed to Preview – langchainjs-docs April 12, 2024 04:11 View deployment

seuha516 added 2 commits April 12, 2024 13:11

fix chat model request headers

24d3587

fix llm request headers

c419108

vercel bot deployed to Preview – langchainjs-docs April 12, 2024 04:24 View deployment

vercel bot deployed to Preview – langchainjs-api-refs April 12, 2024 04:24 View deployment

update stream inference type checking logic

4195ace

vercel bot deployed to Preview – langchainjs-docs April 12, 2024 07:23 View deployment

seuha516 added 3 commits April 12, 2024 16:23

add streaming field

288b6f0

merge duplicated call api logics

671e4e3

add modelKwargs field

ecec8f0

vercel bot deployed to Preview – langchainjs-api-refs April 12, 2024 07:46 View deployment

vercel bot deployed to Preview – langchainjs-docs April 12, 2024 07:46 View deployment

parameterize for stream

96f3665

vercel bot deployed to Preview – langchainjs-docs April 12, 2024 07:59 View deployment

vercel bot deployed to Preview – langchainjs-api-refs April 12, 2024 07:59 View deployment

seuha516 requested review from andrewnguonly and jacoblee93 April 12, 2024 08:15

andrewnguonly approved these changes Apr 12, 2024

View reviewed changes

dosubot bot added the lgtm PRs that are ready to be merged as-is label Apr 12, 2024

seuha516 added 3 commits April 13, 2024 10:12

revert changes in langchain directory

25432c0

Merge remote-tracking branch 'upstream/main' into friendli-ai

b89c681

revert changes

70a4204

vercel bot deployed to Preview – langchainjs-api-refs April 13, 2024 01:26 View deployment

vercel bot deployed to Preview – langchainjs-docs April 13, 2024 01:27 View deployment

fix minor changes

be9abae

vercel bot deployed to Preview – langchainjs-api-refs April 13, 2024 01:47 View deployment

vercel bot deployed to Preview – langchainjs-docs April 13, 2024 01:47 View deployment

fix blacklisted-endpoints

1f77432

vercel bot deployed to Preview – langchainjs-docs April 13, 2024 01:57 View deployment

vercel bot deployed to Preview – langchainjs-api-refs April 13, 2024 01:57 View deployment

jacoblee93 merged commit 638793f into langchain-ai:main Apr 15, 2024
17 checks passed

naresh4dev mentioned this pull request May 30, 2024

[Snyk] Upgrade langchain from 0.0.153 to 0.1.36 naresh4dev/askDev-slackChatBot#1

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

community[minor]: Integration for `Friendli` LLM and `ChatFriendli` ChatModel. #5004

community[minor]: Integration for `Friendli` LLM and `ChatFriendli` ChatModel. #5004

seuha516 commented Apr 8, 2024

vercel bot commented Apr 8, 2024 •

edited

dosubot bot Apr 8, 2024

dosubot bot Apr 8, 2024

dosubot bot Apr 8, 2024

dosubot bot Apr 8, 2024

dosubot bot Apr 8, 2024

dosubot bot Apr 8, 2024

dosubot bot Apr 8, 2024

andrewnguonly left a comment •

edited

andrewnguonly Apr 11, 2024

jacoblee93 Apr 11, 2024

seuha516 Apr 12, 2024

andrewnguonly Apr 11, 2024

seuha516 Apr 12, 2024

andrewnguonly Apr 11, 2024

seuha516 Apr 12, 2024

andrewnguonly Apr 11, 2024

seuha516 Apr 12, 2024 •

edited

jacoblee93 Apr 11, 2024

seuha516 Apr 12, 2024

seuha516 commented Apr 12, 2024 •

edited

andrewnguonly left a comment

jacoblee93 commented Apr 12, 2024

seuha516 commented Apr 13, 2024

jacoblee93 commented Apr 15, 2024

		@@ -0,0 +1,47 @@
		import { ChatFriendli } from "@langchain/community/chat_models/friendli";

		@@ -0,0 +1,42 @@
		import { Friendli } from "@langchain/community/llms/friendli";

		@@ -3212,6 +3229,15 @@
		"import": "./chat_models/fireworks.js",

		@@ -0,0 +1,378 @@
		import { CallbackManagerForLLMRun } from "@langchain/core/callbacks/manager";

		@@ -0,0 +1,236 @@
		import { CallbackManagerForLLMRun } from "@langchain/core/callbacks/manager";

community[minor]: Integration for Friendli LLM and ChatFriendli ChatModel. #5004

community[minor]: Integration for Friendli LLM and ChatFriendli ChatModel. #5004

Conversation

seuha516 commented Apr 8, 2024

Description

Twitter handle

vercel bot commented Apr 8, 2024 • edited

dosubot bot Apr 8, 2024

Choose a reason for hiding this comment

dosubot bot Apr 8, 2024

Choose a reason for hiding this comment

dosubot bot Apr 8, 2024

Choose a reason for hiding this comment

dosubot bot Apr 8, 2024

Choose a reason for hiding this comment

dosubot bot Apr 8, 2024

Choose a reason for hiding this comment

dosubot bot Apr 8, 2024

Choose a reason for hiding this comment

dosubot bot Apr 8, 2024

Choose a reason for hiding this comment

andrewnguonly left a comment • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

seuha516 Apr 12, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

seuha516 commented Apr 12, 2024 • edited

andrewnguonly left a comment

Choose a reason for hiding this comment

jacoblee93 commented Apr 12, 2024

seuha516 commented Apr 13, 2024

jacoblee93 commented Apr 15, 2024

community[minor]: Integration for `Friendli` LLM and `ChatFriendli` ChatModel. #5004

community[minor]: Integration for `Friendli` LLM and `ChatFriendli` ChatModel. #5004

vercel bot commented Apr 8, 2024 •

edited

andrewnguonly left a comment •

edited

seuha516 Apr 12, 2024 •

edited

seuha516 commented Apr 12, 2024 •

edited