feature/SupportZepMemoryAndAutoSummary #284

wangerzi · 2023-06-10T04:29:44Z

Support https://docs.getzep.com/sdk/langchain/ in Flowise.

Some limits:

the unique session id should be changed manually in every conversion, because zep server doesn't support session id reuse now

wangerzi · 2023-06-10T04:30:36Z

zep-test Chatflow.zip
a simple chatflow demo.

HenryHengZJ · 2023-06-10T12:53:40Z

thanks @wangerzi for the Zep addition!
can you help me understand the session id? what do you mean by should be changed manually in every conversion?

wangerzi · 2023-06-10T15:27:51Z

thanks @wangerzi for the Zep addition! can you help me understand the session id? what do you mean by should be changed manually in every conversion?

@HenryHengZJ Flowise is an interesting project, I'm very happy to make some contributions.😀

The session id in zep just like a required unique identifier of a conversion, zep server use session id to find memory, make summary, etc...

But when we want to create a new conversion in flowise, we must change the session id (or automatic generate a uuid for new conversion), because we can't clear zep chat history in this session by api.

Talk is cheap, I'm already prepared a demo to explain it:

Execution Prepare:

run a zep server locally, refer: https://docs.getzep.com/deployment/quickstart/
manually install langchain and @getzep/zep-js

// import { ZepMemory } from 'langchain/memory/zep'
// import { Memory, Message } from '@getzep/zep-js'
const { ZepMemory } = require('langchain/memory/zep')
const { Memory, Message } = require('@getzep/zep-js')

const sessionId = 'test-3'
const zepURL = 'http://localhost:8000'

const memory = new ZepMemory({
    sessionId,
    memoryKey: 'chat_history',
    returnMessages: true,
    baseURL: zepURL
})

async function init() {
    const history = [
        {
            role: 'human',
            content: 'Who was Octavia Butler?'
        },
        {
            role: 'ai',
            content: 'Octavia Estelle Butler (June 22, 1947 – February 24, 2006) was an American' + ' science fiction author.'
        }
    ]

    const messages = history.map(({ role, content }) => new Message({ role, content }))
    const zepMemory = new Memory({ messages })

    await memory.zepClient.addMemory(sessionId, zepMemory)
}

async function testSearch() {
    await init()
    const data = await memory.loadMemoryVariables({
        lastN: 10
    })
    console.log('search data:', data)
}

async function testClear() {
    await memory.clear()
}

async function testCase1() {
    console.log('first search')
    await testSearch()
    await testClear()
    console.log('already cleared')
    await testSearch()
}

testCase1()

Execution Result told us if we can't use official api memory.clear() to reuse a same session id:

$ node test-zep.js
first search
search data: {
  chat_history: [
    ChatMessage {
      text: 'Who was Octavia Butler?',
      name: undefined,
      role: 'human'
    },
    ChatMessage {
      text: 'Octavia Estelle Butler (June 22, 1947 – February 24, 2006) was an American science fiction author.',
      name: undefined,
      role: 'ai'
    }
  ]
}
already cleared
D:\phpStudy\WWW\github\Flowise\node_modules\@getzep\zep-js\dist\zep-client.js:143
                    throw new exceptions_1.UnexpectedResponseError(`addMemoryAsync got an Unexpected status code: ${error.response.status}`);
                          ^

UnexpectedResponseError: addMemoryAsync got an Unexpected status code: 400
    at ZepClient.<anonymous> (D:\phpStudy\WWW\github\Flowise\node_modules\@getzep\zep-js\dist\zep-client.js:143:27)
    at Generator.throw (<anonymous>)
    at rejected (D:\phpStudy\WWW\github\Flowise\node_modules\@getzep\zep-js\dist\zep-client.js:29:65)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
  responseData: undefined
}

Node.js v18.16.0

References:

Langchain source code: https://github.com/hwchase17/langchainjs/blob/main/langchain/src/memory/zep.ts#L143-L152
Zep document: https://www.getzep.com/zep-langchainjs-memory/

HenryHengZJ · 2023-06-10T16:24:02Z

@wangerzi oh okay in that case, can we just use timestamp string as session id? that way we can save user from inputting/changing session id everytime

wangerzi · 2023-06-10T16:41:25Z

@HenryHengZJ Sounds like a great idea to solve this problem👏, I'll try to use the conversion start time as zep session after 8 hours (after wake up🌞)

wangerzi · 2023-06-11T05:29:28Z

@HenryHengZJ According to the current data structure, I think the first chat message id is a better choice for chat_id

For generality, I found the first chat message in backend, and pass it by options in INode definition init?(nodeData: INodeData, input: string, options?: ICommonObject): Promise<any>

This is the minimum modification in my mind, but I'm not sure if you have better solution for this.

Key Code:

HenryHengZJ · 2023-06-11T11:59:08Z

packages/server/src/ChildProcess.ts

                depthQueue,
                componentNodes,
                incomingInput.question,
+                '',


can we also pass in the chatid from here - https://github.com/FlowiseAI/Flowise/blob/main/packages/server/src/index.ts#L445 ?

It's missing, I have forgot the chatflow variable contains a correct chatflow.id we can use, I'll fix it later!👀

HenryHengZJ · 2023-06-11T12:00:27Z

packages/ui/src/views/chatmessage/ChatMessage.js

        setLoading(true)
        setMessages((prevMessages) => [...prevMessages, { message: userInput, type: 'userMessage' }])
-        addChatMessage(userInput, 'userMessage')
+        // waiting for first chatmessage uploaded, the first chatmessage id will be chatId for every components


the code await addChatMessage(userInput, 'userMessage') looks the same before and after changes, is the comment redundant?

Hi, maybe I should say goodmorning to you! Considering that when the chat message is sent for the first time, if the POST /chatmessage/{id} request and the POST /internal-prediction/${id} request are executed concurrently, the first message id ORM query will fail in the prediction request.

I tried to remove await and make some test, as my last comment, it will trigger error from api /internal-prediction/{id} sometimes, because chatmessage can't be queried in database

packages/components/nodes/memory/ZepMemory/ZepMemory.ts

HenryHengZJ · 2023-06-11T12:04:20Z

@HenryHengZJ According to the current data structure, I think the first chat message id is a better choice for chat_id

For generality, I found the first chat message in backend, and pass it by options in INode definition init?(nodeData: INodeData, input: string, options?: ICommonObject): Promise<any>

This is the minimum modification in my mind, but I'm not sure if you have better solution for this.

Key Code:

https://github.com/FlowiseAI/Flowise/pull/284/files#diff-8e6dfbbfb522575fe7c568c2de518bf9de351de83c9dbb48eaa865cd54450eb3R509-R518

https://github.com/FlowiseAI/Flowise/pull/284/files#diff-473a2e7fb254080e3d1f8e339d5bbf13cb756874dd9aae2b3def6d49a8dfb41dR218

i love the idea of using first chat id, that way when user clear the conversation, the session id will also be refreshed

wangerzi · 2023-06-11T15:36:50Z

@HenryHengZJ I'm already update the code based on our conversation, and I create a new function getChatId at index.ts, it's easy to use but I'm not sure if this is the best place.

HenryHengZJ · 2023-06-12T09:49:49Z

packages/components/nodes/memory/ZepMemory/ZepMemory.ts

+                label: 'Session Id',
+                name: 'sessionId',
+                type: 'string',
+                placeholder: 'if empty, chatId will be used automatically',


we can use description for the message

HenryHengZJ · 2023-06-12T09:52:37Z

packages/server/src/ChildProcess.ts

            const { startingNodeIds, depthQueue } = getStartingNodes(nonDirectedGraph, endingNodeId)

            /*** BFS to traverse from Starting Nodes to Ending Node ***/
+            const chatId = await getChatId(chatflow.id)


this will not work because child processes dont have access to DataSource. You'll have to get the chatid in startChildProcess() and pass it in the IRunChatflowMessageValue - https://github.com/FlowiseAI/Flowise/blob/main/packages/server/src/index.ts#L449. Then retrieve it here - https://github.com/FlowiseAI/Flowise/blob/main/packages/server/src/ChildProcess.ts#L26

packages/server/src/index.ts

wangerzi · 2023-06-12T14:03:43Z

@HenryHengZJ Thank you very much for your patience and guidance, I have changed my code by your comment.😀

em, but I just have a little confusion, I don't known how to test ChildProcess.ts in the UI

HenryHengZJ · 2023-06-12T15:26:46Z

@HenryHengZJ Thank you very much for your patience and guidance, I have changed my code by your comment.😀

em, but I just have a little confusion, I don't known how to test ChildProcess.ts in the UI

In the .env file in packages/server, you can specify the EXECUTION_MODE: https://github.com/FlowiseAI/Flowise/blob/main/packages/server/.env.example#L4

wangerzi · 2023-06-12T15:49:11Z

@HenryHengZJ Thank you very much for your patience and guidance, I have changed my code by your comment.😀
em, but I just have a little confusion, I don't known how to test ChildProcess.ts in the UI

In the .env file in packages/server, you can specify the EXECUTION_MODE: https://github.com/FlowiseAI/Flowise/blob/main/packages/server/.env.example#L4

Thanks for answering, I've updated some code to remove a useless ORM query and tested the ChildProcess.ts by modify .env

HenryHengZJ · 2023-06-12T18:27:09Z

thanks @wangerzi !! tested and it works like a charm!

really appreciate the ground work of session id using chat id, it makes it easy for other memory implementation thats using session id as well

feat: support zep memory

011f6af

feat: unique chat id

023967e

HenryHengZJ reviewed Jun 11, 2023

View reviewed changes

packages/components/nodes/memory/ZepMemory/ZepMemory.ts Show resolved Hide resolved

fix: zep memory

fd9d6fc

HenryHengZJ reviewed Jun 12, 2023

View reviewed changes

packages/server/src/index.ts Show resolved Hide resolved

fix: childprocess chatId.

fe6737a

fix: remove useless query

a5c408d

HenryHengZJ merged commit 9809919 into FlowiseAI:main Jun 12, 2023

Uh oh!

feature/SupportZepMemoryAndAutoSummary #284

feature/SupportZepMemoryAndAutoSummary #284

Uh oh!

Conversation

wangerzi commented Jun 10, 2023

Uh oh!

wangerzi commented Jun 10, 2023

Uh oh!

HenryHengZJ commented Jun 10, 2023

Uh oh!

wangerzi commented Jun 10, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

HenryHengZJ commented Jun 10, 2023

Uh oh!

wangerzi commented Jun 10, 2023

Uh oh!

wangerzi commented Jun 11, 2023

Uh oh!

HenryHengZJ Jun 11, 2023

Choose a reason for hiding this comment

Uh oh!

wangerzi Jun 11, 2023

Choose a reason for hiding this comment

Uh oh!

HenryHengZJ Jun 11, 2023

Choose a reason for hiding this comment

Uh oh!

wangerzi Jun 11, 2023

Choose a reason for hiding this comment

Uh oh!

wangerzi Jun 11, 2023

Choose a reason for hiding this comment

Uh oh!

Uh oh!

HenryHengZJ commented Jun 11, 2023

Uh oh!

wangerzi commented Jun 11, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

HenryHengZJ Jun 12, 2023

Choose a reason for hiding this comment

Uh oh!

HenryHengZJ Jun 12, 2023

Choose a reason for hiding this comment

Uh oh!

Uh oh!

wangerzi commented Jun 12, 2023

Uh oh!

HenryHengZJ commented Jun 12, 2023

Uh oh!

wangerzi commented Jun 12, 2023

Uh oh!

HenryHengZJ commented Jun 12, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

wangerzi commented Jun 10, 2023 •

edited

Loading

wangerzi commented Jun 11, 2023 •

edited

Loading