[LangChain] How to stream from LangChain hub?

[louis030195]

Hi, thanks for this amazing lib!

Trying to stream from a LangChain Hub prompt but cannot find an example (kinda overwhelmed by massive mess of LangChain docs :P)

Followed this which does not stream:

https://docs.smith.langchain.com/hub/dev-setup

import { NextRequest } from 'next/server'
import * as hub from 'langchain/hub' // Import LangChain Hub
import { ChatOpenAI } from 'langchain/chat_models/openai' // Import ChatOpenAI model
import { StreamingTextResponse } from 'ai'

import env from 'lib/env'

export const POST = () => {
  const { messages, id, myInput } = await req.json()

  if (!id) {
    return new Response('Invalid request', { status: 400 })
  }

  const prompt = await hub.pull(id)
  const model = new ChatOpenAI({
    streaming: true,
    openAIApiKey: env.OPENAI_API_KEY,
    modelName: 'gpt-4-turbo-preview',
  })

  const runnable = prompt.pipe(model)
  const stream = await runnable.stream({ myInput: myInput })
})

Client using normal useChat hook

Getting

[cause]: TypeError [ERR_INVALID_ARG_TYPE]: The "chunk" argument must be of type string or an instance of Buffer or Uint8Array. Received an instance of AIMessageChunk

Any help appreciated 🙏

[hinthornw]

I'm not a JS dev, but does this work? https://js.langchain.com/docs/modules/model_io/chat/streaming#using-stream

 });

    return new Response(stream, {
      headers: { "content-type": "text/plain; charset=utf-8" },
    });
  } catch (e: any) {
    return new Response(JSON.stringify({ error: e.message }), {
      status: 500,
      headers: {
        "content-type": "application/json",
      },
    });
  }
}

[lightify97]

@louis030195 Have you found a solution?