Error using callbacks on RetrievalQAWithSourcesChain

[tomatefarcie123]

I get TypeError: 'tuple' object is not callable running this code. I guess it's because a run call doesn't work on a chain with multiple outputs,

How then can I use callbacks on that chain?

from flask import Flask, render_template
from flask_socketio import SocketIO
from initialize_llm_chain import build_chain
from langchain.callbacks.base import BaseCallbackHandler

Create a custom handler to stream llm response

class StreamingHandler(BaseCallbackHandler):
def on_llm_new_token(self, token: str, **kwargs) -> None:
socketio.emit('new_token', token)

def catch_all(*args, **kwargs):
    pass

on_agent_action = on_agent_finish = on_chain_end = on_chain_error = on_chain_start = on_llm_end = on_llm_error = on_llm_start = on_text = on_tool_end = on_tool_error = on_tool_start = catch_all

Build the langchain chain

qa_chain = build_chain()

Instantiate the handler

handler = StreamingHandler()

Initialize flask app

app = Flask(name)
socketio = SocketIO(app)

Define source route

@app.route('/')
def index():
return render_template('index.html')

Define socket query

@socketio.on('query', namespace='/results')
def handle_query(data):
results = qa_chain(data, callbacks=[handler])
('results', results["answer"])

if name == 'main':
socketio.run(app, host='localhost', port=9000, debug=True)

[tomatefarcie123]

Updated to langain 0.0.161 and made it work (but getting the actual stream to work was quite something.)

[Aspyryan]

@tomatefarcie123 can you provide code as to how you got this working?

[xerxes01]

Hey @tomatefarcie123 did you enable streaming via api for local huggingface LLMs?

[tomatefarcie123]

@xerxes01 I'm only using openai