How to convert into AppGenerateService.generate() --> response

[Rameshkumardas]

Self Checks

• This is only for bug report, if you would like to ask a question, please head to Discussions.
• I have searched for existing issues search for existing issues, including closed ones.
• I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• Please do not modify this template :) and fill in all the required fields.

Dify version

latest

Cloud or Self Hosted

Self Hosted (Docker), Self Hosted (Source)

Steps to reproduce

I have Streaming responce now i want to convert into 'AppGenerateService.generate() --> response' generator to that i can pass to return helper.compact_generate_response(response)

Streaming Mode ( Basic Assistant )

  data: {"event": "message", "message_id": "5ad4cb98-f0c7-4085-b384-88c403be6290", "conversation_id": "45701982-8118-4bc5-8e9b-64562b4555f2", "answer": " I", "created_at": 1679586595}
  data: {"event": "message", "message_id": "5ad4cb98-f0c7-4085-b384-88c403be6290", "conversation_id": "45701982-8118-4bc5-8e9b-64562b4555f2", "answer": "'m", "created_at": 1679586595}
  data: {"event": "message", "message_id": "5ad4cb98-f0c7-4085-b384-88c403be6290", "conversation_id": "45701982-8118-4bc5-8e9b-64562b4555f2", "answer": " glad", "created_at": 1679586595}
  data: {"event": "message", "message_id": "5ad4cb98-f0c7-4085-b384-88c403be6290", "conversation_id": "45701982-8118-4bc5-8e9b-64562b4555f2", "answer": " to", "created_at": 1679586595}
  data: {"event": "message", "message_id": : "5ad4cb98-f0c7-4085-b384-88c403be6290", "conversation_id": "45701982-8118-4bc5-8e9b-64562b4555f2", "answer": " meet", "created_at": 1679586595}
  data: {"event": "message", "message_id": : "5ad4cb98-f0c7-4085-b384-88c403be6290", "conversation_id": "45701982-8118-4bc5-8e9b-64562b4555f2", "answer": " you", "created_at": 1679586595}
  data: {"event": "message_end", "id": "5e52ce04-874b-4d27-9045-b3bc80def685", "conversation_id": "45701982-8118-4bc5-8e9b-64562b4555f2", "metadata": {"usage": {"prompt_tokens": 1033, "prompt_unit_price": "0.001", "prompt_price_unit": "0.001", "prompt_price": "0.0010330", "completion_tokens": 135, "completion_unit_price": "0.002", "completion_price_unit": "0.001", "completion_price": "0.0002700", "total_tokens": 1168, "total_price": "0.0013030", "currency": "USD", "latency": 1.381760165997548, "retriever_resources": [{"position": 1, "dataset_id": "101b4c97-fc2e-463c-90b1-5261a4cdcafb", "dataset_name": "iPhone", "document_id": "8dd1ad74-0b5f-4175-b735-7d98bbbb4e00", "document_name": "iPhone List", "segment_id": "ed599c7f-2766-4294-9d1d-e5235a61270a", "score": 0.98457545, "content": "\"Model\",\"Release Date\",\"Display Size\",\"Resolution\",\"Processor\",\"RAM\",\"Storage\",\"Camera\",\"Battery\",\"Operating System\"\n\"iPhone 13 Pro Max\",\"September 24, 2021\",\"6.7 inch\",\"1284 x 2778\",\"Hexa-core (2x3.23 GHz Avalanche + 4x1.82 GHz Blizzard)\",\"6 GB\",\"128, 256, 512 GB, 1TB\",\"12 MP\",\"4352 mAh\",\"iOS 15\""}]}}}

How can i make generator for same like

response = AppGenerateService.generate(
                app_model=app_model,
                user=end_user,
                args=args,
                invoke_from=InvokeFrom.WEB_APP,
                streaming=streaming
            )

#Streaming should be similar to response generator

✔️ Expected Behavior

No response

❌ Actual Behavior

No response

[dosubot]

To convert your streaming responses into a format that AppGenerateService.generate() can process, you'll need to implement a conversion logic that transforms the streaming data into a generator of JSON strings. This involves creating a method that iterates over each piece of the streaming response, converting each part into the appropriate JSON format, and then yielding these JSON strings for further processing.

Here's a simplified example to guide you:

def stream_response_converter(streaming_response):
    for res in streaming_response:
        # Assuming res is a dict representing each message or event
        if res['event'] == 'message':
            yield json.dumps({
                'event': 'message',
                'message_id': res['message_id'],
                'conversation_id': res['conversation_id'],
                'answer': res['answer'],
                'created_at': res['created_at']
            })
        elif res['event'] == 'message_end':
            yield json.dumps({
                'event': 'message_end',
                'id': res['id'],
                'conversation_id': res['conversation_id'],
                'metadata': res['metadata']
            })
        # Add more conditions as needed based on your streaming response structure

To use this converter with AppGenerateService.generate(), you would do something like this:

streaming_response = stream_response_converter(your_streaming_data)
response = AppGenerateService.generate(
    app_model=app_model,
    user=end_user,
    args=streaming_response,
    invoke_from=InvokeFrom.WEB_APP,
    streaming=True
)
final_response = helper.compact_generate_response(response)

Replace your_streaming_data with the actual streaming data you're working with. This example assumes that your streaming data is structured in a way that the stream_response_converter function expects. If your data structure is different, you'll need to adjust the conversion logic accordingly.

This approach allows you to convert streaming responses into a generator of JSON strings, which can then be processed by AppGenerateService.generate() and ultimately compacted into a final response using helper.compact_generate_response(response).

Details

Sources

• api/core/app/apps/base_app_generate_response_converter.py
• api/core/app/apps/advanced_chat/generate_response_converter.py
• api/core/app/apps/agent_chat/generate_response_converter.py
• api/core/app/apps/completion/generate_response_converter.py
• api/core/app/apps/chat/generate_response_converter.py
• api/services/app_generate_service.py
• api/libs/helper.py

About Dosu
This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

To continue the conversation, mention @dosu.