[Bug] is_last_chunk never set #9085
Description
What happened?
I'm not sure if it's because of #8890, but with the built-in ReAct module and streamify, I'm never seeing is_last_chunk on StreamResponse's.
FYSA, I'm running against vLLM using GPT OSS 120b.
Steps to reproduce
Notebook content:
import logging
import dspy # pyright: ignore[reportMissingTypeStubs]
from dspy.streaming import streamify # pyright: ignore[reportMissingTypeStubs]
from dspy.streaming.messages import StreamResponse # pyright: ignore[reportMissingTypeStubs]
from dspy.streaming.streamify import StreamListener # pyright: ignore[reportMissingTypeStubs]
# Set up logging
logging.basicConfig(level=logging.INFO, format="%(message)s")
logger = logging.getLogger(__name__)
# Configure LLM (using your existing client)
lm = dspy.LM(
model="hosted_vllm/gpt-oss-120b",
api_key="None",
api_base="https://my-vllm-server/v1",
)
dspy.configure(lm=lm) # pyright: ignore[reportUnknownMemberType]
dspy.configure_cache(
enable_disk_cache=False,
enable_memory_cache=False,
)
# Use built-in ReAct
react = dspy.ReAct("question -> answer", max_iters=3, tools=[]) # pyright: ignore[reportArgumentType]
# Create stream listener
stream_listeners = [
StreamListener("next_thought", allow_reuse=True),
StreamListener("reasoning", allow_reuse=True),
StreamListener("answer", allow_reuse=True),
]
# Streamify the ReAct module
stream_react = streamify(react, stream_listeners=stream_listeners)
# Stream and log every chunk
chunk_count = 0
last_chunk_count = 0
async for chunk in stream_react(question="What is 2+2?"): # pyright: ignore[reportCallIssue, reportUnknownVariableType]
if isinstance(chunk, StreamResponse):
chunk_count += 1
if chunk.is_last_chunk:
last_chunk_count += 1
print(f"Total chunks: {chunk_count}")
print(f"Chunks with is_last_chunk=True: {last_chunk_count}")
if last_chunk_count == 0:
print("No last chunks found")
else:
print("Some last chunks found")Output in 3.0.4:
Total chunks: 51
Chunks with is_last_chunk=True: 0
No last chunks found
Output in 3.0.3:
Total chunks: 45
Chunks with is_last_chunk=True: 3
Some last chunks found
DSPy version
3.0.4