Skip to content

Fix bug in EOS for data streams#357

Merged
MaxHeimbrock merged 4 commits into
mainfrom
max/datastream-incemental-eos-drain
Jul 20, 2026
Merged

Fix bug in EOS for data streams#357
MaxHeimbrock merged 4 commits into
mainfrom
max/datastream-incemental-eos-drain

Conversation

@MaxHeimbrock

@MaxHeimbrock MaxHeimbrock commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Background

When a burst of packages plus the EOS trailer arrived at the same time, the incremental reading of the stream was broken. If the stream was marked EOS, but the queue still had chunks, they were ignored.

This PR adds the logic to fix the bug plus tests.

Changes

What did you do?

  • Fix bug when EOS message could mark the stream as ended, even when queue was non-empty
  • Add tests to guard the expected behaviour
  • Simplify agent sample code since workaround is not needed anymore

@MaxHeimbrock
MaxHeimbrock changed the base branch from main to max/final-docs-update-for-2.0.0 July 20, 2026 10:35
@MaxHeimbrock MaxHeimbrock changed the title Datastream incemental eos drain Fix bug in EOS for data streams Jul 20, 2026
@MaxHeimbrock
MaxHeimbrock force-pushed the max/datastream-incemental-eos-drain branch from 1c980d4 to 72f668d Compare July 20, 2026 12:14
@MaxHeimbrock
MaxHeimbrock force-pushed the max/datastream-incemental-eos-drain branch from 72f668d to b12adc9 Compare July 20, 2026 12:15
@MaxHeimbrock
MaxHeimbrock changed the base branch from max/final-docs-update-for-2.0.0 to main July 20, 2026 12:16
@MaxHeimbrock
MaxHeimbrock marked this pull request as ready for review July 20, 2026 12:16
reader.PushChunk("C");
reader.PushEos();

CollectionAssert.AreEqual(new[] { "A", "B", "C" }, Drain(reader));

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If all chunks arrived in the same frame together with EOS, before the fix the incremental reader would receive EOS and just be marked as done, disregarding the chunks in the queue.

Comment thread Runtime/Scripts/DataStreams/DataStream.cs Outdated
@MaxHeimbrock
MaxHeimbrock force-pushed the max/datastream-incemental-eos-drain branch from 75d1095 to 2c6e789 Compare July 20, 2026 14:15

@xianshijing-lk xianshijing-lk left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@MaxHeimbrock
MaxHeimbrock merged commit e3cbfa9 into main Jul 20, 2026
11 checks passed
@MaxHeimbrock
MaxHeimbrock deleted the max/datastream-incemental-eos-drain branch July 20, 2026 15:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants