-
Notifications
You must be signed in to change notification settings - Fork 4.5k
Unable to Test Session "with gap duration" Windowing #18630
Copy link
Copy link
Open
Description
Trying to write a unit test to verify the windowing behavior for session with gap duration, but my assumption is that there is a merging of IntervalWindows that normally happens that is not happening for my test pipeline, because my actual pipeline seems to work as expected, but my test fails.
I have been using these resources:
http://www.waitingforcode.com/apache-beam/windows-apache-beam/read
https://beam.apache.org/blog/2016/10/20/test-stream.html
Here is an example of the issue:
String guid = "user1";
UniqueUserKey uniqueUser = makeUniqueUserKey(guid);
// the first value to
makePageView results in the timestamp: new Instant(value)
TimestampedValue<KV<UniqueUserKey, PageLoadEvent>>
homepage = makePageView(1, "HomePage", "homepage", guid, uniqueUser);
TimestampedValue<KV<UniqueUserKey,
PageLoadEvent>> productDetails1 = makePageView(2, "Product Details", "product_details", guid, uniqueUser);
TestStream<KV<UniqueUserKey,
PageLoadEvent>> testStream = TestStream.create(KvCoder.of(ProtoCoder.of(UniqueUserKey.class), ProtoCoder.of(PageLoadEvent.class)))
.addElements(homepage)
.addElements(productDetails1)
.advanceWatermarkTo(new Instant(8))
.advanceWatermarkToInfinity();
IntervalWindow window1 = new IntervalWindow(new Instant(1), new
Instant(3));
// This fails because productDetails1 is not in the window
PAssert.that(firstTransform).inFinalPane(window1).containsInAnyOrder(
homepage.getValue(),
productDetails1.getValue());
pipeline.run().waitUntilFinish();
Imported from Jira BEAM-3755. Original Jira may contain additional context.
Reported by: jbfbell.
Reactions are currently unavailable