Skip to content

Unable to Test Session "with gap duration" Windowing #18630

@kennknowles

Description

@kennknowles

Trying to write a unit test to verify the windowing behavior for session with gap duration, but my assumption is that there is a merging of IntervalWindows that normally happens that is not happening for my test pipeline, because my actual pipeline seems to work as expected, but my test fails.  

I have been using these resources:

http://www.waitingforcode.com/apache-beam/windows-apache-beam/read

https://beam.apache.org/blog/2016/10/20/test-stream.html

 

Here is an example of the issue:


String guid = "user1";
UniqueUserKey uniqueUser = makeUniqueUserKey(guid);
// the first value to
makePageView results in the timestamp: new Instant(value)
TimestampedValue<KV<UniqueUserKey, PageLoadEvent>>
homepage = makePageView(1, "HomePage", "homepage", guid, uniqueUser);
TimestampedValue<KV<UniqueUserKey,
PageLoadEvent>> productDetails1 = makePageView(2, "Product Details", "product_details", guid, uniqueUser);

TestStream<KV<UniqueUserKey,
PageLoadEvent>> testStream = TestStream.create(KvCoder.of(ProtoCoder.of(UniqueUserKey.class), ProtoCoder.of(PageLoadEvent.class)))

   .addElements(homepage)
    .addElements(productDetails1)
    .advanceWatermarkTo(new Instant(8))

   .advanceWatermarkToInfinity();

IntervalWindow window1 = new IntervalWindow(new Instant(1), new
Instant(3));

// This fails because productDetails1 is not in the window
PAssert.that(firstTransform).inFinalPane(window1).containsInAnyOrder(

   homepage.getValue(),
    productDetails1.getValue());

pipeline.run().waitUntilFinish();

Imported from Jira BEAM-3755. Original Jira may contain additional context.
Reported by: jbfbell.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions