Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Flink Runner] Add UseDataStreamForBatch option to Flink runner to enable batch execution on DataStream API #28614

Merged
merged 8 commits into from
Oct 27, 2023
Merged

[Flink Runner] Add UseDataStreamForBatch option to Flink runner to enable batch execution on DataStream API #28614

merged 8 commits into from
Oct 27, 2023

Commits on Oct 26, 2023

  1. Add Flink PipelineOption of UseDataStreamForBatch. 

    Modify the FlinkRunner to use DataStream API for batch job execution if
UseDataStreamForBatch is set to true.
    @jto
    Jiangjie Qin authored and jto committed Oct 26, 2023
    Configuration menu
    Copy the full SHA
    9dd922d View commit details
    Browse the repository at this point in the history

  2. Modify the unit tests and runner validation tests to cover the 

    DataStream execution path of batch jobs.
    @jto
    Jiangjie Qin authored and jto committed Oct 26, 2023
    Configuration menu
    Copy the full SHA
    04c24dc View commit details
    Browse the repository at this point in the history

  3. Update CHANGES.md to record the notable change.

    @jto
    Jiangjie Qin authored and jto committed Oct 26, 2023
    Configuration menu
    Copy the full SHA
    741e2cb View commit details
    Browse the repository at this point in the history

  4. Fix compilation errors

    @jto
    jto committed Oct 26, 2023
    Configuration menu
    Copy the full SHA
    c73aa05 View commit details
    Browse the repository at this point in the history

  5. spotless

    @jto
    jto committed Oct 26, 2023
    Configuration menu
    Copy the full SHA
    a05ea70 View commit details
    Browse the repository at this point in the history

Commits on Oct 27, 2023

  1. Update change list

    @jto
    jto committed Oct 27, 2023
    Configuration menu
    Copy the full SHA
    b87005d View commit details
    Browse the repository at this point in the history

  2. Do not exclude org.apache.beam.sdk.transforms.GroupByKeyTest.testAfte… 

    …rProcessingTimeContinuationTriggerUsingState
    @jto
    jto committed Oct 27, 2023
    Configuration menu
    Copy the full SHA
    e149b36 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'master' into datastream_batch

    @tvalentyn
    tvalentyn committed Oct 27, 2023
    Configuration menu
    Copy the full SHA
    6687bdd View commit details
    Browse the repository at this point in the history