[BEAM-14091] Fixing Interactive Beam show/collect for remote runners

[victorplusc]

• Bug: the show and collect commands under Interactive Beam would eventually call a PipelineFragment to execute the InteractiveRunner corresponding to the given PCollection. The PipelineFragment by default does not wait for pipeline to finish execution, and when a pipeline runs the FlinkRunner remotely, the Interactive Environment is not updated with the results of the pipeline run. Subsequent calls to ib.show/ib.collect would rerun the pipeline instead of directly retrieving the results from the Interactive Environment.
• Setting the blocking to be True when using the FlinkRunner inside the RecordingManager.record() method when calling the .run() method of the PipelineFragment properly saves completed pipeline results in the Interactive Environment and prevents a need to rerun the pipelines again on subsequent calls to ib.show/ib.collect.
• ASF Jira thread: https://issues.apache.org/jira/browse/BEAM-14091

---

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

• Choose reviewer(s) and mention them in a comment (R: @username).
• Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
• Update CHANGES.md with noteworthy changes.
• If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md

GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

[victorplusc]

R: @KevinGG
R: @rohdesamuel

[codecov]

Codecov Report

Merging #17072 (825d604) into master (15e636f) will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##           master   #17072   +/-   ##
=======================================
  Coverage   73.95%   73.95%           
=======================================
  Files         669      669           
  Lines       88156    88159    +3     
=======================================
+ Hits        65193    65197    +4     
+ Misses      21851    21850    -1     
  Partials     1112     1112           

Flag	Coverage Δ
python	83.64% <100.00%> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
...eam/runners/interactive/interactive_environment.py	90.51% <100.00%> (-0.23%)	⬇️
...ache_beam/runners/interactive/recording_manager.py	96.58% <100.00%> (+0.03%)	⬆️
...ks/python/apache_beam/runners/interactive/utils.py	95.41% <100.00%> (+0.17%)	⬆️
sdks/python/apache_beam/transforms/util.py	95.96% <0.00%> (-0.17%)	⬇️
...hon/apache_beam/runners/worker/bundle_processor.py	93.64% <0.00%> (+0.12%)	⬆️
...n/apache_beam/ml/gcp/recommendations_ai_test_it.py	75.51% <0.00%> (+2.04%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 15e636f...825d604. Read the comment docs.

[KevinGG]

LGTM, just a nit

[victorplusc]

Run Python PreCommit

[KevinGG]

LGTM

[victorplusc]

Run Python PreCommit