Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Failing Test]: ViewTest.testTriggeredLatestSingleton extremely flaky on Flink runner after #28614 #29547

Closed
1 of 16 tasks
Abacn opened this issue Nov 28, 2023 · 5 comments
Closed
1 of 16 tasks

Comments

@Abacn
Copy link
Contributor

Abacn commented Nov 28, 2023

What happened?

Error message:

org.apache.beam.sdk.Pipeline$PipelineExecutionException: java.lang.IllegalArgumentException: PCollection with more than one element accessed as a singleton view.
	at org.apache.beam.runners.flink.TestFlinkRunner.run(TestFlinkRunner.java:75)
	at org.apache.beam.sdk.Pipeline.run(Pipeline.java:321)
Caused by: java.lang.IllegalArgumentException: PCollection with more than one element accessed as a singleton view.
	at org.apache.beam.sdk.values.PCollectionViews$SingletonViewFn2.apply(PCollectionViews.java:412)
	at org.apache.beam.sdk.values.PCollectionViews$SingletonViewFn2.apply(PCollectionViews.java:339)
	at org.apache.beam.runners.core.SideInputHandler.get(SideInputHandler.java:163)
	at org.apache.beam.runners.core.SimpleDoFnRunner.sideInput(SimpleDoFnRunner.java:246)
	at org.apache.beam.runners.core.SimpleDoFnRunner.access$1200(SimpleDoFnRunner.java:85)
	at org.apache.beam.runners.core.SimpleDoFnRunner$DoFnProcessContext.sideInput(SimpleDoFnRunner.java:400)
	at org.apache.beam.runners.core.SimpleDoFnRunner$DoFnProcessContext.sideInput(SimpleDoFnRunner.java:487)
	at org.apache.beam.sdk.transforms.ViewTest$7$DoFnInvoker.invokeProcessElement(Unknown Source)

Issue Failure

Failure: Test is flaky

Issue Priority

Priority: 2 (backlog / disabled test but we think the product is healthy)

Issue Components

  • Component: Python SDK
  • Component: Java SDK
  • Component: Go SDK
  • Component: Typescript SDK
  • Component: IO connector
  • Component: Beam YAML
  • Component: Beam examples
  • Component: Beam playground
  • Component: Beam katas
  • Component: Website
  • Component: Spark Runner
  • Component: Flink Runner
  • Component: Samza Runner
  • Component: Twister2 Runner
  • Component: Hazelcast Jet Runner
  • Component: Google Cloud Dataflow Runner
@Abacn
Copy link
Contributor Author

Abacn commented Nov 28, 2023

#28614 added another validate runner test (validatesRunnerBatchWithDataStream) so some tests get executed one more time and adds more chance to flakiness. However this particular test seems most flaky

@Abacn
Copy link
Contributor Author

Abacn commented Nov 28, 2023

CC: @jto I am going to disable this test for now and leave this issue open until this feature is get fixed in Flink runner BatchWithDataStream

@jto
Copy link
Contributor

jto commented Nov 28, 2023

Hey!
I think I fixed this specific issue in another branch. See this commit: 69153e6.
I'll make sure it fixes the issue and make a PR tomorrow :)

@Abacn
Copy link
Contributor Author

Abacn commented Nov 28, 2023

Thank you for prompt reply!

@jto
Copy link
Contributor

jto commented Nov 29, 2023

Here's the PR #29557 :)

@Abacn Abacn closed this as completed Nov 30, 2023
@github-actions github-actions bot added this to the 2.53.0 Release milestone Nov 30, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants