Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FLINK-25809][table-api-java] Add table test program infrastructure #23584

Merged
merged 1 commit into from
Oct 26, 2023

Conversation

twalthr
Copy link
Contributor

@twalthr twalthr commented Oct 24, 2023

What is the purpose of the change

This adds infrastructure for declaring table test programs once and use them for various purposes.

/**
 * A generic declaration of a table program for testing.
 *
 * <p>A test program only defines the basic test pipeline (from source to sink) and required
 * artifacts such as table sources and sinks, configuration options, and user-defined functions.
 *
 * <p>Tests programs are intended to reduce code duplication and test the same SQL statement though
 * different layers of the stack. Different {@link TableTestProgramRunner}s can share the same
 * program and enrich it with custom implementation and assertions.
 *
 * <p>For example, a SQL query such as {@code SELECT * FROM (VALUES (1), (2), (3))} can be declared
 * once and can be shared among different tests for integration testing, optimizer plan testing,
 * compiled plan testing, transformation testing, and others.
 */

Brief change log

  • API and builder patterns added
  • Basic interface for runners added

Verifying this change

Testing is TBD.

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): no
  • The public API, i.e., is any changed class annotated with @Public(Evolving): no
  • The serializers: no
  • The runtime per-record code paths (performance sensitive): no
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
  • The S3 file system connector: no

Documentation

  • Does this pull request introduce a new feature? no
  • If yes, how is the feature documented? JavaDocs

@flinkbot
Copy link
Collaborator

flinkbot commented Oct 24, 2023

CI report:

Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

@twalthr twalthr force-pushed the FLINK-25809 branch 2 times, most recently from bcb15fb to 964891a Compare October 25, 2023 14:29
@twalthr twalthr marked this pull request as ready for review October 25, 2023 14:29
Copy link
Contributor

@dawidwys dawidwys left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we change a single field, which would make it possible to write restore tests. Otherwise looks good.

Comment on lines +32 to +33
public final @Nullable Predicate<List<Row>> expectedBeforeRestore;
public final @Nullable Predicate<List<Row>> expectedAfterRestore;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we switch it to List<String>? That's what TestValuesTable produces. Having a predicate on List<Row> makes it really hard to use with List<String> as it is not straightforward to convert back to a Row.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After an offline discussion we said we'll try to adapt TestValuesTable.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants