Skip to content

Conversation

@leonardBang
Copy link
Contributor

What is the purpose of the change

  • Fix StreamExecOverAggregate and StreamExecTemporalSort operators didn't support LocalZonedTimestampType

Brief change log

  • Use LogicalTypeChecks#isRowtimeAttribute() utils to check the rowtime attribute

Verifying this change

  • Add ITCase to test the two operators.

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): (no)
  • The public API, i.e., is any changed class annotated with @Public(Evolving): (no)
  • The serializers: (no)
  • The runtime per-record code paths (performance sensitive): ( no)
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (no)
  • The S3 file system connector: (no)

Documentation

  • Does this pull request introduce a new feature? (no)
  • If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

dawidwys and others added 30 commits June 3, 2021 09:37
…ming from unaligned checkpoints

In the initial implementation of rescaling channel data we assumed that
the subtasks indices are the same for all input gates. This assumptions
does not hold when there are different partitioners. It can happen e.g.
in case of a broadcast with a keyed input.

The commit tracks the subtask indices separately for each partition and
input gate.

This closes apache#16019
…ng rescale edges in EdgeManagerBuildUtil

This closes apache#16071
…egment in order to avoid extracting it from the bufferBuilder

(cherry picked from commit 3974687)
…rship of the input Buffer

(cherry picked from commit 2293362)
…el#onBuffer in javadoc

(cherry picked from commit bc8c739)
…dBeExecutedInShutdownHook

Hardens the ClusterEntrypointTest.testCloseAsyncShouldBeExecutedInShutdownHook by increasing the timeout from 3s to 10s.
…erEntrypointTest.testCloseAsyncShouldBeExecutedInShutdownHook

This closes apache#16111.
… for SourceTasks

This commit is fixing a bug of checkpointStartDelay being not set in
CheckpointMetrics of SourceStreamTask and SourceOperatorStreamTask.
tweise and others added 28 commits August 25, 2021 06:26
… tests due to Python 3.7 was removed from debian
This commit reverts the change that we store the logs under FLINK_DIR/logs because this can
make the e2e tests fail if the directory does not exist. Now we store the logs under FLINK_DIR/log

This closes apache#16982.
…d to end tests due to Python 3.7 was removed from debian"

This reverts commit f998da0
In order to avoid that we don't log failure causes of failed jobs (e.g. initialization errors),
this commit lets the Dispatcher log the failure cause when archiving the job. This will ensure
that there will be always a record of what happened in the logs of the Dispatcher.

This closes apache#17013.
This commit makes the handler that is used for the testConcurrentQueries shareable so
that Netty won't fail if another pipeline is created.
…ction

This commit replaces the Client.pendingConnections and .establishedConnections
with a single data structure connections of type ServerConnection. The logic of
a pending and established conneciton is moved into this class. This allows to get
rid of concurrent access to the Client data structures.

This closes apache#16915.
The problem was that we output the number of state entries at the time when the checkpoint
is confirmed. However, at this point in time we might have already added more elements to
the map. Hence, we risk reporting more elements than are actually contained in the checkpoint.
The solution is to remember what the element count was when creating the checkpoint.

This closes apache#17028.
…ultDispatcherRunner

This commit logs on info level when the DefaultDispatcherRunner was granted or revoked its leadership.

This closes apache#17034.
… in changelog inference for DAG optimizing

This closes apache#16979

(cherry picked from commit ffd2520)
…ecTemporalSort operator didn't support LocalZonedTimestampType
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.