[sql-jdbc] #20192
Closed
[sql-jdbc] #20192
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…_boot.py This closes #18025.
Use a dedicated thread to run each jar, so that pooled threads can't keep references to user-code (e.g., in a ThreadLocal).
… TableSourceTable This closes #18040 Co-authored-by: guanghxu <xuguangheng1995@gmail.com>
…s not the same with query result's changelog upsert key This closes #18048
This reverts commit 50cd7bb.
…SlotProfileTestingUtils
…rofile to `reservedAllocations` to avoid confusion
…kable out to be a standalone test class in flink-runtime
…r batch jobs even if local recovery is enabled
…to recover after losing and regaining leadership.
…perLeaderElectionITCase#testJobExecutionOnClusterWithLeaderChange` (FLINK-25235) This closes #18066.
…ternalProducer if transaction finalization fails In the KafkaCommitter we retry transactions if they failed during committing. Since we reuse the KafkaProducers we update the used transactionalId to continue committing other transactions. To prevent accidental overwrites we track the transaction state inside the FlinkKafkaInternalProducer. Before this change, the state was not reset on a failures during the transaction finalization and setting a new transactionalId failed. The state is now always reset nevertheless whether finalizing the transaction fails (commit, abort).
…explicitly for job to finish. (cherry picked from commit 7976be0)
while using TableEnvironment in the ITCase, a Flink MiniCluster will be started/stopped automatically in the background. Since the shutdown of the MiniCluster will be called asynchronously, CollectResultFetcher will get data lost sometimes based on race conditions and the unchecked RuntimeException java.lang.IllegalStateException will be thrown that we were not aware of. The solution is to control the lifecycle of the MiniCluster manually in this test. The MiniClusterWithClientResource could be a good fit in this case. (cherry picked from commit fca04c3)
…ngTest#testRemoteClose This closes #18089.
…CKPOINTS as default for externalized-checkpoint-retention
This consistency level is only available on write, so we need to create one builder for reading and one for writing. Some sinks are used for both reading and writing, in that case, reading builder is used. (cherry picked from commit c40bbf1)
… mailbox is closed already
We suspect that the NetworkFailureProxy is causing constant connectivity problems to the brokers during testing resulting in either network timeouts or corrupted results. Since the NetworkFailureProxy is only used for testing the deprecated FlinkKafkaProducer/Consumer we can safely remove it because we will not add new features to the connectors.
…e avro schema (#19705) Co-authored-by: Haizhou Zhao <haizhou_zhao@apple.com>
…to-end tests to avoid flooding the disk space
…kages to clean up more diskspace before starting the E2E tests. Also removing the line that removes `^ghc-8.*` since that doesn't exist anymore on the machines. (cherry picked from commit db6baf4)
…w exception This closes #19241.
…already provided in producer Properties
… PyFlink Table API jobs in batch mode This closes #19816.
…verride method 'merge' is used in cases where 'merge' is used This closes #19817.
This closes #19848.
…ed record This closes #19913
…rtitionSplitReader.fetch() to handle no valid partition case This closes #19979.
…onsumer invocations in split assignment This closes #19982.
…ttl enabled in RetractableTopNFunction This closes #19997
…r only from one thread.
…an continue to use Kubernetes 1.24+ and the `none` driver, since Kubernetes 1.24 has dropped support for Dockershim.
…isk space (cherry picked from commit a339704)
Author
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What is the purpose of the change
When I using the sql-jdbc to transform a big table from mysql to other database, the flink program load the entire table into memory. The source table is too big (16GB), and the taskmanager crashed.
So What can I do, or what about add a new option to limit the speed of reading data (or batch the data )
Brief change log
Verifying this change
Does this pull request potentially affect one of the following parts:
Documentation