MINOR: improve logging of tasks on shutdown #7597
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
guozhangwang
approved these changes
Oct 25, 2019
| @@ -510,6 +510,17 @@ void clear() { | |||
| suspended.clear(); | |||
| } | |||
|
|
|||
| @Override | |||
| public void shutdown(final boolean clean) { | |||
| log.debug("Shutting down all active tasks in" + "\n" + | |||
There was a problem hiding this comment.
nit: in {} mode" + and add clean boolean?.
Ditto otherwise.
ableegoldman
force-pushed
the
MINOR-improve-TM-shutdown-logging
branch
from
357988d
to
e3e927a
Compare
|
Java 11 / 2.13 failed with env issue. Other two runs passed. Retest this please. |
ableegoldman
commented
Oct 25, 2019
| @@ -72,7 +72,8 @@ | |||
| this.eosEnabled = StreamsConfig.EXACTLY_ONCE.equals(config.getString(StreamsConfig.PROCESSING_GUARANTEE_CONFIG)); | |||
| this.stateDirectory = stateDirectory; | |||
|
|
|||
| this.logPrefix = String.format("%s [%s] ", isStandby ? "standby-task" : "task", id); | |||
| final String threadIdPrefix = String.format("stream-thread [%s] ", Thread.currentThread().getName()); | |||
| this.logPrefix = threadIdPrefix + String.format("%s [%s] ", isStandby ? "standby-task" : "task", id); | |||
There was a problem hiding this comment.
@mjsax @guozhangwang Just added this to the PR, what do you think about including the thread id in the prefix (not just the task)? I can split this into a separate PR if you want to just get this one merged, there may be some other places/classes where this would be useful as well.
mjsax
pushed a commit
that referenced
this pull request
Oct 29, 2019
Reviewers: Guozhang Wang <guozhang@confluent.io>, Matthias J. Sax <matthias@confluent.io>
|
Merged to |
ijuma
added a commit
to confluentinc/kafka
that referenced
this pull request
Nov 3, 2019
Fixed a minor conflict in `.gitignore` and fix compiler errors in KafkaUtilities due to `PartitionReplicaAssignment` rename to `ReplicaAssignment`. * apache-github/trunk: (34 commits) HOTFIX: Try to complete Send even if no bytes were written (apache#7622) KAFKA-9080: Revert the check added to validate non-compressed record batch does have continuous incremental offsets KAFKA-8972 (2.4 blocker): TaskManager state should always be updated after rebalance (apache#7620) MINOR: Fix Kafka Streams JavaDocs with regard to new StreamJoined class (apache#7627) MINOR: Fix sensor retrieval in stand0by task's constructor (apache#7632) MINOR: Replace some Java 7 style code with Java 8 style (apache#7623) KAFKA-8868: Generate SubscriptionInfo protocol message (apache#7248) MINOR: Correctly mark offset expiry in GroupMetadataManager's OffsetExpired metric KAFKA-8972 (2.4 blocker): bug fix for restoring task (apache#7617) KAFKA-9093: NullPointerException in KafkaConsumer with group.instance.id (apache#7590) KAFKA-8980: Refactor state-store-level streams metrics (apache#7584) MINOR: Fix documentation for updateCurrentReassignment (apache#7611) MINOR: Preserve backwards-compatibility by renaming the AlterPartitionReassignment metric to PartitionReassignment KAFKA-8972 (2.4 blocker): clear all state for zombie task on TaskMigratedException (apache#7608) KAFKA-9077: Fix reading of metrics of Streams' SimpleBenchmark (apache#7610) KAFKA-8972 (2.4 blocker): correctly release lost partitions during consumer.unsubscribe() (apache#7441) MINOR: improve logging of tasks on shutdown (apache#7597) KAFKA-9048 Pt1: Remove Unnecessary lookup in Fetch Building (apache#7576) MINOR: Fix command examples in kafka-reassign-partitions.sh docs (apache#7583) KAFKA-9102; Increase default zk session timeout and replica max lag [KIP-537] (apache#7596) ...
ableegoldman
pushed a commit
to ableegoldman/kafka
that referenced
this pull request
Nov 13, 2019
This reverts commit 0f2f7e6.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The current logging during a close/shutdown claims to list all active tasks but in fact lists only the running and suspended ones. Likewise it lists only the running standby tasks.
Also changed the name of
AssignedTasks#closetoshutdownas that more accurately reflects its current (and only) usage