New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

KAFKA-5157: Options for handling corrupt data during deserialization #3423

Closed
wants to merge 11 commits into
base: trunk
from

Conversation

Projects
None yet
5 participants
@enothereska
Contributor

enothereska commented Jun 23, 2017

@enothereska

This comment has been minimized.

Show comment
Hide comment
@enothereska

enothereska Jun 23, 2017

Contributor

More tests coming, but basic structure should be in place.

Contributor

enothereska commented Jun 23, 2017

More tests coming, but basic structure should be in place.

@asfgit

This comment has been minimized.

Show comment
Hide comment
@asfgit

asfgit Jun 23, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk8-scala2.12/5648/
Test PASSed (JDK 8 and Scala 2.12).

asfgit commented Jun 23, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk8-scala2.12/5648/
Test PASSed (JDK 8 and Scala 2.12).

@asfgit

This comment has been minimized.

Show comment
Hide comment
@asfgit

asfgit Jun 23, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk7-scala2.11/5662/
Test PASSed (JDK 7 and Scala 2.11).

asfgit commented Jun 23, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk7-scala2.11/5662/
Test PASSed (JDK 7 and Scala 2.11).

@enothereska enothereska changed the title from KAFKA-5157: Options for handling corrupt data during deserialization [WiP] to KAFKA-5157: Options for handling corrupt data during deserialization Jun 23, 2017

@enothereska

This comment has been minimized.

Show comment
Hide comment
@enothereska

enothereska Jun 23, 2017

Contributor

@mjsax @dguy @bbejeck @guozhangwang this is the implementation for KIP-161. Have a look when you can. Thanks.

Contributor

enothereska commented Jun 23, 2017

@mjsax @dguy @bbejeck @guozhangwang this is the implementation for KIP-161. Have a look when you can. Thanks.

@asfgit

This comment has been minimized.

Show comment
Hide comment
@asfgit

asfgit Jun 23, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk7-scala2.11/5667/
Test PASSed (JDK 7 and Scala 2.11).

asfgit commented Jun 23, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk7-scala2.11/5667/
Test PASSed (JDK 7 and Scala 2.11).

@asfgit

This comment has been minimized.

Show comment
Hide comment
@asfgit

asfgit Jun 23, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk8-scala2.12/5653/
Test PASSed (JDK 8 and Scala 2.12).

asfgit commented Jun 23, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk8-scala2.12/5653/
Test PASSed (JDK 8 and Scala 2.12).

@mjsax

Thanks for the patch. Some minor comments.

Show outdated Hide outdated streams/src/main/java/org/apache/kafka/streams/StreamsConfig.java
Show outdated Hide outdated streams/src/main/java/org/apache/kafka/streams/StreamsConfig.java
Show outdated Hide outdated streams/src/main/java/org/apache/kafka/streams/StreamsConfig.java
Show outdated Hide outdated ...java/org/apache/kafka/streams/errors/LogAndContinueExceptionHandler.java
Show outdated Hide outdated ...java/org/apache/kafka/streams/errors/LogAndContinueExceptionHandler.java
Show outdated Hide outdated ...va/org/apache/kafka/streams/processor/internals/GlobalStateTaskTest.java
Show outdated Hide outdated ...va/org/apache/kafka/streams/processor/internals/GlobalStateTaskTest.java
Show outdated Hide outdated ...t/java/org/apache/kafka/streams/processor/internals/RecordQueueTest.java
Show outdated Hide outdated ...t/java/org/apache/kafka/streams/processor/internals/RecordQueueTest.java
@@ -205,7 +206,7 @@ public ProcessorTopologyTestDriver(final StreamsConfig config,
final GlobalStateManagerImpl stateManager = new GlobalStateManagerImpl(globalTopology, globalConsumer, stateDirectory);
globalStateTask = new GlobalStateUpdateTask(globalTopology,
new GlobalProcessorContextImpl(config, stateManager, streamsMetrics, cache),
stateManager
stateManager, new LogAndContinueExceptionHandler()

This comment has been minimized.

@mjsax

mjsax Jun 24, 2017

Member

as above

@mjsax

mjsax Jun 24, 2017

Member

as above

This comment has been minimized.

@mjsax

mjsax Jul 5, 2017

Member

Guess you missed this one :)

@mjsax

mjsax Jul 5, 2017

Member

Guess you missed this one :)

@asfgit

This comment has been minimized.

Show comment
Hide comment
@asfgit

asfgit Jun 26, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk7-scala2.11/5705/
Test PASSed (JDK 7 and Scala 2.11).

asfgit commented Jun 26, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk7-scala2.11/5705/
Test PASSed (JDK 7 and Scala 2.11).

@asfgit

This comment has been minimized.

Show comment
Hide comment
@asfgit

asfgit Jun 26, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk8-scala2.12/5691/
Test PASSed (JDK 8 and Scala 2.12).

asfgit commented Jun 26, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk8-scala2.12/5691/
Test PASSed (JDK 8 and Scala 2.12).

@asfgit

This comment has been minimized.

Show comment
Hide comment
@asfgit

asfgit Jun 26, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk7-scala2.11/5707/
Test PASSed (JDK 7 and Scala 2.11).

asfgit commented Jun 26, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk7-scala2.11/5707/
Test PASSed (JDK 7 and Scala 2.11).

@asfgit

This comment has been minimized.

Show comment
Hide comment
@asfgit

asfgit Jun 26, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk8-scala2.12/5693/
Test PASSed (JDK 8 and Scala 2.12).

asfgit commented Jun 26, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk8-scala2.12/5693/
Test PASSed (JDK 8 and Scala 2.12).

@enothereska

This comment has been minimized.

Show comment
Hide comment
@enothereska

enothereska Jul 3, 2017

Contributor

@dguy @guozhangwang any further comments? Thanks.

Contributor

enothereska commented Jul 3, 2017

@dguy @guozhangwang any further comments? Thanks.

@dguy

Thanks @enothereska - i've left a few comments

Show outdated Hide outdated ...java/org/apache/kafka/streams/errors/LogAndContinueExceptionHandler.java
Show outdated Hide outdated ...ain/java/org/apache/kafka/streams/errors/LogAndFailExceptionHandler.java
Show outdated Hide outdated ...ache/kafka/streams/processor/internals/SourceNodeRecordDeserializer.java
Show outdated Hide outdated ...ache/kafka/streams/processor/internals/SourceNodeRecordDeserializer.java
// catch and process if we have a deserialization handler
try {
return deserialize(rawRecord);
} catch (Exception e) {

This comment has been minimized.

@dguy

dguy Jul 3, 2017

Contributor

Should we add a metric for records skipped due to deserialization errors?

@dguy

dguy Jul 3, 2017

Contributor

Should we add a metric for records skipped due to deserialization errors?

This comment has been minimized.

@enothereska

enothereska Jul 4, 2017

Contributor

Yes good idea. I'll piggyback on this KIP.

@enothereska

enothereska Jul 4, 2017

Contributor

Yes good idea. I'll piggyback on this KIP.

Show outdated Hide outdated ...va/org/apache/kafka/streams/processor/internals/GlobalStateTaskTest.java
Show outdated Hide outdated ...t/java/org/apache/kafka/streams/processor/internals/RecordQueueTest.java
Show outdated Hide outdated ...ache/kafka/streams/processor/internals/SourceNodeRecordDeserializer.java
@asfgit

This comment has been minimized.

Show comment
Hide comment
@asfgit

asfgit Jul 4, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk7-scala2.11/5887/
Test PASSed (JDK 7 and Scala 2.11).

asfgit commented Jul 4, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk7-scala2.11/5887/
Test PASSed (JDK 7 and Scala 2.11).

@enothereska

This comment has been minimized.

Show comment
Hide comment
@enothereska

enothereska Jul 4, 2017

Contributor

Thanks @dguy

Contributor

enothereska commented Jul 4, 2017

Thanks @dguy

@asfgit

This comment has been minimized.

Show comment
Hide comment
@asfgit

asfgit Jul 4, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk7-scala2.11/5889/
Test PASSed (JDK 7 and Scala 2.11).

asfgit commented Jul 4, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk7-scala2.11/5889/
Test PASSed (JDK 7 and Scala 2.11).

@asfgit

This comment has been minimized.

Show comment
Hide comment
@asfgit

asfgit Jul 4, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk8-scala2.12/5874/
Test PASSed (JDK 8 and Scala 2.12).

asfgit commented Jul 4, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk8-scala2.12/5874/
Test PASSed (JDK 8 and Scala 2.12).

@dguy

Thanks @enothereska. I've left one comment. I think the only other thing remaining is whether or not we should implement the pattern for setting state that @guozhangwang suggested

@enothereska

This comment has been minimized.

Show comment
Hide comment
@enothereska

enothereska Jul 4, 2017

Contributor

I've left one comment. I think the only other thing remaining is whether or not we should implement the pattern for setting state.

Thanks @dguy I'm not sure what this refers to. I looked back but I don't see @guozhangwang 's comment. Could you elaborate? Thanks.

Contributor

enothereska commented Jul 4, 2017

I've left one comment. I think the only other thing remaining is whether or not we should implement the pattern for setting state.

Thanks @dguy I'm not sure what this refers to. I looked back but I don't see @guozhangwang 's comment. Could you elaborate? Thanks.

@dguy

This comment has been minimized.

Show comment
Hide comment
@dguy

dguy Jul 4, 2017

Contributor

@enothereska doh! I've got my PRs crossed! ignore me

Contributor

dguy commented Jul 4, 2017

@enothereska doh! I've got my PRs crossed! ignore me

@asfgit

This comment has been minimized.

Show comment
Hide comment
@asfgit

asfgit Jul 5, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk7-scala2.11/5899/
Test PASSed (JDK 7 and Scala 2.11).

asfgit commented Jul 5, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk7-scala2.11/5899/
Test PASSed (JDK 7 and Scala 2.11).

@dguy

1 more comment otherwise LGTM. @mjsax can you please make another pass over it?

@asfgit

This comment has been minimized.

Show comment
Hide comment
@asfgit

asfgit Jul 5, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk8-scala2.12/5884/
Test PASSed (JDK 8 and Scala 2.12).

asfgit commented Jul 5, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk8-scala2.12/5884/
Test PASSed (JDK 8 and Scala 2.12).

@asfgit

This comment has been minimized.

Show comment
Hide comment
@asfgit

asfgit Jul 5, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk7-scala2.11/5903/
Test PASSed (JDK 7 and Scala 2.11).

asfgit commented Jul 5, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk7-scala2.11/5903/
Test PASSed (JDK 7 and Scala 2.11).

@asfgit

This comment has been minimized.

Show comment
Hide comment
@asfgit

asfgit Jul 5, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk8-scala2.12/5888/
Test PASSed (JDK 8 and Scala 2.12).

asfgit commented Jul 5, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk8-scala2.12/5888/
Test PASSed (JDK 8 and Scala 2.12).

@mjsax

Some comments.

Show outdated Hide outdated ...ava/org/apache/kafka/streams/errors/DeserializationExceptionHandler.java
Show outdated Hide outdated ...ava/org/apache/kafka/streams/errors/DeserializationExceptionHandler.java
Show outdated Hide outdated ...ava/org/apache/kafka/streams/errors/DeserializationExceptionHandler.java
Show outdated Hide outdated ...ava/org/apache/kafka/streams/errors/DeserializationExceptionHandler.java
Show outdated Hide outdated ...java/org/apache/kafka/streams/errors/LogAndContinueExceptionHandler.java
Show outdated Hide outdated ...ain/java/org/apache/kafka/streams/errors/LogAndFailExceptionHandler.java
Show outdated Hide outdated .../org/apache/kafka/streams/processor/internals/GlobalStateUpdateTask.java
Show outdated Hide outdated ...ache/kafka/streams/processor/internals/SourceNodeRecordDeserializer.java
@@ -205,7 +206,7 @@ public ProcessorTopologyTestDriver(final StreamsConfig config,
final GlobalStateManagerImpl stateManager = new GlobalStateManagerImpl(globalTopology, globalConsumer, stateDirectory);
globalStateTask = new GlobalStateUpdateTask(globalTopology,
new GlobalProcessorContextImpl(config, stateManager, streamsMetrics, cache),
stateManager
stateManager, new LogAndContinueExceptionHandler()

This comment has been minimized.

@mjsax

mjsax Jul 5, 2017

Member

Guess you missed this one :)

@mjsax

mjsax Jul 5, 2017

Member

Guess you missed this one :)

Show outdated Hide outdated ...ain/java/org/apache/kafka/streams/processor/internals/ProcessorNode.java
@asfgit

This comment has been minimized.

Show comment
Hide comment
@asfgit

asfgit Jul 6, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk8-scala2.12/5904/
Test PASSed (JDK 8 and Scala 2.12).

asfgit commented Jul 6, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk8-scala2.12/5904/
Test PASSed (JDK 8 and Scala 2.12).

@asfgit

This comment has been minimized.

Show comment
Hide comment
@asfgit

asfgit Jul 6, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk7-scala2.11/5919/
Test PASSed (JDK 7 and Scala 2.11).

asfgit commented Jul 6, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk7-scala2.11/5919/
Test PASSed (JDK 7 and Scala 2.11).

@asfgit

This comment has been minimized.

Show comment
Hide comment
@asfgit

asfgit Jul 7, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk8-scala2.12/5943/
Test FAILed (JDK 8 and Scala 2.12).

asfgit commented Jul 7, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk8-scala2.12/5943/
Test FAILed (JDK 8 and Scala 2.12).

@asfgit

This comment has been minimized.

Show comment
Hide comment
@asfgit

asfgit Jul 7, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk7-scala2.11/5958/
Test FAILed (JDK 7 and Scala 2.11).

asfgit commented Jul 7, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk7-scala2.11/5958/
Test FAILed (JDK 7 and Scala 2.11).

@enothereska

This comment has been minimized.

Show comment
Hide comment
@enothereska

enothereska Jul 7, 2017

Contributor

Could not write standard input into: Gradle build daemon

Contributor

enothereska commented Jul 7, 2017

Could not write standard input into: Gradle build daemon

@enothereska

This comment has been minimized.

Show comment
Hide comment
@enothereska

enothereska Jul 7, 2017

Contributor

retest this please

Contributor

enothereska commented Jul 7, 2017

retest this please

@asfgit

This comment has been minimized.

Show comment
Hide comment
@asfgit

asfgit Jul 7, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk7-scala2.11/5959/
Test FAILed (JDK 7 and Scala 2.11).

asfgit commented Jul 7, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk7-scala2.11/5959/
Test FAILed (JDK 7 and Scala 2.11).

@enothereska

This comment has been minimized.

Show comment
Hide comment
@enothereska

enothereska Jul 7, 2017

Contributor

org.apache.kafka.streams.integration.QueryableStateIntegrationTest.shouldAllowToQueryAfterThreadDied failed. There is already a JIRA for it https://issues.apache.org/jira/browse/KAFKA-5566.

Contributor

enothereska commented Jul 7, 2017

org.apache.kafka.streams.integration.QueryableStateIntegrationTest.shouldAllowToQueryAfterThreadDied failed. There is already a JIRA for it https://issues.apache.org/jira/browse/KAFKA-5566.

@asfgit

This comment has been minimized.

Show comment
Hide comment
@asfgit

asfgit Jul 7, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk8-scala2.12/5944/
Test FAILed (JDK 8 and Scala 2.12).

asfgit commented Jul 7, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk8-scala2.12/5944/
Test FAILed (JDK 8 and Scala 2.12).

@enothereska

This comment has been minimized.

Show comment
Hide comment
@enothereska

enothereska Jul 7, 2017

Contributor

@dguy I don't have anything else to add to this. Fixing the QueryableState in separate PR. Thanks. Also I just merged with trunk.

Contributor

enothereska commented Jul 7, 2017

@dguy I don't have anything else to add to this. Fixing the QueryableState in separate PR. Thanks. Also I just merged with trunk.

@asfgit

This comment has been minimized.

Show comment
Hide comment
@asfgit

asfgit Jul 7, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk7-scala2.11/5962/
Test PASSed (JDK 7 and Scala 2.11).

asfgit commented Jul 7, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk7-scala2.11/5962/
Test PASSed (JDK 7 and Scala 2.11).

@asfgit

This comment has been minimized.

Show comment
Hide comment
@asfgit

asfgit Jul 7, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk8-scala2.12/5947/
Test PASSed (JDK 8 and Scala 2.12).

asfgit commented Jul 7, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk8-scala2.12/5947/
Test PASSed (JDK 8 and Scala 2.12).

@enothereska

This comment has been minimized.

Show comment
Hide comment
@enothereska

enothereska Jul 10, 2017

Contributor

@guozhangwang any more comments?

Contributor

enothereska commented Jul 10, 2017

@guozhangwang any more comments?

@dguy

This comment has been minimized.

Show comment
Hide comment
@dguy

dguy Jul 10, 2017

Contributor

retest this please

Contributor

dguy commented Jul 10, 2017

retest this please

@asfgit

This comment has been minimized.

Show comment
Hide comment
@asfgit

asfgit Jul 10, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk7-scala2.11/6015/
Test PASSed (JDK 7 and Scala 2.11).

asfgit commented Jul 10, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk7-scala2.11/6015/
Test PASSed (JDK 7 and Scala 2.11).

@asfgit

This comment has been minimized.

Show comment
Hide comment
@asfgit

asfgit Jul 10, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk8-scala2.12/6000/
Test PASSed (JDK 8 and Scala 2.12).

asfgit commented Jul 10, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk8-scala2.12/6000/
Test PASSed (JDK 8 and Scala 2.12).

@dguy

dguy approved these changes Jul 10, 2017

Thanks @enothereska, LGTM

@asfgit asfgit closed this in a1f97c8 Jul 10, 2017

@guozhangwang

Left some comments on this PR. Sorry for the late reply.

sourceNode.nodeMetrics.sourceNodeSkippedDueToDeserializationError.record();
}
}
return null;

This comment has been minimized.

@guozhangwang

guozhangwang Jul 10, 2017

Contributor

We should return null in the else branch above, in case we add more options in the response that are beyond Fail or Continue.

@guozhangwang

guozhangwang Jul 10, 2017

Contributor

We should return null in the else branch above, in case we add more options in the response that are beyond Fail or Continue.

@@ -186,6 +187,7 @@ public NodeMetrics(StreamsMetrics metrics, String name, String sensorNamePrefix)
this.nodeCreationSensor = metrics.addLatencyAndThroughputSensor(scope, sensorNamePrefix + "." + name, "create", Sensor.RecordingLevel.DEBUG, tagKey, tagValue);
this.nodeDestructionSensor = metrics.addLatencyAndThroughputSensor(scope, sensorNamePrefix + "." + name, "destroy", Sensor.RecordingLevel.DEBUG, tagKey, tagValue);
this.sourceNodeForwardSensor = metrics.addThroughputSensor(scope, sensorNamePrefix + "." + name, "forward", Sensor.RecordingLevel.DEBUG, tagKey, tagValue);
this.sourceNodeSkippedDueToDeserializationError = metrics.addThroughputSensor(scope, sensorNamePrefix + "." + name, "skippedDueToDeserializationError", Sensor.RecordingLevel.DEBUG, tagKey, tagValue);

This comment has been minimized.

@guozhangwang

guozhangwang Jul 10, 2017

Contributor

As this PR gets in already, we probably need to rebase the metrics fix PR to remove the . + name as well for this new metric as well.

@guozhangwang

guozhangwang Jul 10, 2017

Contributor

As this PR gets in already, we probably need to rebase the metrics fix PR to remove the . + name as well for this new metric as well.

This comment has been minimized.

@guozhangwang

guozhangwang Jul 10, 2017

Contributor

Also I'm wondering what's the right approach of this metric: we already have a thread-level skippedRecordsSensor that covers this as it records total fetched records - total enqueued records. And then we have this source-node level metrics that only tracks the skippedRecordsDuetoDeserError. On the other hand, on the thread-level we do not have any other throughput metrics except the skippedRecordsSensor.

So should we just have a higher-hierarchy sensor at the thread level for number of skipped records rates (for whatever reason), or do we want to have a lower source-node level sensor for skipped records rates of different causes? In either way I think the current hierarchy is a bit messed and should be fixed. WDYT @dguy @enothereska ?

@guozhangwang

guozhangwang Jul 10, 2017

Contributor

Also I'm wondering what's the right approach of this metric: we already have a thread-level skippedRecordsSensor that covers this as it records total fetched records - total enqueued records. And then we have this source-node level metrics that only tracks the skippedRecordsDuetoDeserError. On the other hand, on the thread-level we do not have any other throughput metrics except the skippedRecordsSensor.

So should we just have a higher-hierarchy sensor at the thread level for number of skipped records rates (for whatever reason), or do we want to have a lower source-node level sensor for skipped records rates of different causes? In either way I think the current hierarchy is a bit messed and should be fixed. WDYT @dguy @enothereska ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment