KAFKA-6376: Document skipped records metrics changes #4922
Conversation
|
@guozhangwang @mjsax @bbejeck, |
| @@ -101,6 +101,16 @@ <h1>Upgrade Guide and API Changes</h1> | |||
|
|
|||
| <!-- TODO: verify release verion and update `id` and `href` attributes (also at other places that link to this headline) --> | |||
| <h3><a id="streams_api_changes_120" href="#streams_api_changes_120">Streams API changes in 1.2.0</a></h3> | |||
| <p> | |||
| We have removed the <code>skippedDueToDeserializationError-rate</code> and <code>skippedDueToDeserializationError-total</code> metrics. | |||
| Deserialization errors, and all other causes of record skipping, are now accounted for in the pre-existing metrics | |||
There was a problem hiding this comment.
nit: could we list the possible reasons of skipped records here?
There was a problem hiding this comment.
yeah, I considered doing that but was overcome by laziness. ;) I'll add it.
There was a problem hiding this comment.
I'm working my way though this, but just had this thought... if we don't want people to depend on record skipping for correctness, is there an argument that we shouldn't document the behavior?
There was a problem hiding this comment.
I think "correctness" here is really dependent on users app case-by-case: if I do not expect ANY skipped records, then I will monitor on this metric > 0, if I do expect, that sometimes there are null keys from the source topic that will be skipped, then I will probably not monitor on this metric or just monitor on it not larger than a very big value indicating spiky anomaly.
|
@guozhangwang I've added the reasons. |
Reviewers: Bill Bejeck <bill@confluent.io>, Guozhang Wang <wangguoz@gmail.com>
Reviewers: Bill Bejeck <bill@confluent.io>, Guozhang Wang <wangguoz@gmail.com>
Document the metrics changes in ed51b2c .
Committer Checklist (excluded from commit message)