Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Kafka-5676: MockStreamsMetrics should be in o.a.k.test #3657

Closed

Conversation

DevelopersWithPassion
Copy link

  1. Change package for MockStreamsMetrics class
  2. Removed metric object from MockStreamsMetrics constcruor

@asfgit
Copy link

asfgit commented Aug 11, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk8-scala2.12/6690/
Test FAILed (JDK 8 and Scala 2.12).

@asfgit
Copy link

asfgit commented Aug 11, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk7-scala2.11/6705/
Test FAILed (JDK 7 and Scala 2.11).

@DevelopersWithPassion DevelopersWithPassion changed the title Kafka-5676 Kafka-5676 MockStreamsMetrics should be in o.a.k.test Aug 19, 2017
@guozhangwang
Copy link
Contributor

Thanks for the PR @DevelopersWithPassion Could you rebase with the latest trunk?

Copy link
Contributor

@guozhangwang guozhangwang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some comments on how to make the MockStreamsMetrics really mocking.

@@ -25,7 +24,7 @@
import org.apache.kafka.streams.kstream.SessionWindows;
import org.apache.kafka.streams.kstream.Windowed;
import org.apache.kafka.streams.processor.Processor;
import org.apache.kafka.streams.processor.internals.MockStreamsMetrics;
import org.apache.kafka.test.MockStreamsMetrics;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: reorder the imports alphabetically

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure will do that.

import org.apache.kafka.test.MockStateStoreSupplier;
import org.apache.kafka.test.MockTimestampExtractor;
import org.apache.kafka.test.TestUtils;
import org.apache.kafka.test.*;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We do not recommend using wildcards in imports.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok .I will change it

public MockStreamsMetrics(Metrics metrics) {
super(metrics, "mock-stream-metrics",
public MockStreamsMetrics() {
super(new Metrics(), "mock-stream-metrics",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So far this mock object is still actually implementing the full functionalities of a StreamsMetricsImpl since the mock actually extends it; the "mocking" is that it is passing in a default group name and an empty tag map.

A real "mock" class, would be not implementing any real functionalities (so not needing an underlying Metrics object) but just implementing the StreamsMetrics interface as no-ops, or at best recording the times that some APIs has been called etc. For cases that really needs a functional StreamsMetricsImpl object, we should just pass it instead of the mocking object. Please see my comments in the related class.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok let me try other way . I have to change lot of test cases . Thanks for the information :-)

@@ -115,7 +109,8 @@
private final byte[] recordKey = intSerializer.serialize(null, 1);
private final String applicationId = "applicationId";
private final Metrics metrics = new Metrics();
private final StreamsMetrics streamsMetrics = new MockStreamsMetrics(metrics);
private final StreamsMetrics streamsMetrics = new StreamsMetricsImpl(metrics, "mock-stream-metrics",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As we mentioned in the ticket, the only test case that requires the underlying metrics object is testMetrics. For this purpose we should just rewrite this test function, to not reuse the class field task which is relying on the MockStreamsMetrics, but create another task that does pass in a real StreamsMetricsImpl.

@@ -45,7 +44,7 @@

@Before
public void setUp() throws Exception {
streamMetrics = new MockStreamsMetrics(new Metrics());
streamMetrics = new MockStreamsMetrics();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similarly as in StreamTaskTest, we should just use a different cache object in the testMetrics function than the class field cache which gets a real StreamsMetricsImpl directly.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similarly for ProcessorNodeTest: the only difference is that for that test case we can introduce another overloaded constructor for MockProcessorContext which takes a StreamsMetrics object, and then pass a real StreamsMetricsImpl only for that testMetrics class while for all others just use the other constructor which always use a MockStreamsMetrics object.

@guozhangwang
Copy link
Contributor

Also you can rename your PR title as KAFKA-5676: ... and see if the JIRA gets auto updated.

@DevelopersWithPassion DevelopersWithPassion changed the title Kafka-5676 MockStreamsMetrics should be in o.a.k.test Kafka-5676: MockStreamsMetrics should be in o.a.k.test Aug 26, 2017
@mjsax mjsax added the streams label Jan 30, 2018
@guozhangwang
Copy link
Contributor

@DevelopersWithPassion are you still working on this PR?

@DevelopersWithPassion
Copy link
Author

@guozhangwang no i am not working on it . will continue after some time . please assign it to somebody if priority has changed

@guozhangwang
Copy link
Contributor

Just checking, no priority changed for now :)

@mjsax mjsax added the tests Test fixes (including flaky tests) label Dec 28, 2022
@mjsax
Copy link
Member

mjsax commented Dec 29, 2022

Closing this old PR due to inactivity. PR would need to be rebased anyway.

@mjsax mjsax closed this Dec 29, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
streams tests Test fixes (including flaky tests)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants