Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

KAFKA-6741: Disable Selector's idle connection timeout in testNetworkThreadTimeRecorded() test #4824

Merged
merged 2 commits into from
Apr 5, 2018
Merged

KAFKA-6741: Disable Selector's idle connection timeout in testNetworkThreadTimeRecorded() test #4824

merged 2 commits into from
Apr 5, 2018

Conversation

omkreddy
Copy link
Contributor

@omkreddy omkreddy commented Apr 4, 2018

No description provided.

@omkreddy
Copy link
Contributor Author

omkreddy commented Apr 4, 2018 •
edited
Loading

@rajinisivaram trace logs shows that connection is getting closed due to idle time out. increased the idle connection timeout to 10secs for this test alone. Pls take a look.

[2018-04-05 00:21:35,390] TRACE About to close the idle connection from 0 due to being idle for 5170 millis (org.apache.kafka.common.network.Selector:639)
[2018-04-05 00:21:35,393] DEBUG Connection with /127.0.0.1 disconnected (org.apache.kafka.common.network.Selector:527)
java.io.IOException: Broken pipe
	at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
	at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47)
	at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
	at sun.nio.ch.IOUtil.write(IOUtil.java:65)
	at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:471)
	at org.apache.kafka.common.network.SslTransportLayer.flush(SslTransportLayer.java:211)
	at org.apache.kafka.common.network.SslTransportLayer.write(SslTransportLayer.java:644)
	at org.apache.kafka.common.network.SslTransportLayer.write(SslTransportLayer.java:677)
	at org.apache.kafka.common.network.SslTransportLayer.write(SslTransportLayer.java:702)
	at org.apache.kafka.common.network.ByteBufferSend.writeTo(ByteBufferSend.java:60)
	at org.apache.kafka.common.network.KafkaChannel.send(KafkaChannel.java:239)
	at org.apache.kafka.common.network.KafkaChannel.write(KafkaChannel.java:210)
	at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:509)
	at org.apache.kafka.common.network.Selector.poll(Selector.java:424)
	at org.apache.kafka.common.network.NioEchoServer.run(NioEchoServer.java:145)

@omkreddy omkreddy changed the title KAFKA-6741: Increase Selector's idle connection timeout of testNetwokkThreadTimeRecorded() test KAFKA-6741: Increase Selector's idle connection timeout of testNetworkThreadTimeRecorded() test Apr 4, 2018
hachikuji
hachikuji reviewed Apr 5, 2018
View reviewed changes
clients/src/test/java/org/apache/kafka/common/network/SslTransportLayerTest.java Outdated
@@ -635,7 +635,7 @@ public void testApplicationBufferResize() throws Exception {
@Test
public void testNetworkThreadTimeRecorded() throws Exception {
selector.close();
this.selector = new Selector(NetworkReceive.UNLIMITED, 5000, new Metrics(), Time.SYSTEM,
this.selector = new Selector(NetworkReceive.UNLIMITED, 10000, new Metrics(), Time.SYSTEM,
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it make sense to disable idle timeout with NO_IDLE_TIMEOUT_MS for this test? Do we want it to be at play?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree, We can set timeout to NO_IDLE_TIMEOUT_MS for this test.

@omkreddy omkreddy changed the title KAFKA-6741: Increase Selector's idle connection timeout of testNetworkThreadTimeRecorded() test KAFKA-6741: Disable Selector's idle connection timeout in testNetworkThreadTimeRecorded() test Apr 5, 2018
@rajinisivaram
Copy link
Contributor

@omkreddy Is the test failing consistently without this fix and passing consistently with this fix? The test is waiting for a receive and if it fails to receive, then it would hit idle timeout since there is no other traffic, but I am not sure idle timeout is the cause of the failure. There could be some other timing issue that is causing the receive to fail.

@omkreddy
Copy link
Contributor Author

omkreddy commented Apr 5, 2018

@rajinisivaram Yes, test is failing most of the times without this patch and passing consistently with the fix. ran 100 times without any error. so i thought, may be system slowness/load is causing the test to fail.

rajinisivaram
rajinisivaram approved these changes Apr 5, 2018
View reviewed changes
Copy link
Contributor

@rajinisivaram rajinisivaram left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@omkreddy Thanks for the PR, LGTM.

@rajinisivaram
Copy link
Contributor

@omkreddy Thanks for the PR, merging to trunk.

@rajinisivaram rajinisivaram merged commit 77c79df into apache:trunk Apr 5, 2018
@omkreddy omkreddy deleted the SSLTransport branch July 3, 2018 15:44
ying-zheng pushed a commit to ying-zheng/kafka that referenced this pull request Jul 6, 2018
@omkreddy @ying-zheng
KAFKA-6741: Disable Selector's idle connection timeout in testNetwork…
1bc993a 
…ThreadTimeRecorded() test (apache#4824)

Reviewers: Jason Gustafson <jason@confluent.io>, Rajini Sivaram <rajinisivaram@googlemail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@hachikuji hachikuji hachikuji left review comments

@rajinisivaram rajinisivaram rajinisivaram approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@omkreddy @rajinisivaram @hachikuji