New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

implement lazy selector registration #388

Merged
merged 1 commit into from May 4, 2018

Conversation

Projects
None yet
3 participants
@weissi
Copy link
Contributor

weissi commented May 3, 2018

Motivation:

Previously we would eagerly register any fd with the selector
(epoll/kqueue) which works well with kqueue. With epoll however, you'll
get EPOLLHUP immediately if you supplied a socket that hasn't had
connect/bind called on it.
We got away with this because we took quite a bit of care to always make
sure we call connect/bind pretty much immediately after we registered
it. And crucially before we ever asked the selector to tell us about
new events.

Modifications:

made selector registration lazy (when we try activating the channel by
calling connect/bind)

Result:

the user can now register and connect whenever they feel like it, fixes #380

@weissi weissi force-pushed the weissi:jw-late-reg branch from baf42bf to 79108f2 May 3, 2018

@weissi weissi requested review from Lukasa and normanmaurer May 3, 2018

@Lukasa
Copy link
Contributor

Lukasa left a comment

Cool, mostly looks really good! Some notes!

@@ -18,14 +18,16 @@ private struct SocketChannelLifecycleManager {
// MARK: Types
private enum State {
case fresh
case registered
case lazilyRegistered // register() has been run but the selector doesn't know about it yet
case fullyRegistered // fully registered, ie. the selector knows about it

This comment has been minimized.

@Lukasa

Lukasa May 3, 2018

Contributor

Nit: I don't love these names. Can we call the new state preRegistered, maybe?

case activated
case closed
}

private enum Event {
case activate
case register
case registerLazily
case registerFully

This comment has been minimized.

@Lukasa

Lukasa May 3, 2018

Contributor

Similarly, these names aren't necessarily right. Maybe beginRegistration and completeRegistration?

if self.lifecycleManager.isRegisteredLazily {
try! becomeFullyRegistered0()
if self.lifecycleManager.isRegisteredFully {
becomeActive0(promise: promise)

This comment has been minimized.

@Lukasa

Lukasa May 3, 2018

Contributor

Do these callouts not get triggered by register0?

This comment has been minimized.

@weissi

weissi May 4, 2018

Author Contributor

@Lukasa right now register0 only does the pre-registration. I didn't change its name as it's on ChannelCore. But we could obviously rename it to preRegister0

guard self.selectableEventLoop.isOpen else {
let error = EventLoopError.shutdown
self.pipeline.fireErrorCaught0(error: error)
self.close0(error: error, mode: .all, promise: nil)
promise?.fail(error: error)

This comment has been minimized.

@normanmaurer

normanmaurer May 4, 2018

Member

I think we should just pass the promise into self.close0(...) which will ensure correct ordering as well.

So basically:

self.close0(error: error, mode: .all, promise: promise)
@@ -337,6 +337,11 @@ internal final class SelectableEventLoop: EventLoop {
_addresses.deallocate()
}

/// Is this `SelectableEventLoop` still open (ie. not shutting down or shut down)
public var isOpen: Bool {

This comment has been minimized.

@normanmaurer

normanmaurer May 4, 2018

Member

@weissi this is not safe as lifecycleState is not thread-safe. Maybe change to internal and assert that the caller is the EventLoop thread ?

This comment has been minimized.

@weissi

weissi May 4, 2018

Author Contributor

@normanmaurer it's fine as SelectableEventLoop is an internal type (not public) but regardless I changed this to internal and added an assertion

XCTAssertNoThrow(try server.register().wait())
XCTAssertNoThrow(try server.eventLoop.submit {
XCTAssertFalse(server.isActive)
}.wait())

This comment has been minimized.

@normanmaurer

normanmaurer May 4, 2018

Member

remove 4 spaces.

@@ -2317,6 +2317,43 @@ public class ChannelTests: XCTestCase {

}

func testLazyRegistrationWorksForServerSockets() throws {
let group = MultiThreadedEventLoopGroup(numThreads: 1)

This comment has been minimized.

@normanmaurer

normanmaurer May 4, 2018

Member

add:

defer {
    XCTAssertNoThrow(try group.syncShutdownGracefully())
}

This comment has been minimized.

@weissi

weissi May 4, 2018

Author Contributor

done

}

func testLazyRegistrationWorksForClientSockets() throws {
let group = MultiThreadedEventLoopGroup(numThreads: 1)

This comment has been minimized.

@normanmaurer

normanmaurer May 4, 2018

Member

add:

defer {
    XCTAssertNoThrow(try group.syncShutdownGracefully())
}

This comment has been minimized.

@weissi

weissi May 4, 2018

Author Contributor

done

implement lazy selector registration
Motivation:

Previously we would eagerly register any fd with the selector
(epoll/kqueue) which works well with kqueue. With epoll however, you'll
get `EPOLLHUP` immediately if you supplied a socket that hasn't had
connect/bind called on it.
We got away with this because we took quite a bit of care to always make
sure we call connect/bind pretty much immediately after we registered
it.  And crucially before we ever asked the selector to tell us about
new events.

Modifications:

made selector registration lazy (when we try activating the channel by
calling connect/bind)

Result:

the user can now register and connect whenever they feel like it.

@weissi weissi force-pushed the weissi:jw-late-reg branch from 79108f2 to 1ec5999 May 4, 2018

@Lukasa

Lukasa approved these changes May 4, 2018

Copy link
Contributor

Lukasa left a comment

This LGTM, great job @weissi!

@Lukasa Lukasa added this to the 1.7.0 milestone May 4, 2018

@normanmaurer normanmaurer merged commit 2f55baf into apple:master May 4, 2018

2 checks passed

pull request validation (4.0.3) Build finished.
Details
pull request validation (4.1) Build finished.
Details
@normanmaurer

This comment has been minimized.

Copy link
Member

normanmaurer commented May 4, 2018

Thanks @weissi ... merged :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment