Cocotron (AppKit) backend rework

[LubosD]

This issue documents how Cocotron's integration to lower API layers should work.

Backends for X11 and Wayland

• AppKit should do CGS* API calls to interface with the window server. This means that backends from cocotron/AppKit/*.backend must go. There's a nice reversed collection of CGS API declarations here.
• Implement new backends in cocotron/CoreGraphics/*.backend that implement the new CGS interfaces.
  • The X11 backend shall use the XInput2 API to receive events.

Event Processing

• All events must exist as CGSEventRecords prior to becoming CGEvents (CGEventCreateWithEventRecord()) and later NSEvents (for AppKit apps).
  • All CGS events run via an event port (CGSGetEventPort())
  • Call stack of event reception from the window server:

              CoreGraphics`CGEventCreateFromDataAndSource+0xbce
              CoreGraphics`CGSDecodeEventRecord+0x6a
              CoreGraphics`CGSDispatchDatagramsFromStream+0x28f
              CoreGraphics`snarfEvents+0x12a
              CoreGraphics`CGSGetNextEventRecordInternal+0x9f
              CoreGraphics`CGEventCreateNextEvent+0x2c
              HIToolbox`PullEventsFromWindowServerOnConnection(unsigned int, unsigned char)+0x58
              CoreFoundation`__CFMachPortPerform+0x75
              CoreFoundation`CFRunLoopRunSpecific+0xf51
              CoreFoundation`CFRunLoopRunInMode+0x58
              HIToolbox`RunCurrentEventLoopInMode+0x11b
              HIToolbox`ReceiveNextEventCommon+0x176
              HIToolbox`BlockUntilNextEventMatchingListInMode+0x6a
              AppKit`_DPSNextEvent+0x291
              AppKit`-[NSApplication nextEventMatchingMask:untilDate:inMode:dequeue:]+0x80
              AppKit`-[NSApplication run]+0x31b

Another call stack:

1 com.apple.framework.IOKit 0x98832c8e IOHIDEventCreateWithBytes + 136
2 com.apple.framework.IOKit 0x9882f772 IOHIDEventCreateWithData + 61
3 com.apple.CoreGraphics 0x9b145a47 CGEventCreateFromDataAndSource + 3165
4 com.apple.CoreGraphics 0x9b144d88 CGSDecodeEventRecord + 105
5 com.apple.CoreGraphics 0x9b144d15 event_datagram_handler + 71
6 com.apple.CoreGraphics 0x9b14433a CGSDatagramReadStream::dispatch_next_datagram() + 806
7 com.apple.CoreGraphics 0x9b144007 CGSDatagramReadStream::dispatch_datagrams() + 47
8 com.apple.CoreGraphics 0x9b143e19 CGSDatagramReadStreamDispatchDatagramsWithData + 160
9 com.apple.CoreGraphics 0x9b143b2e CGSSnarfAndDispatchDatagrams + 432
10 com.apple.CoreGraphics 0x9b143864 CGSGetNextEventRecordInternal + 84
11 com.apple.CoreGraphics 0x9b1437cb CGEventCreateNextEvent + 39
12 com.apple.HIToolbox 0x927497bd PullEventsFromWindowServerOnConnection(unsigned int, unsigned char, __CFMachPortBoost*) + 99
13 com.apple.HIToolbox 0x92749726 MessageHandler(__CFMachPort*, void*, long, void*) + 52
14 com.apple.CoreFoundation 0x91fadfe8 __CFMachPortPerform + 440
15 com.apple.CoreFoundation 0x91fade15 __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE1_PERFORM_FUNCTION__ + 53
16 com.apple.CoreFoundation 0x91fadd7b __CFRunLoopDoSource1 + 523
17 com.apple.CoreFoundation 0x91f9e4b2 __CFRunLoopRun + 2130
18 com.apple.CoreFoundation 0x91f9d9ea CFRunLoopRunSpecific + 394
19 com.apple.CoreFoundation 0x91f9d84b CFRunLoopRunInMode + 123
20 com.apple.HIToolbox 0x92740b5d RunCurrentEventLoopInMode + 259
21 com.apple.HIToolbox 0x92740777 ReceiveNextEventCommon + 163
22 com.apple.HIToolbox 0x9278baca AcquireNextEventInMode + 75
23 com.apple.HIToolbox 0x928db9da _AcquireNextEvent + 58
24 com.apple.HIToolbox 0x928c9324 RunApplicationEventLoop + 225

Call stack of event reception by AppKit:

6   com.apple.CoreFoundation           0x00007fff8e1eded8 CFRunLoopRunSpecific + 296
7   com.apple.HIToolbox                0x00007fff99fa1935 RunCurrentEventLoopInMode + 235
8   com.apple.HIToolbox                0x00007fff99fa1677 ReceiveNextEventCommon + 184
9   com.apple.HIToolbox                0x00007fff99fa15af _BlockUntilNextEventMatchingListInModeWithFilter + 71
10  com.apple.AppKit                   0x00007fff8ba0defa _DPSNextEvent + 1067
11  com.apple.AppKit                   0x00007fff8ba0d32a -[NSApplication _nextEventMatchingEventMask:untilDate:inMode:dequeue:] + 454

HIToolbox (Carbon) APIs return the event as an EventRef, which is converted to CGSEventRecord by AppKit using _GetEventPlatformEventRecord() and then to NSEvent using NSEvent _initWithCGSEvent.

NSEvent most likely contains both CGSEventRecord and EventRef: +[NSEvent eventWithCGEvent:] calls CGEventRecordPointer() and CreateEventWithCGEvent() to pass the return values to -[NSEvent _initWithCGSEvent:eventRef:].

Event Lifecycle

The lifecycle of a HID event seems to be:

1. Receive event from X11/Wayland
2. Convert it into a CGSEventRecord
3. Post it to the Mach port returned by CGSGetEventPort()
4. When the event is read with CGEventCreateNextEvent(), convert it into a CGEventRef
5. Carbon then converts CGEventRef into an EventRef and stores it in an internal queue
6. When AppKit calls Carbon to get the next EventRef, it then converts the EventRef into a CGSEventRecord
7. The CGSEventRecord is converted to an NSEvent and passed through further processing

[TheBrokenRail]

Nice to see Wayland being worked on! What is the reasoning behind rewriting the X11 backend as part of CoreGraphics? Do some programs rely on internal CoreGraphics API calls?

[LubosD]

1. Yes
2. The main reason is to also properly integrate CGEvents and later Carbon.

[noah-nuebling]

Not sure this is useful but I just stepped through CreateEventWithCGEvent() and looking at the argument registers x0 - x8, one of the arguments is of type HIDEvent *, also known as its __bridged type IOHIDEventRef. Maybe that's the type of the event being created. You can find the source code for HIDEvent * in Apple's open source IOKit code.