iOS/tvOS New GLKit Window Controller / Renderer

[danoli3]

Support for GLKit

Moving forward with Apple's GLKit view and window controller for optimised OpenGL ES 2.0 + rendering and window control.

I've wrapped and kept the Core Animation Windowing system for legacy reasons, however I think the GLKit View much faster and the draw/update loop is controlled directly from the core rather than our own timers. Currently the Update/Draw was called within the same timer loop. When using the GLKit View Controller this is split into seperate Update and Draw from the Core Rendering Loop from within GLKit.

• This new GLKit Window Controller supports 120 fps iOS rendering (new generation of devices).
• New MSAA rendering (4x only)
• Support for Different Color Formats
• Support for Depth16 / Depth 24
• Support for Stencil
• Last frame UIImage* usable from Update
• Allows for GLKTextureLoading on seperate GL Sharegroup context
• iOS 5+

How to Use

settings.glesVersion = OFXIOS_RENDERER_ES2; // type of renderer to use, ES1, ES2, etc. settings.windowControllerType = ofxiOSWindowControllerType::GL_KIT; // or CORE_ANIMATION settings.colorType = ofxiOSRendererColorFormat::RGBA8888; // or RGB565, SRGBA8888 settings.depthType = ofxiOSRendererDepthFormat::DEPTH_NONE; // or DEPTH_16, DEPTH_24 settings.stencilType = ofxiOSRendererStencilFormat::STENCIL_NONE; // or STENCIL_8

Defaults still use CORE_ANIMATION so nothing will change to that Window Controller Type.

Tests

• Regression Tested Examples
• Tested private game projects both window controllers

[arturoc]

I'm not sure if merging this right away or waiting after we release 0.10. This seems to be quite a big change so i'm a bit concerned of merging it right before the release without much time to be tested by others but it also seems and important feature that we want to have so not sure. I'll let you decide @danoli3 feel free to merge it if you think it's important enough to have it.

[danoli3]

Since the old window controller is still the default and works exactly the same.
I do think this is good to merge for 0.10 as all old examples and even the example project will still use the old way, unless the developer wants to set these values to gain this power.

I did plan for this to be for the future release, however I've been testing it consistently for 2 weeks and it's very solid. Lets push making this the default window controller to 0.11 though.

[danoli3]

I'll do some more regression tests first just to make sure all good with old projects.

[danoli3]

Okay yeah found 1 issue with Custom App Delegate Projects with the Frame Init. All fixed.

So I think this is good to merge.

The only issue I can see people having is missing GLKit.framework from their old projects so need to make sure the project builder adds that. @ofTheo @arturoc does that generate by just parsing the Core Frameworks from the core project?

[danoli3]

Project Error without GLKit Framework will be:

Undefined symbols for architecture arm64:
  "_OBJC_METACLASS_$_GLKView", referenced from:
      _OBJC_METACLASS_$_EAGLKView in libofxiOS_iphoneos_Debug.a(EAGLKView.o)
  "_OBJC_CLASS_$_GLKView", referenced from:
      _OBJC_CLASS_$_EAGLKView in libofxiOS_iphoneos_Debug.a(EAGLKView.o)
  "_OBJC_METACLASS_$_GLKViewController", referenced from:
      _OBJC_METACLASS_$_ofxiOSGLKViewController in libofxiOS_iphoneos_Debug.a(ofxiOSGLKViewController.o)
  "_OBJC_CLASS_$_GLKViewController", referenced from:
      _OBJC_CLASS_$_ofxiOSGLKViewController in libofxiOS_iphoneos_Debug.a(ofxiOSGLKViewController.o)
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

[arturoc]

if you need projects to use a new frameworks it should go in the CoreOF.xcconfig file: https://github.com/openframeworks/openFrameworks/blob/master/libs/openFrameworksCompiled/project/osx/CoreOF.xcconfig

if it's there it'll work even for old projects

[danoli3]

Oh my... yeah it seems we totally missed that line on iOS / tvOS... I'm going to add this now and not cry too much in shame ;P!

[hamoid]

@danoli3 Hi :) I'm about to release an iOS app.

I think the GLKit View much faster

Have you measured the performance difference? Should we merge this in our project before releasing?

[danoli3]

@hamoid Yeah not only is it faster, the input system seems to be more responsive. I would definitely recommend trying it out and comparing. Should not too difficult just change the Main.mm commands.

One major difference is GLKit only supports 4xMSAA or none so keep that in mind if you had custom AA code for different devices.

[danoli3]

Okay I think this is good to merge.

[arturoc]

thanks!

[thinium]

Hi @danoli3, I just tried to use the GLKit window controller with the latest master by adding this setting:
settings.windowControllerType = ofxiOSWindowControllerType::GL_KIT;

However, it crashes when trying to get the window size:

the instance in [ofxiOSGLKView getInstance] is actually a ofxiOSEAGLView, and therefore it crashes.

I noticed that it's still creating a EAGLView, but not a GLKView:

How do I correctly use the GKKit window controller in OF? Thanks!

[danoli3]

Oh hey, using a custom App Delegate yeah? I think I'll write up a quick example for this case. Thanks! I'll do this in the next coupe of hours

…

On 10 April 2018 at 19:39, mfl ***@***.***> wrote: Hi @danoli3 <https://github.com/danoli3>, I just tried to use the GLKit window controller with the latest master by adding this setting: settings.windowControllerType = ofxiOSWindowControllerType::GL_KIT; However, it crashes when trying to get the window size: [image: screen shot 2018-04-10 at 11 22 06] <https://user-images.githubusercontent.com/1383198/38548941-4d97af00-3cb3-11e8-825e-26d9a2ef3fd7.png> the instance in [ofxiOSGLKView getInstance] is actually a ofxiOSEAGLView, and therefore it crashes. [image: screen shot 2018-04-10 at 11 21 28] <https://user-images.githubusercontent.com/1383198/38549061-8b8f5e2a-3cb3-11e8-821c-1c540ef2d2f1.png> I noticed that it's still creating a EAGLView, but not a GLKView: [image: screen shot 2018-04-10 at 11 37 43] <https://user-images.githubusercontent.com/1383198/38549140-b2ca4cca-3cb3-11e8-85f7-9ca3e2088c13.png> How do I correctly use the GKKit window controller in OF? Thanks! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#5928 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAytHOXQv_HWklAdFUpk-Cec_OCZyf9_ks5tnH3jgaJpZM4SlFzA> .

[thinium]

Yes, good catch :) Thanks a lot and looking forward to the example! On Tue, Apr 10, 2018 at 12:11 PM, Daniel Rosser <notifications@github.com> wrote:

…

Oh hey, using a custom App Delegate yeah? I think I'll write up a quick example for this case. Thanks! I'll do this in the next coupe of hours On 10 April 2018 at 19:39, mfl ***@***.***> wrote: > Hi @danoli3 <https://github.com/danoli3>, I just tried to use the GLKit > window controller with the latest master by adding this setting: > settings.windowControllerType = ofxiOSWindowControllerType::GL_KIT; > > However, it crashes when trying to get the window size: > [image: screen shot 2018-04-10 at 11 22 06] > <https://user-images.githubusercontent.com/1383198/ 38548941-4d97af00-3cb3-11e8-825e-26d9a2ef3fd7.png> > > the instance in [ofxiOSGLKView getInstance] is actually a ofxiOSEAGLView, > and therefore it crashes. > [image: screen shot 2018-04-10 at 11 21 28] > <https://user-images.githubusercontent.com/1383198/ 38549061-8b8f5e2a-3cb3-11e8-821c-1c540ef2d2f1.png> > > I noticed that it's still creating a EAGLView, but not a GLKView: > [image: screen shot 2018-04-10 at 11 37 43] > <https://user-images.githubusercontent.com/1383198/ 38549140-b2ca4cca-3cb3-11e8-85f7-9ca3e2088c13.png> > > How do I correctly use the GKKit window controller in OF? Thanks! > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <https://github.com/openframeworks/openFrameworks/ pull/5928#issuecomment-380038411>, > or mute the thread > <https://github.com/notifications/unsubscribe- auth/AAytHOXQv_HWklAdFUpk-Cec_OCZyf9_ks5tnH3jgaJpZM4SlFzA> > . > — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#5928 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABUbHlwe40IiTUs24PY3L9N2d7HewVAoks5tnIVdgaJpZM4SlFzA> .

[danoli3]

So basically, for a custom delegate and view controller you will need to get them to inherit from the GLKViewController like so:

#import "ofxiOSGLKViewController.h"

@interface SquareAppViewController : ofxiOSGLKViewController

@end

Then when setting up the frame

@implementation SquareAppViewController

- (id) initWithFrame:(CGRect)frame app:(ofxiOSApp *)app {
    
    ofxiOSGetOFWindow()->setOrientation( OF_ORIENTATION_DEFAULT );   //-- default portait orientation.    
    
    return self = [super initWithFrame:frame app:app sharegroup:nil];

[danoli3]

@ukelady #5962

[thinium]

@danoli3 Thanks for the example! I managed to get it build. But unfortunately it shows a black screen, the cpu at 100%. In the console it doesn't show any error message so it's hard for me to pin point what the issue is. Any suggestions how to debug this issue?

ps. I've tested the normal version with latest master. It's working fine.

[danoli3]

Black screen? Not really sure what could cause that. I haven’t experienced this issue, would you mind sharing me with the repo or emailing me the source code so I may dig deeper. danoli3 at gmail

…

On Wed, 11 Apr 2018 at 2:57 am, mfl ***@***.***> wrote: @danoli3 <https://github.com/danoli3> Thanks for the example! I managed to get it build. But unfortunately it shows a black screen. In the console it doesn't show any error message so it's hard for me to ping point what the issue is. Any suggestions how to debug this issue? ps. I've tested the normal version with latest master. It's working fine. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#5928 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAytHBZg_iOBAxjcLz6Lv6Ot6Txp74sOks5tnOR7gaJpZM4SlFzA> .

[thinium]

Hi @danoli3, thanks a lot for willing to take a look! I've narrowed down the issue into a tiny example (attached). In the app you should see a red triangle and some texts. If you un-comment the #define in TriangleApp.mm, line 4, you'll see a black screen.
The cause of the black screen is that we use FBO in draw function.

iosNativeExample.zip

Thanks again!

[thinium]

Hi @danoli3, have you got a chance to look into the example?

[hamoid]

@danoli3 We figured out the issue with @ukelady: drawing to the fbo inside the draw() method instead of the update() method. This is not an issue in Linux nor in iOS pre-GLKit. Somehow GLKit does not allow drawing to the fbo inside draw(). We discuss it here and may open a different issue, if it is actually an issue.