Support OpenGL ES

[einsteinx2]

On the projectM website, it says that it has been ported to OpenGL ES 2.0 for Android and iOS use. Has that code ever been released anywhere? All I see in the repo seems to be ES 1.0 support and so far I've only been able to get that to half work in a test iOS app (it renders but not correctly).

Is the ES 2.0 version of the library closed source?

[revmischa]

The android and iOS ports are unfortunately closed-source and I don't have the code. I tried to port to ES myself but I don't really have the chops or time to complete it. If you want to continue the effort I'd be happy to point you in the right direction. Sorry.

[revmischa]

Notes on the old openGL immediate-mode calls that need to be ported are here: https://github.com/projectM-visualizer/projectm/blob/gles/glES-porting.txt

If anyone wants to work on this, please be my guest!

[einsteinx2]

Damn I was hoping that wouldn't be the case.

I actually have a "working" version using ES 1.0 and an iOS test app here: https://github.com/einsteinx2/libprojectm-ios. I even backfilled some missing rendering functions and got it compiling properly as an iOS static library. However, I'm totally stuck on some rendering issues.

I posted to the /r/opengl subreddit, but didn't get any answers, however I have a detailed description there: https://www.reddit.com/r/opengl/comments/6kyvk6/looking_for_help_porting_projectm_to_ios_about_90/

Also, here is a video of the rendering differences between the Mac SDL version and my iOS version: https://www.youtube.com/watch?v=6otFBzkbd3Y

If you have a little OpenGL experience, maybe you can figure out what's wrong. I feel like I'm 90% there and am just missing something simple with the rendering.

It doesn't seem to be blending properly and also isn't rendering at the correct resolution. If I could at least get a working iOS test app using the ES 1.0 code, I could try and expand it from there.

[einsteinx2]

Also possibly of interest, the latest Milkdrop 2 source code has been open sourced: https://sourceforge.net/projects/milkdrop2/

However it's written in DX9, so it would require porting to OpenGL first to be used anywhere but Windows. Though I did find Valve's ToGL DX9 > OpenGL wrapper here: https://github.com/ValveSoftware/ToGL so maybe there's hope without rewriting the whole thing.

Ideally it would be even cooler to get the actual MilkDrop source running, as ProjectM doesn't render exactly the same if I recall correctly.

[revmischa]

Very cool! That looks like good progress. Send me a PR of any improvements you can manage to make. I'd love to ditch the immediate-mode GL calls. I have lots of ideas for uses for projectM but they all require ES.

I think the main difference between milkdrop and projectM is that certain waveforms aren't supported in projectM? Adding the missing waveforms and/or improving shader support might be a lot easier than porting milkdrop2. I don't know this for sure at all though. projectM is mostly compatible with milkdrop, just needs a few missing things filled in, far as I'm aware.

@struktured @psperl any thoughts?

[einsteinx2]

I’m basically stuck at the output shown in the video. I’ve spent hours since then getting nowhere fast.

If any of you could take a quick stab at seeing what the issue is, that would be amazing. I have a feeling it’s something simple, considering that it’s essentially exactly the same code as the Mac version.

I’m not even sure if the problem is in the library itself or how I’m setting up the GLView. I tried getting it working using SDL for iOS to use the exact same code and remove one variable from the equation but that didn’t render at all.

If you guys have a bit more GL experience, maybe you can spot a problem in my view setup or something obvious in the changes I made to get it to compile?

I also have another branch where I tried using an OpenGL translation layer called Regal to skip porting to ES at all, but while it compiles, it crashes when rendering.

At this point I’m 100% stuck. Each of my experiments is in a different branch, with the master branch being the standard ES 1 port.

[revmischa]

Okay I don't really know much about OpenGL.
What specifically is the issue?

[einsteinx2]

@revmischa I'm in the same boat regarding OpenGL knowledge, however I'm just starting to read through this online book which is already proving to be fantastic: https://paroj.github.io/gltut/

Unfortunately, I don't really know what's wrong. The best description would be in this youtube video: https://www.youtube.com/watch?v=6otFBzkbd3Y

You can see that the iOS side is rendering, however it's not displaying the background or blending the layers properly. Also it's doesn't seem to be respecting the window size values I'm providing it.

I wish I knew more... For now I have to set this aside as I got a bit hooked on it and was ignoring real work I have :)

But I'm going to continue to read up on OpenGL and computer graphics and maybe in the future I can contribute more.

[elvetemedve]

Wouldn't it make more sense to port it to the new Vulkan API rahter than OpenGL ES?

[revmischa]

I think that'd be sweet @elvetemedve! What would be involved in that? Would there be any compatibility issues with older systems? I thought Vulkan looked cool but I have zero actual experience with it. Not that I have barely any with OpenGL.

[revmischa]

Someone did in fact say they got OpenGL 1.2 and projectM on a raspberry pi recently so maybe there's hope

[elvetemedve]

It's just an idea. Unfortunately I'm not an expert, but since Vulkan is multi platform (ranging from embeded devices to desktop PCs) supporting various devices / OS-es could be easier.

Backward compatibility would be an issue, as old hardware won't get Vulkan support.

[revmischa]

It's a cool idea. I just don't know what's involved in it

[StefanBruens]

@elvetemedve The port to OpenGL ES is a port from a immediate mode API to a retained mode API. OpenGL 1.x is a immediate mode API, while every current API (OpenGL >= 3, OpenGL ES >= 2, Vulkan, ...) is a retained mode API.
If you have done the port to OpenGL ES, you have done the big conceptual change, everything else is comparatively minor.
The RPI hardware supports OpenGL 2.0, which is true for a lot of embedded hardware, but lots of other embedded drivers do not support non-ES OpenGL.

[revmischa]

This is mostly academic since there's no OpenGL ES port... yet... hopefully someone can figure that out!
I don't think there's a lot of work to be done. There are very few old calls that need updating.

[einsteinx2]

If anyone can pick up where I left off on the ES 1.0 port at least we'd have something that can be included in Android and iOS apps. It was relatively easy to get going, it just doesn't work correctly. Unfortunately I never made any more progress since then.

I also tried other options like shims that supposedly allow desktop OpenGL to run on iOS, but couldn't make any progress there either, so I gave up completely.

Maybe one of these days haha. I just want to add it as a visualizer to my (now free) open source iOS music app called iSub (https://github.com/einsteinx2/isub3). I've stopped developing it, but it still works and is used by many people including myself and I've always wanted to have ProjectM as a visualizer option (currently it just has a crappy visualizer I wrote, or rather adapted from some example code).

[revmischa]

There is now GLES support! #70
Closing this. Give it a try.

[einsteinx2]

@revmischa whoa super cool! When I get some time I'll give it a go!