Oculus Rift support

[Camron]

The SDK is available after registering at https://developer.oculusvr.com/

Development info can be found here:
https://developer.oculusvr.com/forums/
http://www.mtbs3d.com/phpBB/

[mrwonko]

Yeah, I had already been planning to do this at some point. I don't have one myself yet, I'm likely waiting for the consumer version, but that need not stop me from adding support. It'll be easier if my uni gets a Dev Kit (quite possible, here's hoping), but I'm sure it can be done with the help of some outside tester as well.

Be aware that this would also require mod support for the head tracking to work. A VR mod could surely be part of OpenJK, but it should (for obvious reasons) not be mandatory.

Oh, and I'd also like to add Razer Hydra support. At least I've already got one of those. And how cool is a motion controlled lightsaber? Yeah...

[mrwonko]

But keep in mind that this game was not designed with VR in mind. There are a lot of open questions:

• HUDs don't work too well in VR. But how do you convey active force, amount of force power left, health and selected item without a HUD?
• Decoupling view and aim, or not doing it. With the Razer Hydra it's fairly simple - right analog stick for turning horizontally, head movement for looking around, aiming via... actually aiming. But the Hydra should not be mandatory...
• Cutscenes. They don't really work in VR. Ideally we'd rescript all of them to allow them to be experienced from a character's point of view. This is easier (although still a lot of work since you're often not supposed to look around, parts of the level may be missing) when the player is actually present, and more tricky when it's taking place elsewhere.
• Losing control over head movement (outside of cutscenes). I'm thinking of the medium style forward jumping flip attack and rolling in particular - that could be quite nauseating. But players can always just avoid it, and with motion control at least the flip wouldn't even be available. (Just doing motion control properly is probably a tough thing in and by itself.)

Most of this will likely be solved by trying out a bunch of stuff and seeing what works. Also in other games. By the time I get started on adding this there will likely already be some games with support out. TF2 already is. We can learn from that.

[xScooper]

I'm actually waiting for mine to arrive. And this is something I wast intending to look into.

[cadika-orade]

@mrwonko

• HUDs can work in VR with some adjustment. In another issue it has been suggested to make the HUD elements adjustable. Making the scale, relative location, and stereoscopic rendering depth configurable would probably suffice. The effect would be similar to Iron Man's HUD. Example: http://ironman3news.com/wp-content/gallery/iron-man-3-hud/iron-man-3-hud.jpg As a side note, this would help me fulfill my personal agenda to use the Oculus to make a decent Iron Man simulator game.
• Control with the Hydra would indeed be easy, but there are other means. I see no reason why aiming couldn't be done with the mouse, where the mouse controls your aim relative to where you are looking. I am personally hoping to see Wii remote support, though. Kinect+Wii-mote support would be my ideal, but I'm not getting my hopes too high yet.
• I see what you mean with cutscenes. Personally, I'd be alright with the view switching to 3rd-person at these points, so long as support for 1st-person cutscenes is included for custom levels. I would expect someone to go through and patch up the official levels to allow 1st-person cutscenes.
• Err... I do see the issue here. Perhaps if the camera simply didn't flip during certain animations? For animations that are triggered intentionally, I don't see much reason to have the camera spinning around at all. Getting knocked over or otherwise being tossed in an unwanted way would logically force your view around and would definitely make lightsaber battles more realistic.

As you mention, TF2 is already pioneering the realm of VR FPS action gaming. They are already a model for this burgeoning corner of the industry.

[mrwonko]

HUDs can work in VR with some adjustment.

Possibly, but no HUD at all is usually superior. But I agree, a HUD like the one in Crysis 2 would work well.

Control with the Hydra would indeed be easy, but there are other means.

Sure there are, and I don't want to make the Hydra mandatory. Just saying I'd like to see that as well.

I would expect someone to go through and patch up the official levels to allow 1st-person cutscenes.

You realize how much work that would be? Maybe someone will, but I wouldn't count on it.

[cadika-orade]

Yes, it would be a lot of work. Fortunately, much of it is already done and there's still a community of die-hard modders and artists for this game.

I am working on putting together a collection of replacement assets that will effectively replace the base assets. There are many little "tweak" mods that include custom models, textures, and maps. It will probably take ages, but eventually I think we can do it. They did it for DOOM II and Quake.

[mrwonko]

@cadika-orade I sure hope you're getting permission from all the modders whose work you're using. Also, using existing models could be a little hard since you'll have to redo the skeleton and if you don't get it quite right you'll get glitchy animation.

Keep in mind that Doom 2 and Quake did not have 20000+ frames of animations.

[cadika-orade]

Oh absolutely. I wouldn't steal someone else's artwork. I'm trying to involve the modders directly, if possible.

It will be a challenge, yes, but it is possible. If nobody tries, it will never happen.

[mrwonko]

Oh absolutely. I wouldn't steal someone else's artwork.

Good, because that seems to be the latest trend. Well, Tim already started a couple of years ago, but yeah... Well, good luck with that! You should get in contact with the other people who are looking into free assets, see here, for example.

[cadika-orade]

Nifty! Thank you!

Sorry I have so greatly derailed this page.

[mrwonko]

Just a quick update: Oculus Rift support is impossible since we're on the GPL and the Oculus SDK is GPL-incompatible. Welcome to the world of copyleft.

[mrwonko]

Unless we somehow circumvent this via some elaborate separate program communicating over sockets. I shall try that at some unspecified point in the future.

[eezstreet]

Or a DLL

Sent from my Windows Phone

---

From: Willi Schinmeyermailto:notifications@github.com
Sent: ‎4/‎17/‎2013 10:15 AM
To: Razish/OpenJKmailto:OpenJK@noreply.github.com
Subject: Re: [OpenJK] Oculus Rift support (#83)

Unless we somehow circumvent this via some elaborate separate program communicating over sockets. I shall try that at some unspecified point in the future.

---

Reply to this email directly or view it on GitHub:
#83 (comment)

[mrwonko]

Nah, you know about the problems with DLL, the result is a single program, not two, so the GPL spreads.

[eezstreet]

Not if you call LoadLibrary() on it.

Sent from my Windows Phone

---

From: Willi Schinmeyermailto:notifications@github.com
Sent: ‎4/‎17/‎2013 10:42 AM
To: Razish/OpenJKmailto:OpenJK@noreply.github.com
Cc: eezstreetmailto:eezstreet@live.com
Subject: Re: [OpenJK] Oculus Rift support (#83)

Nah, you know about the problems with DLL, the result is a single program, not two.

---

Reply to this email directly or view it on GitHub:
#83 (comment)

[mrwonko]

Even then if you share data structures. See the GPL FAQ on gnu.org

[eezstreet]

I never said that the DLL was going to share data structures, nor did I imply it.

Sent from my Windows Phone

---

From: Willi Schinmeyermailto:notifications@github.com
Sent: ‎4/‎17/‎2013 10:53 AM
To: Razish/OpenJKmailto:OpenJK@noreply.github.com
Cc: eezstreetmailto:eezstreet@live.com
Subject: Re: [OpenJK] Oculus Rift support (#83)

Even then if you share data structures. See the GPL FAQ on gnu.org

---

Reply to this email directly or view it on GitHub:
#83 (comment)

[mrwonko]

No, that was me thinking you were talking about just taking the existing library runtime and LoadLibrary()ing that.

[eezstreet]

I was going to use a middleware lib of sorts with an import table that flips endianness or something

Sent from my Windows Phone

---

From: Willi Schinmeyermailto:notifications@github.com
Sent: ‎4/‎17/‎2013 12:31 PM
To: Razish/OpenJKmailto:OpenJK@noreply.github.com
Cc: eezstreetmailto:eezstreet@live.com
Subject: Re: [OpenJK] Oculus Rift support (#83)

No, that was me thinking you were talking about just taking the existing library runtime and LoadLibrary()ing that.

---

Reply to this email directly or view it on GitHub:
#83 (comment)

[eezstreet]

Any more info as to the socket communication?

[Fredz66]

You could also use the OpenHMD library which supports the Oculus Rift, is available for Windows, OS X and Linux and is published under the Boost license, which is compatible with the GNU GPL.

[mrwonko]

It's just a shame OpenHMD doesn't do a proper clean-room approach and could thus get into legal trouble...

[Fredz66]

Why do you think they didn't do a clean-room approach ? And if it's the case, how is it a problem ?

[mrwonko]

I think they didn't do it because they admitted to it. It's a problem in that Oculus claims copyright on derivates of the SDK.

[Fredz66]

Thanks for the pointer. I suppose they didn't simply copy/paste the code and translated into C, that would be quite stupid of them. Anyway I doubt it since their code is multi-platform.

If it's their own code it can't be considered as a derivative of the SDK, and after all it's basically only an implementation of sensor data reading from an USB HID device and a warping shader. Both things are pretty straightforward to implement and the only unknown is the packet format.

I can understand they didn't feel like sniffing a USB port to try-guess it. And even then, I don't think it's illegal in most countries, it's not in the US and Europe at least.

[sigboe]

Sorry if this is not possible, but what about looking into this 3D driver to integrate functionality directly into the game.
The driver is opensource, and is specifically made to retrofit games to work with Oculus Rift.

http://vire.io/

[eezstreet]

I've been looking into a pipe API that has support for plugins. We could possibly use this for Oculus Rift support.

[mrwonko]

Here's what I'm currently thinking about this: Valve apparently has a nice VR API in the Steamworks SDK which would be better than just using the Oculus SDK, since that only supports Oculus HMDs. We're not on Steam, but they're planning to open to everybody in the future, at which point we could look into this.

That would require Steam for VR, that may be a problem for some?

[eezstreet]

But will it support GPL?

Sent from my Windows Phone

---

From: Willi Schinmeyermailto:notifications@github.com
Sent: ‎3/‎8/‎2014 5:39 AM
To: JACoders/OpenJKmailto:OpenJK@noreply.github.com
Cc: eezstreetmailto:eezstreet@live.com
Subject: Re: [OpenJK] Oculus Rift support (#83)

Here's what I'm currently thinking about this: Valve apparently has a nice VR API in the Steamworks SDK which would be better than just using the Oculus SDK, since that only supports Oculus HMDs. We're not on Steam, but they're planning to open to everybody in the future, at which point we could look into this.

That would require Steam for VR, that may be a problem for some?

---

Reply to this email directly or view it on GitHub:
#83 (comment)

[ensiform]

Leave this to a 3rd party closing.

[shinyquagsire23]

So with the OR's second revision being out, I think it might be a good idea to discuss the topic of Oculus rift integration once again. Unfortunately, it hasn't changed it licence to anything which is GPL compatible as of now, but it seems a few projects like libvr and OpenHMD have popped up, which allow a GPL compatible alternative to the default SDK. Would it be possible to use these libraries to allow Oculus Rift support, or have the plans for VR support been scrapped entirely?

Even if we can't get the stuff for head tracking and such in, it might be worthwhile to implement a VR shader so that it can get the stereoscopic vision in at least. OpenHMD seems to be our best bet in terms of Rift support, since it's licenced under the Boost licence which is compatible with the GPL license given by Raven. It even has some neat demos, with one being a nice little barebones OpenGL demo which looks fairly simple.

[ensiform]

The master repo has no plans for it at this time.

[sigboe]

I don't know if you have a Rift, but without head tracking there is really no point in implementing it. It is really uncomfortable use the Rift without head tracking. On devices like the Sony H series that has like 30 or 40 degrees of fov its okay, but the rift has closer to 100 degrees, and its really uncomfortable.

[eezstreet]

If the license isn't GPL-compliant, then we're not going to bother with it, sorry.

Sent from my Windows Phone

---

From: Sigurd Bøemailto:notifications@github.com
Sent: ‎8/‎7/‎2014 12:22 AM
To: JACoders/OpenJKmailto:OpenJK@noreply.github.com
Cc: eezstreetmailto:eezstreet@live.com
Subject: Re: [OpenJK] Oculus Rift support (#83)

I don't know if you have a Rift, but without head tracking there is really no point in implementing it. It is really uncomfortable use the Rift without head tracking. On devices like the Sony H series that has like 30 or 40 degrees of fov its okay, but the rift has closer to 100 degrees, and its really uncomfortable.

---

Reply to this email directly or view it on GitHub:
#83 (comment)

[shinyquagsire23]

@eezstreet
The Boost license in OpenHMD is GPL compatible, hence why I brought to this up again. It'd be kinda pointless to revive this just to get it shot down again due to GPL licensing :P

[xycaleth]

I don't know if you have a Rift, but without head tracking there is really no point in implementing it. It is really uncomfortable use the Rift without head tracking

@sigboe The DK2 does have head tracking. That's why there's an extra little camera to track points on the Rift.

[sigboe]

@xycaleth I know, I have both DK1 and DK2, I was referring to @shinyquagsire23 who said that even if one didn't get to play with head tracking it would be better than nothing. In which case my stance is that playing without head tracking is nauseating, I mean I am well seasoned with simulator sickness, but if I lose head tracking for more than a little while I remove the headset out of discomfort.

[sigboe]

https://github.com/xLAva/JediAcademyLinux/tree/HmdSupportGPL

[shinyquagsire23]

I think I remember seeing that repository earlier. He did a lot of modifications to get that working iirc. Which reminds me of something I've been meaning to ask. Would it be better to create an all new renderer for VR (ie new renderer module, separate from vanilla or rend2) or just add in VR mode rendering with a cvar setting and s few checks in places?

And yes @sigboe, I forgot how bad simulation sickness can be with VR, and I don't think forced head movements are very friendly in that area. I actually recall that repository removing any forced movements for cut scenes and player actions to prevent sickness, so that might be good to keep that in mind.

Also, there seems to be one slight shortsight I had with OpenHMD, and it's the lack of DK2 support. I believe the devs are planning support, but at the moment they are still waiting for their DK2's to arrive.

[ensiform]

Alas it also looks like from his repo, that it cannot be fully accomplished without mod changes.

[Grix]

I for one would love DK2 support.

[ensiform]

Being that we have already closed the issue, I don't expect that you'll see it being something we will look into for the master repo.

[shinyquagsire23]

I guess Ensiform has a point in that even if it was considered, I don't believe many of the main developers have a interest in implementing it (or a Rift to test it with). I might take a poke at it when I get my DK2 in a few months, but I can't exactly make many guarantees since my C/C++ is lacking a bit (I mostly do Java stuff and embedded C/ASM).

[shinyquagsire23]

As much as I know how much you guys hate me reviving this, but I figured I'd at least give a small status report on things. I've basically concluded that the chances of this actually getting into OpenJK are very slim to none due to the sheer amount of hacks and messy code it took to get it working in the first place. However, that being said, I managed to port xLava's HMD code over to OpenJK as well as filling in the skeleton code xLava had for the official oculus SDK. I initially tried to get it working under OpenHMD, but I found two things:

• The rotation tracking for DK2 drifts. A lot.
• There isn't a proper warp mesh for DK2 (and pixel shaders cannot be used)

Implementing a proper warp shader was pretty much impossible for me, so I used the official SDK and managed to get it working. Kinda. It only rendered one eye, and it seems the issue was that on Linux the 32 bit binaries for the libOVR library don't properly render the right eye. So I ported most of xLava's work over here on my own personal branch (xLava's branch doesn't have x64 support) and the right eye works now.
(I just realized that the screenshot function doesn't pull both eyes, Look like that'll need fixing. Either way it works.)

For now I think the best option for me (or any other VR branches looking to use the Oculus SDK instead and disregard GPL) is to just make a fork of OpenJK similar to what the developers did over at Dolphin-VR and remain unaffiliated with the official project. Or if we're lucky, we could try and get Raven to grant use of the Rift SDK within the GPL (although I think we went over this when we tried switching licenses). I'll probably keep maintaining my VR branch for personal use, and I'll keep it git-cherrypick-able in case anyone wants to implement it in their own projects they have.

TLDR; Rift SDK is a mess to get working, I have it working, but I'll keep it out of the official OpenJK branch. OpenHMD isn't ready for official support yet, but if it does get to a good position I'll consider porting it for the official OpenJK repo (if you guys want).

[HEXcube]

@xLAva (Jochen Leopold) has added Occulus Rift DK2 support to Jedi Academy. Check out the latest release - https://github.com/xLAva/JediAcademyLinux/releases . For source code, see the DK2 branch - https://github.com/xLAva/JediAcademyLinux/tree/HmdSupportDK2 . Looks like Virtual Reality is the "way of the Force"! :-)

[Grix]

Fantastic!