-
Notifications
You must be signed in to change notification settings - Fork 21
Virtual Reality HMDs, TrackIR, HAT Track
Jump to TrackIR
Jump to HAT-Track
GlovePIE supports two kinds of Virtual Reality head mounted displays. The Vuzix (previously known as Icuiti) VR920, and the eMagin Z800.
The Trimersion could theoretically also be used with GlovePIE, if you have Windows 2000, XP, or Vista because GlovePIE supports reading multiple mice and multiple keyboards individually, and the Trimersion acts as a mouse and keyboard.
The VR920 has 3 accelerometers, and 3 magnetometers (a compass). The accelerometers are for measuring gravity which tells you pitch and roll, and the compass is for measuring yaw.
The Z800 also has 3 accelerometers, and 3 magnetometers (a compass). But unlike the VR920, it also has 3 rate gyros which help make the other measurements more accurate.
The main values that you will use are the yaw, pitch, and roll. Which are angles in degrees.
Yaw is the angle when you turn your head left and right. Positive means to the right. Pitch is the angle when you look up and down. Positive means up. Roll is the angle when you tilt your head to the side. Positive means to the right.
Normally you will want to either emulate the mouse for mouse-look games (either first person, or third person ones), or emulate TrackIR for games that support TrackIR (mostly flight simulators, or car racing games, but some other types of games).
For a list of TrackIR enabled games, see here: http://www.naturalpoint.com/trackir/03-enhanced-games/enhanced-games-all.html (Note from Ravbug: Carl Kenner has very strong political beliefs, so the license of this program forbids you from playing America's Army. Kenner writes, "America’s Army is not a game, it is a military recruitment drive. Real people die because of it.")
In mouse-look games, it is not possible to get head-tracking to work perfectly. Almost all mouse-look games use some sort of acceleration based on how fast the mouse is moving, and you can’t switch it off. They also often cap the maximum amount that you can move the mouse in a certain period. These mean that when you turn 90 degrees in one direction and then turn back 90 degrees, you will not end up looking exactly the same direction you were at the start, only nearly. Also when you look up and down, the vertical angle in game can get out of sync with the angle of your head. Ideally you want the game to have a key which re-centres the view.
See the VR920Scripts folder for examples of TrackIR emulation, and mouse-look emulation.
You can read the following angles from the VR920:
VR920.yaw, VR920.pitch, and VR920.roll
Yaw will be between -180 and +180. Pitch will be between -90 and +90. Roll will be between -180 and 180.
There is also VR920.ContinuousYaw which is between –infinity and +infinity. It keeps counting when you keep turning in the same direction. So if you spin around twice, it will go up to 720 degrees.
For First Person Shooters you should always use VR920.ContinuousYaw.
These values are all read straight from the VR920 SDK supplied by Vuzix. These values will shake a little bit, because the sensors are not precise. You also need to run the iWear Calibration program that came with your VR920. It is very important that it is calibrated correctly.
You may want to use GlovePIE’s Smooth function. It averages several frames together. This causes lag, but it also makes it smoother. For example:
Var.yaw = smooth(vr920.yaw, 10)
will smooth the current frame by averaging it with the previous 10 frames. You can choose how many frames to use. The maximum is 30.
Don’t use the SmoothPitch, and SmoothRoll values for smoothness. They are not actually smoothed, they are the same as the Wiimote.SmoothPitch and Wiimote.SmoothRoll, and are calculated from the acceleration, making the false assumption that all acceleration is due to gravitỵ.
GlovePIE also lets you read the VR920’s acceleration and compass sensors directly. This completely bypasses the SDK, and reads the values straight from the USB port. Before this will work, you need to choose VR920 Accelerometer Calibration from the CP-Settings menu, and calibrate the accelerometers. This calibration only affects GlovePIE and has nothing to do with other programs that use the VR920.
RawForceX, RawForceY, and RawForceZ give you the raw, uncalibrated accelerometer values. You don’t need to use the accelerometer calibration for this to work. RawMagnetX, RawMagnetY, and RawMagnetZ give you the raw compass values. Note that the Z value will always be high, presumably due to the magnets in the VR920 earphones.
gx, gy, and gz give you the calibrated accelerometer values. This includes both the opposite of gravity plus the real acceleration of the glasses. They will be zero for no force, or +/-1 for 1G of force. If you drop the glasses (not recommended!!) all the values will be zero. But if you are holding them up, then gy will be 1 and gx and gz will be 0.
RawAccX, RawAccY and RawAccZ are the same as gx, gy, and gz, but are measured in metres per second per second, instead of Gs. So they will be 9.8 m/s/s for 1G of force.
RelAccX, RelAccY, and RelAccZ are intended to be just the acceleration with the force of gravity filtered out. This is not completely possible, and thus not completely accurate. But it gives you a reasonable approximation of the acceleration of the glasses. It is measured in 9.8 m/s/s. Unfortunately the acceleration of the glasses is normally caused by turning your head, rather than by moving your head.
StrictPitch, and StrictRoll are the pitch and roll of the glasses calculated by GlovePIE from the accelerometers. It does not use the SDK, and is only affected by GlovePIE’s accelerometer calibration, not by the iWear calibration. It is only calculated when the glasses are not accelerating and thus the total force is about 1G.
SmoothPitch and SmoothRoll are the pitch and roll, but they are updated (wrongly) even when the glasses are accelerating. This will produce a slightly wrong answer when the glasses are accelerating, but it will be smoother instead of jerky.
BedYaw and BedPitch are the equivalent of Yaw and Pitch, but for when you are lying down in bed.
I don’t have a Z800, so many of these features aren’t really tested.
Z800 has yaw, pitch, and roll, measured in degrees.
But it also has many other values from all the various sensors.
In addition to reading the inputs, you can also set the screen brightness, colours, etc.
I’ll document this more later. In the meantime, you can look at VR920 examples and apply the same techniques.
I don’t have a Trimersion, and nobody with a Trimersion has tested my program, so this section is entirely theoretical.
The Trimersion head-tracker acts as an ordinary mouse. So you can read it like this:
Var.mickeys = 1200
Var.yaw = MapRange(mouse2.DirectInputX, -var.mickeys,var.mickeys, -90 degrees,90 degrees)
Var.pitch = MapRange(-mouse2.DirectInputY, -var.mickeys,var.mickeys, -90 degrees,90 degrees)
You may need to experiment with changing var.mickeys to other numbers. I don’t know how many mickeys the Trimersion emulates per 90 degrees of rotation.
That code allows you to read it independently of the real mouse, which will probably be mouse1. There is no guarantee that the Trimersion will be mouse2. If you are using a laptop, then the touchpad will probably be mouse1, the mouse will probably be mouse2, and the Trimersion will probably be mouse3.
Normally, all the mice, including the Trimersion, will control the system mouse, and thus control the view in an FPS.
But GlovePIE can swallow the mouse input so that it doesn’t control the system mouse anymore, if that is what you want…
mouse.Swallow = true
mouse.DirectInputX += Delta(mouse1.DirectInputX)
mouse.DirectInputY += Delta(mouse1.DirectInputY)
mouse.LeftButton = mouse1.LeftButton
mouse.RightButton = mouse1.RightButton
mouse.MiddleButton = mouse1.MiddleButton
mouse.XButton1 = mouse1.XButton1
mouse.XButton2 = mouse1.XButton2
debug = var.yaw+', '+var.pitch
Adding the above code will allow mouse1 to control the system mouse, but the Trimersion and other mice will not have any effect on the system mouse.
This is good for if you want to emulate TrackIR with the Trimersion, while controlling other things with the mouse, or for emulating the joystick with the Trimersion.
To emulate a TrackIR you would add:
FakeTrackIR.yaw = var.yaw
FakeTrackIR.pitch = var.pitch
If you have more than one real mouse, you might want to add “or mouse2.LeftButton1”, and “+ Delta(mouse2.DirectInputX)”, etc.
The Trimersion also acts as a keyboard. And GlovePIE can also read multiple keyboards individually. But unlike with the mouse, GlovePIE can’t swallow the keyboard while still reading keyboards individually. So you can tell whether input came from the real keyboard or the Trimersion, but you can’t suppress the input from one while still allowing the input from another. However you can tell the game to ignore the input from the regular keys on both, and have the regular keys on the Trimersion emulate a different set of keys, and have the regular keys on the keyboard emulate a separate different set of keys. In that way, the keyboard and the Trimersion can control different things.
Trimension acts like the following keys (you can use either keyboard2.whatever, or key2.whatever, it makes no difference):
Key2.R, Key2.Space, Key2.E, Key2.F, Key2.Esc, Key2.Enter, Key2.Up, Key2.Down, Key2.Left, Key2.Right, Key2.G, Key2.Tab, Key2.X, Key2.Q, Key2.Z, Key2.LCtrl, Key2.W, Key2.A, Key2.S, Key2.D
Mouse2.LeftButton, Mouse2.RightButton, Mouse2.DirectInputX, Mouse2.DirectInputY
GlovePIE allows you to read from a real TrackIR, or emulate a fake TrackIR. You can even do both at once. You can also read from other Optitrack devices.
Currently reading from TrackIR the same way a game does is not working, although it will read from the fake TrackIRs created by other TrackIR emulators. To read from a real TrackIR you need to use GlovePIE’s support for the OptiTrack SDK.
Warning! TrackIR emulation support adds settings to your registry. The first time you run a script that uses fake TrackIR emulation, GlovePIE will add a registry setting which tells programs where to look for the TrackIR driver. GlovePIE tells it that the TrackIR driver is in the GlovePIE directory. The TrackIR driver in the GlovePIE directory is a fake one. This fake driver will then be loaded by any TrackIR game, even if GlovePIE is not running. The fake driver just returns zeroes if GlovePIE isn’t running. GlovePIE will also add another registry entry which says where the real driver is, so that GlovePIE can still use the real driver, but other programs can’t.
To restore the registry to the correct setting, use the CP-Settings > Restore Real TrackIR menu. You only need to do that if you have a real TrackIR and you want to be able to use it instead of the fake one.
There are six values that TrackIR tells the games: Yaw, pitch, roll, x, y, and z.
The first three are angles in degrees. Yaw is turning to the right, pitch is looking up, and roll is tilting your head to the right. The last three are the position of your head: x is how far you moved your head to the right, y is how far you moved your head up, and z is how far you moved the head towards the screen. The position is measured in metres.
In your script you can set them like this:
TrackIR.yaw = VR920.yaw
But I recommend that you be more specific that it is the FakeTrackIR:
FakeTrackIR.yaw = vr920.yaw
They both work, but to prevent ambiguity for things like TrackIR.yaw = TrackIR.yaw + 1, you can specify RealTrackIR.yaw or FakeTrackIR.yaw. Normally reading TrackIR.yaw uses the real TrackIR, but writing it uses the fake one.
When you set the roll, you need to think about whether you want the game world to turn the same direction as your head to exaggerate the movement (like real TrackIR), or do you want the game world to turn in the opposite direction as your head turns, to counteract the movement (like in VR glasses)?
For Virtual Reality, you want roll to be the opposite, like this:
FakeTrackIR.roll = -vr920.roll
For a fixed screen not attached to your head, you want roll to be in the same direction, like this:
FakeTrackIR.roll = wiimote.roll
In both cases it will feel backwards if you do it wrong.
Unfortunately, how a game responds to TrackIR input is not entirely standardised. The value that turns you 180 degrees in one game, might only turn you 90 degrees in another game. I tried to guess which behaviour was most common and used that as the standard, but it still varies from game to game. So sometimes you will have to multiply it or divide it to make it right:
FakeTrackIR.yaw = vr920.yaw / 2
Some games need a totally different multiplier or divisor for the roll than for pitch and yaw. And some games may invert the roll.
Many TrackIR games limit the maximum amount you can turn in each direction. But some games allow you the complete 360 degrees.
To tell which TrackIR game is running, you can read FakeTrackIR.ProfileID. It will be a different number for different games.
You can read FakeTrackIR.Active to tell whether any TrackIR game is currently reading from the fake TrackIR. Games set their ProfileID when they start, but often only set Active while in the main game play part.
FakeTrackIR.RequestFormat tells you which data the TrackIR game wants. GlovePIE provides the same complete data regardless of what the game wants. I’m guessing the RequestFormat is a bitflag representing the x, y, z, yaw, pitch, and roll fields. This may allow you to work out which fields a game supports. But games sometimes ask for the wrong RequestFormat.
FakeTrackIR.WindowHandle tells you the handle of the window that the game registered with TrackIR.
FakeTrackIR.Cursor is true or false and can be turned on or off by games. I don’t know what it means. It may be an attempt to switch between TrackIR being used for mouse-look and being used for TrackIR.
FakeTrackIR.Version is the only value that you can set and the game reads. Games often ask for the version of TrackIR. You can tell it whatever you want. I don’t know the format of this value.
To get a list of at least 87 TrackIR games you can try, see: http://www.naturalpoint.com/trackir/03-enhanced-games/enhanced-games-all.html
The license agreement prohibits use with America’s Army.
TrackIR emulation won’t work with applications or games that use the Optitrack SDK, eg. ProgrammerArt’s "Le Cauchemar".
NOTE! This isn’t working yet, except for reading from fake TrackIRs from other emulators.
In theory, if you have a real TrackIR, then GlovePIE can read from it the same way a game would. GlovePIE pretends to be a game, and asks for the Yaw, Pitch, Roll, x, y, and z values. The TrackIR software must be running for this to work.
You can read it like this:
RealTrackIR.yaw, RealTrackIR.pitch, RealTrackIR.roll, RealTrackIR.x, RealTrackIR.y, and RealTrackIR.z
Note that these values are NOT the angles and position of the physical vector clip. They are the angles and position of the virtual in-game head which is normally a large exaggeration of the physical movements. You can change that behaviour if you want.
Also note that TrackIR may change the way the x, y ,z values move based on the yaw, pitch, and roll values. Which is good if you are only interested in the virtual head, but bad if you are interested in the real head. You can turn that feature off in TrackIR.
You can also read the TrackIR version:
RealTrackIR.version
Raw TrackIR camera input
You can also use the Optitrack SDK to read from a physical TrackIR camera. This existed in previous versions, but didn’t work, because I forgot to start the camera. In theory it should be fixed now. Although some of the names have been changed so as not to clash with the new TrackIR stuff.
The TrackIR software should NOT be running when using the TrackIR like this.
TrackIR.Illumination controls the IR LEDs on the TrackIR that shine on things so TrackIR can see them. It is true or false.
TrackIR.GreenLED controls the green status LED. TrackIR.RedLED controls the red LED. TrackIR.BlueLED controls the blue LED.
You may need to set the lights a couple of times if their actual state didn’t match the state GlovePIE thought they were in.
VectorClipX, VectorClipY, and VectorClipZ give the 3D position of the vector clip.
VectorClipYaw, VectorClipPitch, and VectorClipRoll give the orientation of the vector clip.
You can tell whether it can see all the dots that make up the clip by reading AllDotsVisible.
You can set Dist01, Dist02, Dist12 and DistoL if you have a custom vector clip that doesn’t match the standard one. They are the distances between dots and between certain points on the clip.
If you don’t have a vector clip, or if you just want to read dots individually, you can use:
Dot0x, Dot0y, Dot1x, Dot1y, Dot2x, Dot2y.
You may need to read the width and the height of the camera resolution, which are in Width, and Height. You can also read the framerate of the camera in FrameRate. Note that this might not match GlovePIE’s frame rate.
You can also read Serial, ModelNumber, ModelName, RevisionNumber, and RevisionName.
VRinsight’s HAT-Track is a wired head-tracking device similar to a TrackIR with the VectorClip Pro. It has a frame rate of 60Hz and a field of view of 30 degrees.
To attach the HAT-Track receiver to your screen, you will need to open the base of the receiver (it opens at the horizontal cut). The receiver can also swivel on it’s base so that it is facing the right direction. The receiver plugs into the transmitter. You should run the software on the CD that comes with it, and then plug in the USB port.
Without GlovePIE, the HAT-Track software can output in 5 different ways… For flight simulators it can use FSUIPC or SimConnect, for mouse-look games it can control the mouse, for joystick games it can emulate the joystick using PPJoy, and for applications designed specifically for the HAT-Track (like GlovePIE) it uses something called HAT3DIPC.
To use HAT-Track with GlovePIE you will need to run the HAT-Track software and click on the Settings menu. Then you need to choose “HAT3DIPC”.
When the HAT-Track software is running and working, it will show three small crosses connected by two lines, that move when you move your head. The receiver will also flash blue. The crosses and lines will move when you move your head. You need to press the space bar when your head is looking at the centre of the screen, so that the deadzone will be in the right place. If you can’t get the crossses and lines to show up, then make sure the transmitter is plugged into the receiver correctly and that your transmitter is roughly vertical and up the right way and pointing at the receiver. I had lots of trouble getting it to show up, but it worked eventually.
The sensitivity settings in the HAT-Track software use confusing names for the 6 axes… DX means Yaw, and DY means Pitch. But DZ does not mean roll, DZ means the Z position, also known as zoom. DT means roll (tilt). DH means X position (horizontal), and DV means Y position (vertical). These settings can be left as default, or set to 100, for use with GlovePIE.
Reading from HAT-Track in GlovePIE
HAT-Track is called either hattrack or hat_track in GlovePIE. It has 6 useful values that you can read, plus some raw values that you won’t need.
Hattrack.Yaw is the yaw angle in degrees, it will be between -180 and 180. Positive is right, negative is left. It is the angle to use in-game, not the real-world angle.
Hattrack.Pitch is the pitch angle in degrees, it will be between -180 and 180. Positive is up and negative is down. It is the angle to use in-game, not the real-world angle.
Hattrack.Roll is the roll angle in degrees, between -180 and 180. Positive is right, negative is left. It is the angle to use in-game, not the real-world angle.
Hattrack.X is the horizontal position in metres. Positive is right, negative is left. It is the distance to use in-game, not the real-world distance.
Hattrack.Y is the vertical position in metres. Positive is up, negative is down. It is the distance to use in-game, not the real-world distance.
Hattrack.Z is the forwards/backwards position in metres. Positive is forwards, negative is back. It is the distance to use in-game, not the real-world distance.
You can also read the XYZ position as a vector with Hattrack.pos, it is in metres in-game, and not the real world position.
You can also read the raw values, but they don’t provide any additional information, so their usage is not recommended. RawX, RawY, RawZ, RawPitch, RawYaw, and RawRoll are integer values with 16383 representing the centre. Raw values starting with “g” are decimal values scaled so that 0 is the centre and it is between roughly -1 and 1. gDx is Yaw, gDy is Pitch, gDz is Z, gDt is Roll, gPx is X, and gPy is Y. They do not provide any additional information, and the axes are not always measured in the same direction as the processed values above.
You can emulate TrackIR easily like this:
FakeTrackIR.yaw = HatTrack.yaw
FakeTrackIR.pitch = HatTrack.pitch
FakeTrackIR.roll = HatTrack.roll
FakeTrackIR.x = HatTrack.x
FakeTrackIR.y = HatTrack.y
FakeTrackIR.z = HatTrack.z
You might want to scale the values though, by dividing or multiplying by something. For example…
FakeTrackIR.roll = HatTrack.roll / 3
You might also want to leave out the roll altogether if you find it annoying.
You can (theoretically) emulate a HAT-Track, for any game or application that uses the HAT3DIPC interface. This allows you to play a HAT-Track game with a TrackIR, VR920, Z800, or whatever. Just set the HatTrack.yaw, HatTrack.Pitch, etc. values instead of reading them. Eg.
HatTrack.yaw = VR920.yaw
HatTrack.pitch = VR920.pitch
HatTrack.roll = VR920.roll