Skip to content

Commit

Permalink
VR support: Add OculusVRToolbox and associated basic enablement.
Browse files Browse the repository at this point in the history
Psychtoolbox/PsychHardware/OculusVRToolbox/ contains M-Files with
functions for use of Oculus VR hardware. Currently the basic
high-level support driver PsychOculusVR() which itself uses the
PsychOculusVRCore mex file.

A new pair of vertex- and fragment shaders OculusRiftCorrectionShader.frag.txt
and OculusRiftCorrectionShader.vert.txt provides plugins for Screens
imaging pipeline to do the necessary client distortion rendering to
correct for the Rifts lens distortion, chromatic aberration and vignetting.

PsychImaging supports a new task 'UseOculusVRHMD' to setup an onscreen
window for display on the Oculus Rift. It does necessary stereo setup,
panel fitter setup, imaging pipeline setup, and then calls into helper
functions in PsychOculusVR.m to do the work of setting up shaders,
display lists etc. for configuring the post-processing in PTB's imaging
pipeline.

OculusVRDemo.m is the start of a very hacky demo, used for testing
and debugging during development.

-> All of these will change substantially. This is just a backup commit
   to prevent loss of the current working state in case of mistakes or
   disasters.

-> Currently we can display a basic properly undistorted 2D stereo view
   on the Rift DK2 under Linux. However proper scaling is not yet worked
   out and highly hacky. This needs work. Actual 3D rendering is missing.
  • Loading branch information
kleinerm committed Sep 7, 2015
1 parent 39dbee3 commit 97c1afe
Show file tree
Hide file tree
Showing 5 changed files with 557 additions and 1 deletion.
53 changes: 53 additions & 0 deletions Psychtoolbox/PsychDemos/OculusVRDemo.m
@@ -0,0 +1,53 @@
function OculusVRDemo
% OculusVRDemo - A very basic demo for the Oculus VR Rift DK2

% History:
% 05-Sep-2015 mk Written.

% For debugging only:
%PsychDebugWindowConfiguration;

% Setup unified keymapping and unit color range:
PsychDefaultSetup(2);

% Select screen with highest id as Oculus output display:
screenid = max(Screen('Screens'));

% Check if at least one Oculus HMD is connected and available:
if PsychOculusVR('GetCount') > 0
% Yes. Open and initialize connection to first detected HMD:
fprintf('Opening the first connected Oculus VR headset.\n');
oculus = PsychOculusVR('Open', 0);
else
% No. Open an emulated/simulated HMD for basic testing and debugging:
fprintf('No HMD detected. Opening a simulated HMD.\n');
oculus = PsychOculusVR('Open', -1);
end

% Setup default rendering parameters:
PsychOculusVR('SetupRenderingParameters', oculus);

% Open our fullscreen onscreen window with black background clear color:
PsychImaging('PrepareConfiguration');
PsychImaging('AddTask', 'General', 'UseOculusVRHMD', oculus);
[win, rect] = PsychImaging('OpenWindow', screenid, 0, [1680, 0, 1680+1080, 1920])

for eye = 0:1
Screen('SelectStereoDrawBuffer', win, eye);
Screen('FillRect', win, [0 0 1]);
Screen('FrameRect', win, [1 1 0], [], 20);
Screen('FillOval', win, [1 1 1]);
Screen('TextSize', win, 200);
DrawFormattedText(win, sprintf('HELLO\nWORLD!\n%i', eye), 'center', 'center', [0 1 0]);
%Screen('FillOval', win, [0 1 0 1]);
end

Screen('Flip', win);

KbStrokeWait;
sca;

% Close HMD, shutdown OculusVR driver:
PsychOculusVR('Close');

end
105 changes: 104 additions & 1 deletion Psychtoolbox/PsychGLImageProcessing/PsychImaging.m
Expand Up @@ -1191,6 +1191,27 @@
% RemapMouse() to perform the neccessary coordinate transformation.
%
%
% * 'UseOculusVRHMD' Display this onscreen window on a Oculus VR Head mounted
% display (HMD), e.g., the Rift DK1 or Rift DK2. This enables display of
% stereoscopic visual stimuli on Virtual reality headsets from Oculus VR.
% You need to have the Oculus VR runtime installed on your machine for this
% to work.
%
% Usage:
%
% 1. Open a connection to a Oculus HMD and get a handle for the device:
% hmd = PsychOculusVR('Open' ...);
%
% 2. Perform basic configuration of the HMD via PsychOculusVR.
%
% 3. Add a PsychImaging task for the HMD and pass in its device handle 'hmd':
% PsychImaging('AddTask', 'General', 'UseOculusVRHMD', hmd);
%
% This sequence will perform the necessary setup of panel fitter, stereo display
% mode and image post-processing for geometry correction, color aberration
% correction and vignette correction for a fullscreen window on the HMD.
%
%
% * More actions will be supported in the future. If you can think of an
% action of common interest not yet supported by this framework, please
% file a feature request on our Wiki (Mainpage -> Feature Requests).
Expand Down Expand Up @@ -1326,6 +1347,8 @@
% 17.09.2014 Add 'Native16BitFramebuffer' support for Linux + FOSS + AMD. (MK)
% 03.11.2014 Make panelfitter compatible with Retina displays. (MK)
% 04.11.2014 Add new task 'UseRetinaResolution' for Retina displays. (MK)
% 06.09.2015 Add basic support for "Client distortion rendering" on the Oculus VR
% Rift DK1/DK2 virtual reality headsets. (MK)

persistent configphase_active;
persistent reqs;
Expand All @@ -1337,10 +1360,12 @@
% These flags are global - needed in subfunctions as well (ugly ugly coding):
global ptb_outputformatter_icmAware;
global isASideBySideConfig;
global maxreqarg;

if isempty(configphase_active)
configphase_active = 0;
ptb_outputformatter_icmAware = 0;
maxreqarg = 10;
end

if nargin < 1 || isempty(cmd)
Expand Down Expand Up @@ -1399,7 +1424,6 @@
% extend each requirement vector to some number of max elements, so all
% rows in the cell array have the same length:
x = varargin;
maxreqarg = 10;
if length(x) < maxreqarg
for i=length(x)+1:maxreqarg
x{i}='';
Expand Down Expand Up @@ -2275,6 +2299,7 @@
global ptb_outputformatter_icmAware;
global psych_gpgpuapi;
global isASideBySideConfig;
global maxreqarg;

% Reset flag to "no":
isASideBySideConfig = 0;
Expand Down Expand Up @@ -2618,6 +2643,56 @@
imagingMode = mor(imagingMode, kPsychNeedOtherStreamInput);
end

% Want to use a Oculus VR Head mounted display (HMD), e.g., Rift DK1/DK2?
floc = find(mystrcmp(reqs, 'UseOculusVRHMD'));
if ~isempty(floc)
% Yes: We need a peculiar configuration, which involves the panelfitter
% to allow for a custom resolution of the virtual framebuffers for left
% eye and right eye - much higher than output resolution, so we have enough
% excess information to deal with geometric undistortion warps, color aberration,
% and dynamic display warping for head motion correction. We also need a
% special stereo processing shader that does geometric distortion correction,
% color aberration correction, vignetting correction, and dynamic display warping
% in one go, as processing speed is crucial for VR experience.
[rows cols] = ind2sub(size(reqs), floc(1));
row = rows(1);

% Extract first parameter - This should be the handle of the Oculus VR device:
oculusHandle = reqs{row, 3};

% Verify it is already open:
if ~PsychOculusVR('IsOpen', oculusHandle)
error('UseOculusVRHMD: Invalid Oculus HMD handle specified. No such device opened.');
end

% We must use stereomode 6, so we get separate draw buffers for left and
% right eye, and the stereo compositor (merger) to fuse both eyes into a
% single output framebuffer, but with all internal buffers at at least
% full output framebuffer resolution. This will generate anaglyph shaders
% which we will need to replace with a very special shader for the Oculus HMD:
stereoMode = 6;

% We need fast backing store support for virtual framebuffers:
imagingMode = mor(imagingMode, kPsychNeedFastBackingStore);

% Append our generated 'UsePanelFitter' task to setup the panelfitter for
% our needs at 'OpenWindow' time:
clientRes = PsychOculusVR('GetClientRenderbufferSize', oculusHandle);
x{1} = 'General';
x{2} = 'UsePanelFitter';
x{3} = clientRes;
x{4} = 'Full';

% Pad to maxreqarg arguments:
if length(x) < maxreqarg
for i=length(x)+1:maxreqarg
x{i}='';
end
end

reqs = [reqs ; x];
end

% Display replication needed?
if ~isempty(find(mystrcmp(reqs, 'MirrorDisplayTo2ndOutputHead')))
% Yes: Must use dual window output mode. This implies
Expand Down Expand Up @@ -4107,6 +4182,34 @@
end
% --- End of setup for stereo crosstalk reduction ---

% --- Custom processing setup for the stereo compositor ---

% --- Oculus VR Headset support (e.g., Rift DK1/DK2 etc.)
floc = find(mystrcmp(reqs, 'UseOculusVRHMD'));
if ~isempty(floc)
[row col] = ind2sub(size(reqs), floc);

% Extract first parameter - This should be the handle of the Oculus VR device:
oculusHandle = reqs{row, 3};

% Verify it is already open:
if ~PsychOculusVR('IsOpen', oculusHandle)
sca;
error('In UseOculusVRHMD: Invalid Oculus HMD handle specified. No such device opened.');
end

% Ok, perform setup after onscreen window is open, e.g., setting up the special
% shaders for the stereo compositor:
if ~PsychOculusVR('PerformPostWindowOpenSetup', oculusHandle, win)
sca;
error('In UseOculusVRHMD: Failed to setup image post-processing for the Oculus VR HMD.');
end

% Ready to rock the HMD!
end
% --- End of Oculus VR Headset support code.


% --- FROM HERE ON ONLY OUTPUT FORMATTERS, NOTHING ELSE!!! --- %


Expand Down

0 comments on commit 97c1afe

Please sign in to comment.