Skip to content
Original file line number Diff line number Diff line change
Expand Up @@ -79,11 +79,32 @@ The transform rotation.
The <b>Position</b> value should be expressed in real-world coordinates in units of meters. The coordinate system of both position and orientation should be right-handed Cartesian as shown in the following diagram.

<img alt="Right-handed Cartesian coordinate system" src="images/MFCameraExtrinsic_Diagram.png"/>
<div class="alert"><b>Important</b>  <p class="note">The position and orientation are expressed as transforms toward the reference frame or origin. For example, a <b>Position</b> value of {-5, 0, 0} means that the origin is 5 meters to the left of the sensor, and therefore the sensor is 5 meters to the right of the origin. A sensor that is positioned 2 meters above the origin should specify a <b>Position</b> of {0, -2, 0} because that is the translation from the sensor to the origin.
<div class="alert"><b>Important</b>
<p class="note">The <b>Position</b> and <b>Orientation</b> have different interpretations for stand-alone sensor groups, and for Windows Mixed Reality devices.
</div>

### Stand-alone sensor groups
<p class="note">For stand-alone sensor groups, the position and orientation express the location of the origin of the group in the sensor's coordinate system. For example, a <b>Position</b> value of {-5, 0, 0} means that the origin is 5 meters to the left of the sensor, and therefore the sensor is 5 meters to the right of the origin. A sensor that is positioned 2 meters above the origin should specify a <b>Position</b> of {0, -2, 0} because that is the position of the origin relative to the sensor.

<p class="note">If the sensor is aligned with the origin, the rotation is the identity quaternion and the forward vector is along the -Z axis {0, 0, -1}. If the sensor is rotated +30 degrees around the Y axis from the origin, then the <b>Orientation</b> value should be a rotation of -30 degrees around the Y axis, because it represents the rotation from the sensor to the origin.
<p class="note">If the sensor is aligned with the origin, the rotation is the identity quaternion and the forward vector is along the -Z axis {0, 0, -1}. If the sensor is rotated +30 degrees around the Y axis from the origin (i.e. to the left), then the <b>Orientation</b> value should be a rotation of -30 degrees around the Y axis, because it represents the orientation of the origin, relative to the sensor.

### Windows Mixed Reality devices
<p class="note">Sensors attached to a Windows Mixed Reality device (e.g. a HoloLens) express their calibration using the opposite convention, so the position and orientation indicate the location of the <em>sensor</em> in the <em>device's</em> coordinate system. For example, a sensor which is located 5 centimeters to the right of the origin of the device will have a <b>Position</b> of {0.05, 0, 0} because that is the position of the sensor, relative to the device. Likewise, if a sensor is rotated +10 degrees around the Y axis from the device (i.e. to the left), then the <b>Orientation</b> value will be a rotation of +10 degrees about the Y axis, because it represents the orientation of the sensor, relative to the device.

In addition, the <b>CalibrationId</b> can be passed to <a href="https://docs.microsoft.com/en-us/uwp/api/windows.perception.spatial.preview.spatialgraphinteroppreview.createlocatorfornode">SpatialGraphInteropPreview.CreateLocatorForNode</a> to create a <a href="https://docs.microsoft.com/en-us/uwp/api/windows.perception.spatial.spatiallocator">SpatialLocator</a> which represents the Windows Mixed Reality device. This can be used to compute the location of the sensor in space for a given frame, by querying the location of the <b>SpatialLocator</b> at the time the frame was captured, and combining this with the the <b>Position</b> and <b>Orientation</b> of the <b>MFCameraExtrinsic_CalibratedTransform</b>. To query the device's location for a frame, pass the timestamp of the frame given by <a href="https://docs.microsoft.com/en-us/windows/win32/api/mfobjects/nf-mfobjects-imfsample-getsampletime">IMFSample::GetSampleTime</a> to <a href="https://docs.microsoft.com/en-us/uwp/api/windows.perception.perceptiontimestamphelper.fromsystemrelativetargettime">PerceptionTimestampHelper.FromSystemRelativeTargetTime</a>, and pass the resulting <b>PerceptionTimestamp</b> to <a href="https://docs.microsoft.com/en-us/uwp/api/windows.perception.spatial.spatiallocator.trylocateattimestamp">SpatialLocator.TryLocateAtTimestamp</a>.

To determine whether an <b>IMFSample</b> came frame a sensor attached to a Windows Mixed Reality device, look for the following property attached to the frame, set to `MFCameraExtrinsicsCalibrationType_SpatialGraphDynamicNode`:

```cpp
typedef enum _MFCameraExtrinsicsCalibrationType
{
MFCameraExtrinsicsCalibrationType_Unknown = 0,
MFCameraExtrinsicsCalibrationType_SpatialGraphDynamicNode = 1
} MFCameraExtrinsicsCalibrationType;

DEFINE_GUID(MFSampleExtension_CameraExtrinsicsCalibrationType, 0x3524e3b0, 0xc355, 0x49f1, 0xb8, 0xb0, 0xd0, 0xdd, 0x62, 0xde, 0x37, 0xa7);
```

</div>
<div> </div>


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ The principal point of the camera.



For square pixels, the X and Y fields of the <b>FocalLength</b> should be the same.
For square pixels, the X and Y fields of the <b>FocalLength</b> should be the same. The units of the focal length fields are pixels per meter, so dividing the width and height of the image by the X and Y fields of the focal length, respectively, yields the physical size in meters of the image at a distance of one meter from the sensor.

The <b>PrincipalPoint</b> field is expressed in pixels, not in normalized coordinates. The origin [0,0] is the bottom, left corner of the image.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -59,12 +59,12 @@ Represents a pinhole camera intrinsic model for a specified resolution.

### -field Width

The width for the pinhole camera intrinsic model.
The width for the pinhole camera intrinsic model, in pixels.


### -field Height

The height for the pinhole camera intrinsic model.
The height for the pinhole camera intrinsic model, in pixels.


### -field CameraModel
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ ms.custom: 19H1
## -description


Contains zero or 1 pinhole camera intrinsic models that describe how to project a 3D point in physical world onto the 2D image frame of a camera.
Contains zero or 1 pinhole camera intrinsic models that describe how to project a 3D point in physical world onto the 2D image frame of a camera. The conventions assumed by this structure imply a left-handed 3D coordinate system, with +X pointing to the right of the sensor, +Y pointing upwards from the sensor, and +Z pointing forward out of the sensor through the center (principal point) of the image.


## -struct-fields
Expand Down