-
Notifications
You must be signed in to change notification settings - Fork 92
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
MediaFrame, Camera capture #88
Comments
You can use the OpenXR XR_MSFT_spatial_graph_bridge extension to locate the camera on HoloLens 2. We've not publish the sample code on this yet, but here are some code snippet for you to get started. We will clean up our internal sample and publish here to github soon. Here's how to find the dynamic node Id from the MediaFoundation winrt APIs MediaFrameReference. std::optional<std::pair<winrt::guid, XrPosef>> FindDynamicNode(const MediaFrameReference& frame) {
// MFStreamExtension_CameraExtrinsics
constexpr winrt::guid CameraExtrinsicsGuid(0x6b761658, 0xb7ec, 0x4c3b, {0x82, 0x25, 0x86, 0x23, 0xca, 0xbe, 0xc3, 0x1d});
if (auto inspectable = frame.Properties().TryLookup(CameraExtrinsicsGuid)) {
auto propertyValue = inspectable.try_as<winrt::Windows::Foundation::IPropertyValue>();
if (propertyValue && propertyValue.Type() == winrt::Windows::Foundation::PropertyType::UInt8Array) {
winrt::com_array<uint8_t> bytes;
propertyValue.GetUInt8Array(bytes);
if (bytes.size() >= sizeof(MFCameraExtrinsics)) {
const auto* const extrinsics = reinterpret_cast<const MFCameraExtrinsics*>(bytes.data());
if (extrinsics->TransformCount > 0) {
const MFCameraExtrinsic_CalibratedTransform& transform = extrinsics->CalibratedTransforms[0];
winrt::guid dynamicNodeId;
xr::CopyGuid(dynamicNodeId, transform.CalibrationId);
return std::make_pair(dynamicNodeId, xr::math::Pose::MakePose(transform.Orientation, transform.Position));
}
}
}
}
return std::nullopt;
} Here's how to create an OpenXR space handle from the dynamic node id. void CreateCameraXrSpace(const std::pair<winrt::guid, XrPosef>& dynamicNode) {
const auto& [nodeId, pose] = dynamicNode;
XrSpatialGraphNodeSpaceCreateInfoMSFT spaceCreateInfo{XR_TYPE_SPATIAL_GRAPH_NODE_SPACE_CREATE_INFO_MSFT};
spaceCreateInfo.nodeType = XR_SPATIAL_GRAPH_NODE_TYPE_DYNAMIC_MSFT;
xr::CopyGuid(spaceCreateInfo.nodeId, nodeId);
spaceCreateInfo.pose = pose;
CHECK_XRCMD(
m_context.Extensions.xrCreateSpatialGraphNodeSpaceMSFT(m_context.Session.Handle, &spaceCreateInfo, m_cameraSpace.Put()));
} |
Thanks a lot, I will look into that! |
I could not find any info on that, but is there any functionality with OpenXR to grab mediaframes together with their respective SpatialCoordinateSystem? This is necessary for use with the locatable camera. I especially need the transformations from camera to world space.
The text was updated successfully, but these errors were encountered: