Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

C++ get the coordinates of the hand, how do tell if it's left or right? #5411

Open
kyn817046 opened this issue May 16, 2024 · 1 comment
Open
Assignees
Labels
legacy:hands Hand tracking/gestures/etc os:windows MediaPipe issues on Windows platform:c++ Issues specific to C++ framework in mediapipe

Comments

@kyn817046
Copy link

Have I written custom code (as opposed to using a stock example script provided in MediaPipe)

Yes

OS Platform and Distribution

windows

MediaPipe Tasks SDK version

0.10.14

Task name (e.g. Image classification, Gesture recognition etc.)

hand recongnistion

Programming Language and version (e.g. C++, Python, Java)

C++

Describe the actual behavior

get the coordinates of the hand

Describe the expected behaviour

how do tell if it's left or right?

Standalone code/steps you may have used to try to get what you need

// Wrap Mat into an ImageFrame.
	auto input_frame = absl::make_unique<mediapipe::ImageFrame>(
		mediapipe::ImageFormat::SRGB, camera_frame_RGB.cols, camera_frame_RGB.rows,
		mediapipe::ImageFrame::kDefaultAlignmentBoundary);

	camera_frame_RGB.copyTo(mediapipe::formats::MatView(input_frame.get()));

	//std::cout << "Wrap Mat into an ImageFrame." << std::endl;

	// Send image packet into the graph.
	size_t frame_timestamp_us =
		(double)cv::getTickCount() / (double)cv::getTickFrequency() * 1e6;

	MP_RETURN_IF_ERROR(m_Graph.AddPacketToInputStream(
		m_kInputStream, mediapipe::Adopt(input_frame.release())
		.At(mediapipe::Timestamp(frame_timestamp_us))));
	//std::cout << "Send image packet into the graph." << std::endl;


	// Get the graph result packet, or stop if that fails.
	mediapipe::Packet packet;
	mediapipe::Packet packet_landmarks;
	if (!m_pPoller->Next(&packet))
		return absl::OkStatus();

	if (m_pPoller_landmarks->QueueSize() > 0)
	{
		if (m_pPoller_landmarks->Next(&packet_landmarks))
		{

			std::vector<mediapipe::NormalizedLandmarkList> output_landmarks = packet_landmarks.Get<std::vector<mediapipe::NormalizedLandmarkList>>();
			//GestureRecognitionResult tempGestureResult;
			//auto handness= output_landmarks[0].classification().label();
			for (int m = 0; m < output_landmarks.size(); ++m)
			{
				mediapipe::NormalizedLandmarkList single_hand_NormalizedLandmarkList = output_landmarks[m];
				//single_hand_NormalizedLandmarkList.mutable_landmark()
				std::vector<PoseInfo> singleHandGestureInfo;
				singleHandGestureInfo.clear();
				for (int i = 0; i < single_hand_NormalizedLandmarkList.landmark_size(); ++i)
				{
					PoseInfo info;
					const mediapipe::NormalizedLandmark landmark = single_hand_NormalizedLandmarkList.landmark(i);
					info.x = landmark.x() * camera_frame.cols;
					info.y = landmark.y() * camera_frame.rows;
					singleHandGestureInfo.push_back(info);
				}
}

Other info / Complete Logs

No response

@kuaashish kuaashish self-assigned this May 16, 2024
@kuaashish kuaashish added os:windows MediaPipe issues on Windows platform:c++ Issues specific to C++ framework in mediapipe legacy:hands Hand tracking/gestures/etc labels May 16, 2024
@kyn817046
Copy link
Author

0.0 Please help me ....

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
legacy:hands Hand tracking/gestures/etc os:windows MediaPipe issues on Windows platform:c++ Issues specific to C++ framework in mediapipe
Projects
None yet
Development

No branches or pull requests

2 participants