New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
get skeleton joints to color coordinates (x,y) #56
Comments
It's technically possible. However, the current version of dp.kinect2 doesn't expose that particular coordinate mapping for the skeleton. The current version does...
You have a lot of knowledge in this topic area. So please ignore the following if you already tried it. You could simulate/approx this by configuring your GL camera to match the physical/lens attributes of the Kinect. Then when you draw anything using skeleton coordinates in GL, it could align with a depth/colormap you also project in GL with something like (jit.gl.videoplane) or (jit.gl.cornerpin). I did something similar once when I aligned a GL camera to match a real-world HD projector that was projecting its image on a dance stage. If ideas like the above don't work for your needs...what functionality do you need?
Given time and understanding your specific needs, e.g. questions above, its technically possible for me to make an incremental update to dp.kinect2 to add feature(s) to map the skeleton coordinates to depth/color. I have some concern in testing the boundary cases and how the feature would interact with all the transform possibilities |
Hi, |
Continuing thinking, exploring ideas, so nothing is decided or bad... :-) What are your thoughts? Can you share more details on your specific use-cases/scenarios? New methods
|
Hi, On my side my feeling go to "Three separate messages for joints in skeleton, color, and depth-space" maybe easier to manage in max ... The "New methods skeltocolor and skeltodepth" seems good but i don't know the latency if we ask calculation with 4 characters&24 joints.... For the "impossible values" i completly agree with you to let the max patch manage the datas ;-) Thank you so much for the discussion! Best |
Hi. I think it is possible to include some of these enhancements to dp.kinect2. Would you be willing to try and test some early builds? And could you help exercise/test it so we can be sure it meets your needs and performs well? |
Hi,
So cool! For sure i can test it!
Here a nice work in progress i made with your external :-)
https://www.youtube.com/watch?v=ZQDWquKLCzc&t=16s
Le jeu. 23 avr. 2020 à 16:33, Dale Phurrough <notifications@github.com> a
écrit :
… Hi. I think it is possible to include some of these enhancements to
dp.kinect2. Would you be willing to try and test some early builds? And
could you help exercise/test it so we can be sure it meets your needs and
performs well?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#56 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ADR3XULICGKMSIUTUOFAKHLROBGVJANCNFSM4LPMNY7A>
.
|
I'm unsure of the guaranteed order of the three skeleton coordinate messages. These messages (0, 1, 2, or all 3) would be output before any of the matrices (pointcloud, player, ir, color, depth) since outlet order is right->left. Approach 1 would use Approach 2 would be somewhat the same as outlet output (right->left). Meaning My instinct tells me approach 1, but I don't have any facts to support it. Do you have an opinion on this topic? |
I don't see any problems with these 2 scenarios . I like too the concept of
skel as "anchor" easier to understand . But we can easily manage the datas
with max so the 2 scenarios are perfect for me!
Best,
Le ven. 24 avr. 2020 à 01:32, Dale Phurrough <notifications@github.com> a
écrit :
… I'm unsure of the guaranteed order of the three skeleton coordinate
messages. These messages (0, 1, 2, or all 3) would be output before any of
the matrices (pointcloud, player, ir, color, depth) since outlet order is
right->left.
Approach 1 would use skel as the änchor"and consistently final message.
The other two could be either order. Perhaps skelcolor, skeldepth, skel
Approach 2 would be somewhat the same as outlet output (right->left).
Meaning skel, skelcolor, skeldepth.
My instinct tells me approach 1, but I don't have any facts to support it.
Do you have an opinion on this topic?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#56 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ADR3XUKYYVMEZIIZ6BGWIVDRODFYXANCNFSM4LPMNY7A>
.
|
Hi. I have two questions.
Here is the scenario.
Does the message I can code either approach. But I don't yet see the benefit of the 2nd (transform+color). Because the joint's color coordinate that would be output would not align with the colormap matrix being output on outlet 2. So while I could do it...is that useful? I guess you could use I'm unsure. What is your thinking on this? |
Hi,
I would prefer the message * skelcolor 1 l_hand give color coordinates of
the un-transformed joint* .
I don't see any advantage to apply the joint transforms and output the
color coordinate for the moment...
Maybe i need some practices of these features in order to see
the possibilities...
mathieu-
Le mar. 5 mai 2020 à 19:56, Dale Phurrough <notifications@github.com> a
écrit :
… Hi. I have two questions.
1. I have a early build you can try. Would you contact me by email
@diablodale <https://github.com/diablodale> and ask for it? It has the
depthtoskel, depthtocolor, skeltocolor, skeltodepth methods that you
can give one or more coordinates and it will return answers for them like
pixeltoskel does today.
2. I am unsure how skelcolor and skeldepth messages on the 5th outlet
output relate to @distmeter, @position, @quat, @rotate, @rotatexyz,
and @scale. I do understand how to use @flipx and @align.
Here is the scenario.
1. A patch tells dp.kinect2 to transform joint data using one or more
of the attributes @distmeter, @position, @quat, @rotate, @rotatexyz,
and @scale.
2. A patch turns on output of the skelcolor messages for outlet 5.
Does the message skelcolor 1 l_hand give color coordinates of the *un*-transformed
joint?
Or does it apply the joint transform(s) and then calculate/output the
color coordinate?
I can code either approach. But I don't yet see the benefit of the 2nd
(transform+color). Because the joint's color coordinate that would be
output would not align with the colormap matrix being output on outlet 2.
So while I *could* do it...is that useful? I guess you could use @position
or @quat to make the skeleton joints "beside" you...like a twin. And then
use the output of skelcolor to draw the skeleton "twin" on the colormap
matrix.
I'm unsure. What is your thinking on this?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#56 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ADR3XUM5ZYX3LSVM5HKBC3TRQBHOFANCNFSM4LPMNY7A>
.
|
I've released the v1.2 feature update to include these ideas. Available to download at the usual locations like https://hidale.com/shop/dp-kinect2/ |
Hi,
I've some problems to properly project datas (Skeleton joints) on top of the color and depth streams
One solution i guess could be to convert the skeleton joints to color coordinates (x,y)
Is there any technic to do this?
i've seen Microsoft sdk provide CoordinateMapper for that : CoordinateMapper’s job is to identify whether a point from the 3D space corresponds to a point in the color or depth 2D space
Sadly i'm only a maxer and can't use this CoordinateMapper ...
Maybe you've a solution?
Thank you very much
Mathieu
The text was updated successfully, but these errors were encountered: