Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Increase the expression parameter synced data limit beyond 128 bits #15

Closed
NGenesis opened this issue Feb 17, 2022 · 13 comments
Closed
Labels
enhancement New feature or request

Comments

@NGenesis
Copy link

NGenesis commented Feb 17, 2022

The current 128 bit limit on expression parameters is insufficient to allow for combined eye and face tracking with reasonable fidelity and needs to be signficantly increased to accommodate for this, especially if there is a desire to add more functionality to an avatar on top of face and eye tracking.

[Edited to remove content which violates VRChat's TOS]

@NGenesis NGenesis added the enhancement New feature or request label Feb 17, 2022
@Codel1417
Copy link

I think this would be more appropriate for a canny post. https://feedback.vrchat.com/

@regzo2
Copy link

regzo2 commented Feb 17, 2022

I just want to add some context to this, the mod allows you to set every facial tracking parameter to a 'Binary' parameter version, which essentially acts as a lower resolution float parameter. This does allow you to get nearly full facial tracking with some space to spare. So there is definitely some ways to compact parameters to be able to fit all face tracking parameters.

Would be nice to have some extra expression parameters, or even a dedicated parameter list for facial tracking!

@momo-the-monster momo-the-monster added this to the Avatar Face Control milestone Feb 17, 2022
@momo-the-monster
Copy link
Contributor

Hm - if the 'problem to solve' here is to enable high-fidelity eye and face tracking, then we aim to tackle that as part of the Avatar Face Control milestone, which I've attached to this issue.

@AeroScripts
Copy link

There are many use cases for more than 128 bits worth of parameters, face tracking needs it too for sure, but this is more of a fundamental enhancement that would open up a lot of doors for general OSC stuff too

@momo-the-monster
Copy link
Contributor

@AeroScripts I agree. There still needs to be some limit, but we'll consider raising it with the work we're doing on Avatars this year.

@alexderpyfox
Copy link

For multiple other reasons this would be nice to have especially when you got an avatar like the rexouim which got 38 face blendshapes and also a couple of eye ones it would really add more to realism

@alexderpyfox
Copy link

Raise it to 512 bits sounds reasonable

@Dervali-git
Copy link

256bit would be great, 512 amazing, but yes. the limit needs to be increased with OSC and dynamics around the corner

@Shadowriver
Copy link

With vive pro eye tracking via osc that i made i not even using all the input that hardware gives and i already running out of parameters, there no space to use facial tracker that i also have, so 1+ this.

Also im kind od advocate for native eye tracking support, since incoming new index is rumored have it SteamVR may get support for it which would probably be way better implementation... but even eithout eye tracking vive facial tracker have around 45 diffrent float inputs

@anonmon
Copy link

anonmon commented Feb 18, 2022

256 bits (Only 32 total floats/integers with room for nothing else) would be a nice bump, but still within "Could conceivably easily run out of parameters." depending on what we're trying to do.
512 bits (64 total floats/integers with no room for anything else) is the point where we genuinely have enough space to practically never run out of bits for anything. Unless we're straight up trying to replicate every body joint's rotation with floats or something equally insane.
Most people who don't need more than a few toggles for clothes and such are fine with the low current limit, but people who want to do actually cool things need a lot more than we have now.
Also more IK networked slots would be nice/crucial along with a bump in total number.

@A3yuu
Copy link

A3yuu commented Feb 20, 2022

I want an extra bit for fingers.
Unity has 20 parameters on the finger, so 40 Float (320 bits) is needed on the left and right, but now there are only 16.
Oculus is capable of finger Skeleton Tracking, and SteamVR has a Full Skeleton Input level.

@seanedwards
Copy link

While the VRChat team is considering this proposal, I've written up a doc describing a technique for effectively getting more parameters out of the same number of bits. OSC unlocks this technique in a way that wasn't viable before. Hopefully it's helpful to some folks who are anxious for this. https://github.com/seanedwards/vrc-worldobject/blob/main/docs/parameter-multiplexing.md (Pull requests welcome as always!)

@momo-the-monster
Copy link
Contributor

Now doubled to 256!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests