[Bug]: (Linux) iPhone (VTube Studio) tracking seemingly connects, but fails to track. iFacialMocap advanced options doesn't recognize IP address change.

[Dormoxx]

Description

When connecting to the VTube Studio iPhone app, model shuts eyes and doesn't track. The app says it's connected and goes into streaming mode, but the model doesn't respond. (Before tracking, After tracking)

I've tried my custom model, one from a blendshape tutorial and this one (VRM, Flat) from booth.pm all with the same result.

I've tried both the flatpak version installed in the command line and the github release to the same result.

I've also tried iFacialMoCap with basically the same result but an actual error shows up in the terminal window. But VTube Studio makes it more obvious when it's connected. Changing the IP in vpuppr's iFacialMocap advanced options does nothing, as noted in the log.

I looked through past issues and didn't see any mentioning iPhone tracking problems, just webcam. Apologies if I missed it. If I left anything out or you need more info I'll do so gladly. Thank you for your time! 🙏

Steps to reproduce

1. Run app from terminal
2. Click 'run' button
3. Model -> load model -> load custom model
4. Load VTube Studio on iPhone and enable 3rd party PC clients
5. Tracking -> VTube Studio -> enter iPhone IP -> start -> close tracking window
6. VTube Studio goes into streaming mode, model shuts eyes and doesn't track. Waited up to a minute with no changes.
7. Disable VTube Studio tracking and Close VTube Studio app
8. Open iFacialMocap, setup destination IP to be PC's IP
9. Tracking -> iFacialMocap -> advanced options -> enter PC IP
10. Invalid IP Address in log ([INFO] iFacialMocap 2023-4-1_13:21:34 Listening for data at 192:49983) even though I input my PC's local IP (192.168.1.X).

App version

0.9.0

Operating system

Linux

Relevant log output

I looked in .local/share and .config for log files, but couldn't find anything related to vpuppr, so this is the log from the github release run in the terminal.


[dormoxx@dormoxx-boxpc vpuppr-0.9]$ ./vpuppr.x86_64 
Godot Engine v3.5.1.rc.custom_build.293c3844b - https://godotengine.org
OpenGL ES 3.0 Renderer: NVIDIA GeForce RTX 2070 SUPER/PCIe/SSE2
Async. shader compilation: OFF
 
Arg file flagd could not be opened, ignoring
[INFO] ConfigManager 2023-4-1_13:18:15 Loading metadata
[INFO] ConfigManager 2023-4-1_13:18:15 Finished loading metadata
[INFO] ExtensionManager 2023-4-1_13:18:15 No user extensions found, skipping
[INFO] ExtensionManager 2023-4-1_13:18:15 Parsing default extensions
[INFO] AppManager 2023-4-1_13:18:15 Started. おはよう。[ERROR] VRM Runner 2023-4-1_13:18:22 Tried to load a model at an empty path
[INFO] VRM Runner 2023-4-1_13:18:22 Starting load_model for res://entities/duck/duck.tscn
[INFO] VRM Runner 2023-4-1_13:18:22 Loading res://entities/duck/duck.tscn using load_tscn
[INFO] VRM Runner 2023-4-1_13:18:22 Using scn loader
[INFO] VRM Runner 2023-4-1_13:18:22 Loading config for model
[INFO] ConfigManager 2023-4-1_13:18:22 Loading model config: duck.vpuppr
[INFO] ConfigManager 2023-4-1_13:18:22 Finished loading model config
[INFO] ConfigManager 2023-4-1_13:18:22 Successfully set ModelConfig duck.vpuppr
[INFO] VRM Runner 2023-4-1_13:18:22 Finished loading config
[INFO] VRM Runner 2023-4-1_13:18:22 Finished load_model for res://entities/duck/duck.tscn
[INFO] ConfigManager 2023-4-1_13:18:39 Saving data for duck
[INFO] ConfigManager 2023-4-1_13:18:39 Finished saving data
[INFO] VRM Runner 2023-4-1_13:18:39 Generating preview
[INFO] VRM Runner 2023-4-1_13:18:39 Finished generating preview
[INFO] VRM Runner 2023-4-1_13:18:39 Starting load_model for /home/tai/Documents/VTubing/blend-shaped.vrm
[INFO] VRM Runner 2023-4-1_13:18:39 Loading /home/tai/Documents/VTubing/blend-shaped.vrm using load_vrm
[INFO] VRM Runner 2023-4-1_13:18:39 Using vrm loader
VRM: Using builtin gltf module
/home/tai/Documents/VTubing/blend-shaped.vrm
WARNING: Octahedral compression cannot be used to compress a zero-length vector, please use normalized normal values or disable octahedral compression
     at: norm_to_oct (servers/visual_server.cpp:346)
[INFO] VRM Runner 2023-4-1_13:18:51 Loading config for model
[INFO] ConfigManager 2023-4-1_13:18:51 Loading model config: blend-shaped.vpuppr
[INFO] ConfigManager 2023-4-1_13:18:51 Finished loading model config
[INFO] ConfigManager 2023-4-1_13:18:51 Successfully set ModelConfig blend-shaped.vpuppr
[INFO] VRM Runner 2023-4-1_13:18:51 Finished loading config
[INFO] VRM Runner 2023-4-1_13:18:51 Finished load_model for /home/tai/Documents/VTubing/blend-shaped.vrm
[INFO] AppManager 2023-4-1_13:18:51 Model has ultra nested meshes: .
[INFO] AppManager 2023-4-1_13:18:51 Model has ultra nested meshes: .
[INFO] ConfigManager 2023-4-1_13:19:25 Saving data for blend-shaped
[INFO] ConfigManager 2023-4-1_13:19:25 Finished saving data
[INFO] VTubeStudioTracker 2023-4-1_13:19:45 Starting receiver                                            <--- VTube Studio Tracker attempt
[INFO] ConfigManager 2023-4-1_13:20:40 Saving data for blend-shaped
[INFO] ConfigManager 2023-4-1_13:20:40 Finished saving data
[INFO] VTubeStudioTracker 2023-4-1_13:20:42 Stopping face tracker
[INFO] iFacialMocap 2023-4-1_13:21:34 Listening for data at 192:49983                                    <--- iFacialMocap attempt, I changed the advanced options-> IP to be my PC's IP but the app doesn't recognize the change
ERROR: getaddrinfo failed! Cannot resolve hostname.
   at: _resolve_hostname (drivers/unix/ip_unix.cpp:98)
[INFO] iFacialMocap 2023-4-1_13:21:34 
ERROR: Invalid IP address.
   at: IP_Address (core/io/ip_address.cpp:218)
ERROR: Condition "!p_bind_address.is_valid() && !p_bind_address.is_wildcard()" is true. Returned: ERR_INVALID_PARAMETER
   at: listen (core/io/udp_server.cpp:92)
[INFO] iFacialMocap 2023-4-1_13:22:0 Stopping face tracker
[INFO] ConfigManager 2023-4-1_13:22:1 Saving data for blend-shaped
[INFO] ConfigManager 2023-4-1_13:22:1 Finished saving data
[INFO] ConfigManager 2023-4-1_13:23:26 Saving data for blend-shaped
[INFO] ConfigManager 2023-4-1_13:23:26 Finished saving data
[INFO] ConfigManager 2023-4-1_13:24:31 Saving data for blend-shaped
[INFO] ConfigManager 2023-4-1_13:24:31 Finished saving data
[INFO] VRM Runner 2023-4-1_13:24:31 Generating preview
[INFO] VRM Runner 2023-4-1_13:24:31 Finished generating preview
[INFO] ConfigManager 2023-4-1_13:24:31 Saving data for blend-shaped
[INFO] ConfigManager 2023-4-1_13:24:31 Finished saving data
[INFO] AppManager 2023-4-1_13:24:31 Exiting. おやすみ。

Additional information

System specs in case they're needed:

• EndeavourOS, 6.2.8-zen1-1-zen
• KDE Plasma
• Ryzen 5 2600
• Geforce RTX 2070 Super
• iPhone XR, iOS 16.4

[orowith2os]

The Flatpak logs and such should be in ~/.var/app/pro.vpup.vpuppr. see if anything is there, it'll be easier to navigate compared to the normal directories.

[Dormoxx]

Oh yeah, I totally forgot about flatpaks using a separate directory than the ones I'm used to, thanks! Though, the logs look to be the exact same as the terminal output when i run it from there.

I did some more testing. My laptop had EndeavourOS installed on it, same as my desktop. i tried vpuppr + iFacialMocap and just as my desktop, it didn't work. Neiro was helping me on discord and I noticed in a screenshot they sent they are running PopOS. So out of desperation, I wiped my laptop and installed Pop. And with a little bit of fiddling, I got vpuppr + iFacialMocap to work!

But I much prefer pacman (+ the AUR) compared to apt so I'm currently making a Manjaro ISO to test if vpupper + iFacialMocap works on the laptop with Manjaro if it does, I may just wipe Endeavour and reinstall Manjaro. The last time I tried doing VTubing stuff was a year or so ago when I was running Manjaro and I never encountered this issue.

[orowith2os]

Don't use Manjaro, it'll only make things harder to debug. There's a reason it's not suggested. Distrobox would be better. Did you try and apply that bit of fiddling to EndeavourOS?

[Dormoxx]

The fiddling I mentioned is basically reloading the model with * in the address box until the app accepts * as the address as reported in the terminal. I did get it to finally accept * as an IP, but I get
[INFO] iFacialMocap 2023-4-2_3:50:13 Listening for data at *:49983 ERROR: getaddrinfo failed! Cannot resolve hostname. at: _resolve_hostname (drivers/unix/ip_unix.cpp:98) now.

The Manjaro install with vpupper + iFacialMocap on my laptop worked first try, no issues whatsoever xD.

[CyberGh0st-droid]

Don't use Manjaro, it'll only make things harder to debug. There's a reason it's not suggested. Distrobox would be better. Did you try and apply that bit of fiddling to EndeavourOS?

why would that be the case?

[orowith2os]

Manjaro claims to support many things from Arch, but in reality, causes more breakages rather than stability. Their developers also rush things out the door, as has been shown with their Asahi Linux patches. Their management is iffy, and they've worked together with companies like Pine64 to only support Manjaro on their devices and effectively obfuscate community editions.

[Dormoxx]

Regarding stability, I ran Manjaro from like early 2019 to late 2022 and never had any issues with it. I only reinstalled Endeavour because I felt like changing it up. Since I know Manjaro works I went with that. But if you have a similar arch based distro (preferably with KDE) that you recommend I'd be happy to test if it resolved this problem as easily as it was on Manjaro.

phone edit: come to think of it, this is the second memorable network issue i've has with endeavour in the time i've used it. the first was trying to get a beat saber mod manager to work, but it couldn't connect to the server to refresh the mod list. but it worked on manjaro back when i used it.

[orowith2os]

Everything else says Manjaro isn't stable, and just their practices will end up breaking things. That's a guarantee. But that's also not the point of this issue. We need to figure out why tracking woks on one disto but not another. Firewall, maybe?

[Dormoxx]

it's never broken in my experience but okay lol.

and nope, i haven't touched the firewall settings. the pc is on my home network so i didn't feel the need to mess with anything.

[CyberGh0st-droid]

I've used manjaro for the last two years and I've never had any of those problems the only times I've had things break is when I messed with stuff I didn't know about

[SS-KyleSam]

Exact same issue on NIXos

[vprime]

I was able to resolve the issue connecting VTubeStudioTracker by adding 21412 for TCP & UDP on my firewall, I'm on Manjaro and using Firewalld.