diff --git a/docs/babbleofficaltracker/babbleofficaltracker.mdx b/docs/babbleofficaltracker/babbleofficaltracker.mdx index 634d801..8f798d8 100644 --- a/docs/babbleofficaltracker/babbleofficaltracker.mdx +++ b/docs/babbleofficaltracker/babbleofficaltracker.mdx @@ -1,56 +1,181 @@ --- sidebar_position: 2 -title: Official Babble Tracker +title: Babble Tracker Quickstart --- -# Official Babble Tracker +# Babble Tracker Quickstart -Hi there! If you're in this section, you've purchaced a Babble Mouth Tracker V1! +This guide applies to the Official Babble Tracker, and should cover DIY versions that use similar hardware/firmware. It will walk you through the complete setup process, from initial connection to calibration and troubleshooting. -## Here's how to use the tracker: +## Getting Started +### Initial Setup -### 1) Plug your tracker into your PC! - - This is pretty simple! There is a USB C port on your tracker! Plug it in and you should hear a windows connection sound when plugged in! - - IMPORTANT: Make sure the switch is in the "ON" position! (see image below) - - There should be two very bright green lights on the back of the trackers! +1. **Plug your tracker into your PC** +- Connect using the USB port on your tracker +- Make sure the switch is in the "ON" position, as seen below +- Two bright green lights should appear on the back of the tracker +- You should hear a connection sound when properly connected -![switch](/img/setup_switch.jpg) +![Switch Position](/img/setup_switch.jpg) -### 2) Look at device manager for the COM port of the tracker! - - Press the windows key, then type "Device Manager" and press enter. - - In the device manager, look for "Ports (COM & LPT)" and expand it. - - You should see a new COM port listed under "Ports (COM & LPT)" listed as "USB Serial Device". This is the COM port of your tracker. +2. **Finding your device's name** +- **Windows** + - Press the Windows key, type "Device Manager" and press enter + - Expand "Ports (COM & LPT)" + - Look for a "USB Serial Device" entry - this is your tracker's COM port -![device manager](/img/setup_comm.png) +![Device Manager](/img/setup_comm.png) -### 3) Download the lastest version of the Babble Tracker software on our github! - - You can find the latest version of the software here: https://drive.google.com/file/d/1KmMBYSmAGlqDeTEjgnOOCX8YpWeeBgPb/view?usp=drive_link - - Download the Project_Babble-Setup-vX.X.X.exe file and run it to install the software. +- **macOS/Linux** + - On **macOS/Linux**, the Babble Tracker with appear as a device with a `/dev/ttyACM` prefix . You can list what devices are present with `ls -ld /dev/ttyACM*`. -:::warning -Warning, the software is still in beta and may have bugs! +#### Download and install the software +- **Windows** + - Get the latest version from here [Babble Tracker Software](https://drive.google.com/file/d/1KmMBYSmAGlqDeTEjgnOOCX8YpWeeBgPb/view?usp=drive_link). Download the Project_Babble-Setup-vX.X.X.exe file and run the installer +- **macOS/Linux** + - Use the [installer scripts](https://github.com/project-Babble/projectBabble/?tab=readme-ov-file#macos-and-linux) on the Github + +#### Connect to your tracker +- Open the Project Babble software +- Click on the dropdown next to "Camera Address" +- Select the namely identified in step 2 +- Click "Save and Restart Tracking" to connect + +:::info +If your camera isn't present in the dropdown, just type it in. ::: +![Camera Address Selection](/img/setup_dropdown_comm_select.png) + +## Setting Up The Camera Feed + +Before use, you need to properly set up your camera feed: + +1. **Ensure proper positioning** +- Your face should be evenly and well-illuminated +- The tracker should be able to see your entire lower face and jaw +- Positioned at a slight angle that shows the tip of your nose/nostrils + +2. **Set the crop area** +- Click the "Cropping Mode" button + +![Cropping Mode Button](/img/babbleofficaltracker/1.png) + +- The feed may appear choppy in this mode - *this is normal* +- Click and drag across the image to set the crop area +- Make sure the cropped area includes: + - Tip of your nose/nostrils + - Bottom of your jaw when open + - Sides of your cheeks when puffed + +![Proper Crop Example](/img/babbleofficaltracker/2.png) + +- Don't worry if the image is flipped - mirroring options are available at the bottom of the cam tab + +3. **Switch to tracking mode** +- Once you've set your crop, click "Tracking Mode" + +![Tracking Mode Button](/img/babbleofficaltracker/3.png) + +- You should now see a smooth video feed from your camera + +![Working Camera Feed](/img/setup_working_V.png) + +## Calibration Process + +Proper calibration is essential for accurate facial tracking: + +1. **Navigate to the Calibration tab** + +![Calibration Tab](/img/babbleofficaltracker/4.png) + +2. **Perform Neutral Calibration** +- Set the calibration mode to "Neutral" + +![Neutral Mode](/img/babbleofficaltracker/5.png) + +- Return to the Cam tab +- Check "Enable Calibration" + +![Enable Calibration](/img/babbleofficaltracker/8.png) + +- Relax your face into its natural resting position +- Click "Start Calibration" + +![Start Calibration](/img/babbleofficaltracker/9.png) + +- You'll hear a sound indicating calibration has started +- After a few seconds, click "Stop Calibration" + +![Stop Calibration](/img/babbleofficaltracker/10.png) + +3. **Perform Full Calibration** +- Return to the Calibration tab +- Set the calibration mode to "Full" + +![Full Mode](/img/babbleofficaltracker/6.png) + +- Go back to the Cam tab +- Click "Start Calibration" +- After hearing the start sound, make as many facial expressions as possible: + - Smile, frown, open mouth wide + - Puff cheeks, stick out tongue + - Move jaw side to side +- **Tip:** Keep movements natural - don't exaggerate expressions +- Calibration ends when you hear a different sound + +## Fine-Tuning and Troubleshooting + +If some expressions work better on one side than the other, or you need to adjust tracking sensitivity: + +### Understanding the Calibration Tab + +![Calibration Panel](/img/babbleofficaltracker/11.png) + +The calibration panel allows you to manually adjust tracking values: + +- **Left/Right**: Indicates which side of the face the shape corresponds to + +![Left Right Options](/img/babbleofficaltracker/12.png) + +- **Shapes**: The specific facial movement (left and/or right) being tracked (Mouth Open, Cheek Puff, etc.) + +![Shape Names](/img/babbleofficaltracker/13.png) +![Minimum Value](/img/babbleofficaltracker/14.png) + +- **Min**: The threshold at which the model starts detecting a shape + +![Min Value](/img/babbleofficaltracker/15.png) + +- **Max**: The maximum value the model detects for the shape + +![Maximum Value](/img/babbleofficaltracker/16.png) + +### Adjusting Values + +- For proper tracking, minimum values should generally start at around 0.1 +- Example with CheekPuff: + - Setting Minimum to 0.4: You need significant cheek puffing before detection + - Setting Maximum to 0.1: Slight cheek movement registers as full puffing + +### Fixing Common Issues + +If your jaw appears to lean in your neutral position: +- Adjust the minimum value for JawLeft until it's no longer detected when your face is at rest -### Open the Project Babbble software and select the COM port of your tracker. - - In the Project Babble software, click on the dropdown next to the "Camera Address" text. - - Click the on the desired camera Address. +**Pro Tip:** For precise adjustments, open your VR application's debug menu for the mouth tracking asset. Watch how values change as you make expressions, then return to the calibration tab to fine-tune. -![camera address](/img/setup_dropdown_comm_select.png) +![Maximum Value](/img/babbleofficaltracker/17.png) -### 4) Getting a camera feed running! - - Now click the "Save and Restart Tracking" button to connect. - - IMPORTANT: You won't see a video until you set a crop area. So click on "cropping mode" and click and drag to set a crop area. - - For actual use make sure to make a box around your mouth! - - After this go back to "Tracking Mode" and you should see a video feed from your camera! - -![camera working](/img/setup_working_V.png) +## Next Steps -### 5) Finished! - - To click on one of the links down below to get started in game! - - For people who want to hop in game, click here for [VRChat](docs\software\integrations\vrc.mdx), [Resonite](docs\software\integrations\resonite.mdx), or here for [something else](docs\software\integrations\integrations.mdx). +Your Babble Tracker is now set up and ready to use! To get started with specific VR platforms: -## But I want WiFi! +- [VRChat Integration Guide](docs/software/integrations/vrc.mdx) +- [ChilloutVR](docs/software/integrations/chilloutVR.mdx) +- [Resonite Integration Guide](docs/software/integrations/resonite.mdx) +- [Other Platforms](docs/software/integrations/integrations.mdx) -If you want WiFi, please look at our [Firmware Guide](docs\hardware\Firmware\firmware.mdx). +## WiFi Capability +Want to use your tracker wirelessly? Check out our [Firmware Guide](docs/hardware/Firmware/firmware.mdx) for instructions on enabling and configuring WiFi functionality. \ No newline at end of file diff --git a/static/img/babbleofficaltracker/1.png b/static/img/babbleofficaltracker/1.png new file mode 100644 index 0000000..2dfad6a Binary files /dev/null and b/static/img/babbleofficaltracker/1.png differ diff --git a/static/img/babbleofficaltracker/10.png b/static/img/babbleofficaltracker/10.png new file mode 100644 index 0000000..ede7b46 Binary files /dev/null and b/static/img/babbleofficaltracker/10.png differ diff --git a/static/img/babbleofficaltracker/11.png b/static/img/babbleofficaltracker/11.png new file mode 100644 index 0000000..3934ea6 Binary files /dev/null and b/static/img/babbleofficaltracker/11.png differ diff --git a/static/img/babbleofficaltracker/12.png b/static/img/babbleofficaltracker/12.png new file mode 100644 index 0000000..b532352 Binary files /dev/null and b/static/img/babbleofficaltracker/12.png differ diff --git a/static/img/babbleofficaltracker/13.png b/static/img/babbleofficaltracker/13.png new file mode 100644 index 0000000..9aa2e5b Binary files /dev/null and b/static/img/babbleofficaltracker/13.png differ diff --git a/static/img/babbleofficaltracker/14.png b/static/img/babbleofficaltracker/14.png new file mode 100644 index 0000000..f48246e Binary files /dev/null and b/static/img/babbleofficaltracker/14.png differ diff --git a/static/img/babbleofficaltracker/15.png b/static/img/babbleofficaltracker/15.png new file mode 100644 index 0000000..4d5ed47 Binary files /dev/null and b/static/img/babbleofficaltracker/15.png differ diff --git a/static/img/babbleofficaltracker/16.png b/static/img/babbleofficaltracker/16.png new file mode 100644 index 0000000..de9c991 Binary files /dev/null and b/static/img/babbleofficaltracker/16.png differ diff --git a/static/img/babbleofficaltracker/17.png b/static/img/babbleofficaltracker/17.png new file mode 100644 index 0000000..48ae232 Binary files /dev/null and b/static/img/babbleofficaltracker/17.png differ diff --git a/static/img/babbleofficaltracker/2.png b/static/img/babbleofficaltracker/2.png new file mode 100644 index 0000000..501ca09 Binary files /dev/null and b/static/img/babbleofficaltracker/2.png differ diff --git a/static/img/babbleofficaltracker/3.png b/static/img/babbleofficaltracker/3.png new file mode 100644 index 0000000..bfaadd8 Binary files /dev/null and b/static/img/babbleofficaltracker/3.png differ diff --git a/static/img/babbleofficaltracker/4.png b/static/img/babbleofficaltracker/4.png new file mode 100644 index 0000000..309acae Binary files /dev/null and b/static/img/babbleofficaltracker/4.png differ diff --git a/static/img/babbleofficaltracker/5.png b/static/img/babbleofficaltracker/5.png new file mode 100644 index 0000000..a9a40b1 Binary files /dev/null and b/static/img/babbleofficaltracker/5.png differ diff --git a/static/img/babbleofficaltracker/6.png b/static/img/babbleofficaltracker/6.png new file mode 100644 index 0000000..3934ea6 Binary files /dev/null and b/static/img/babbleofficaltracker/6.png differ diff --git a/static/img/babbleofficaltracker/7.png b/static/img/babbleofficaltracker/7.png new file mode 100644 index 0000000..371d5dc Binary files /dev/null and b/static/img/babbleofficaltracker/7.png differ diff --git a/static/img/babbleofficaltracker/8.png b/static/img/babbleofficaltracker/8.png new file mode 100644 index 0000000..e620bf9 Binary files /dev/null and b/static/img/babbleofficaltracker/8.png differ diff --git a/static/img/babbleofficaltracker/9.png b/static/img/babbleofficaltracker/9.png new file mode 100644 index 0000000..41f7585 Binary files /dev/null and b/static/img/babbleofficaltracker/9.png differ