Tracking in Blender is quite straight forward and with a little bit of python code we can export the resulting tracks to use this wonderful piece of 3D software as a semi-automatic data annotation machine. The current version of the required script has been tested builds of blender up to 2.93.1 If you encounter any issues in using Blender or this script, please let me know so we can fix that as quickly as possible!
Get started by downloading the corresponding python script and downloading the latest version of blender straight from the source!
Simply select the correct version for your operating system and install the downloaded files.
Blender already has a wonderful build-in motion tracker, so all we need to do is show it where to find the motion export script on our machine to export the resulting coordinates as *.csv files. When opening blender click on the + symbol on the top bar to add a new Workspace.
From the drop-down menu select -> VFX -> Motion Tracking
Now your workspace should look like this:
Before we can started tracking our video, we first need to load the motion export script into our Workspace. To run the script within blender, we can replace one of the windows with a Text Editor. In this case we replace the top right window. Click the symbol in the upper left corner of the window to bring up the Editor Type menu and from Scripting select Text Editor.
Next, click Open and select the bash motionExportAddon.py file from your hard drive and click Open Text.
Once the script is loaded, you can click Run Script in the upper right corner of the Text Editor panel.
After running the script a new option should appear on the left side of the Movie Clip Editor panel.
Click on Tracking and Export to bring up the new options panel.
Before we continue, you should save your project to ensure the export location is correctly defined on your system. Otherwise, the exported files may be saved in some cryptic location you may not easily find later on. Simply click on File in the upper left corner of your screen and chose where to save your blender project.
To annotate a new video simply click on the Open button in the Movie Clip Editor panel, which should be in the middle of your screen. Select the file you wish to annotate and hit Open Clip.
After loading your video click on Track on the right side of the Movie Clip Editor panel to bring up additional options. Remember, you can always resize individual Panels within Blender to customize your Workspace and adjust it for your screen and workflow.
Next, define for how many frames you want to track in total by setting Start Frame and End Frame in the Export options of the Tracking and Export tab. In this example we want to track a total of 1000 frames, starting at frame 1.
Click on Add in the Marker menu to begin tracking. Then click on an animal in your loaded video clip and a track marker should appear.
Now for the most important short cuts to make interacting with blender easier. They work in any blender panel, so they are useful to remember:
G: Move – move a selected object around. Click the left mouse button to confirm the new position.
S: Scale – scale a selected object up or down. Here, this will help you to adjust the size of the tracking marker.
R: Rotate – rotate a selected object. Adding the rotation of a marker can help retain its identity across frames.
You can also add constraints to the above commands to specify their effect. After pressing any of the keys above you can additionally use:
X / Y / Z: Restrict the axis in which you are performing the action.
CTRL: Jump between whole number steps
SHIFT: Perform the change in finer increments. Helps a lot when making small changes.
You always confirm your change by clicking the LEFT Mouse Button.
And, of course, CTRL + Z is your friend to revert your last steps. By default, you can go back 32 steps, but you can increase that number under Edit/Preferences…/System menu, if you want to. Use the shortcuts above to position (G), rotate (R), and scale (S) the marker on an animal so that the area covers the entire body excluding its legs (shown below). Make sure the centre of your Marker is also in the centre of animal’s thorax.
This convention is absolutely crucial, to annotate data repeatably in the same way across animals, videos, observers, and datasets.
(close up of the placed marker)
Next, we will need to set up the Tracking Settings, within the Track menu. These settings have to be applied every time you add a new Marker:
Motion Model: Location & Rotation
Match: Keyframe
Prepass: Check
Normalize: Check
Correlation: 0.500
Frames Limit: 100 (optional, will speed up automated tracking)
Once your Track Marker is set up as described above, you can click on the right arrow to start tracking. The right arrow is located to the left of the Movie Clip Editor (center) panel under the Tracking and Export tab. The animal will be tracked for a number of frames set by the Frames Limit.
Whenever the automated tracking stops, or an animal is no longer tracked correctly, use the arrow keys on your keyboard to go back to the last correctly tracked frame. Then, using the shortcuts described above, re-position the Marker and continue the automated tracking.
In simple scenes (plain white background, few individuals) this will work very well and with few corrections required. However, be sure to go back and forth through your material to make sure all tracks adhere to the convention!
Once you are done tracking one animal, repeat the process for every animal you can make out in the scene. Simply Add a new Marker, position (G), rotate (R), and scale (S) it, configure the Tracking Settings, and continue.
Take your time - You will get faster the more material you annotate.
Remember to save your Blender file occasionally so you can return at a later point in time and don’t lose any work in case of an error.
It may be that after reloading the file, you will have to run the bash motionExportAddon.py script again, for the tracking options to reappear!
When you have tracked all animals in your scene, it’s time to export your tracks! Simply select an Export Path to define where to save your results and under Export click All.
After a few (milli) seconds, you should find all exported tracks in your specified folder:
Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.
Please make sure to update tests as appropriate.
© Simon Reichel & Fabian Plum, 2021 MIT License