Home
Cutie-Roto can assist in rotoscoping characters and objects from video clips.
The general workflow is to load a short video clip, and then define a mask by clicking on the image, then propagate the mask across the other frames. Then you can go to a few frames where the mask did not propagate well, and make corrections to the mask.
You can also commit these frames to the applications memory, so it can use these as reference when you propagate again.
In an ideal case, annotating just a few frames can provide a usable mask across your entire video clip. Once you are satisfied with the result, you can export it as a video.
Download the latest version from Releases. You want to download the 'Cutie-Roto-x.x.x.zip' file listed under Assets. NOT the source code.
After downloading the zip file, you will need to extract it to a convenient location on your PC. Note that you will need at least 5-10 GB of available space.
After extracting the zip file, go into the Cutie-Roto folder and run 'install_dependencies.bat'. It will provide you a choice between installing the CPU version, and the CUDA version which is optimized for Nvidia GPUs. The CUDA version is significantly faster, so please select that option if you have an Nvidia GPU. You can always run this script again and select a different option later if you need to.
It may take some time while the dependencies are downloaded and installed.
The application is portable and everything is stored within the Cutie-Roto folder, so simply deleting the folder will "uninstall" it. Note that python may cache some of the downloaded dependencies on your primary drive.
Once the dependencies are installed, you can launch the application by running the file 'run_gui.bat'.
The first time you launch it, some additional files will be downloaded.
Updating to the latest version is simple. In most cases you can simply download the latest release, and extract the files over the existing ones. In some cases it may be necessary to delete the existing version and install again as described above, but I will try to mention this in the changelog when appropriate.
When you run the file 'run_gui.bat' to launch Cutie-Roto, the Launcher will appear. Here you can select the video file that you want to work on. Cutie-Roto does not have a built in trimmer, so you should trim your video down to a short clip (typically a single shot without cuts) prior to loading it. Some sample clips are included in the "examples" folder, so you can get started right away.
Cutie-Roto will automatically create a workspace for each video you load. Each workspace stores the individual video frames, masks, and other intermediate files that might be created.
Because the workspace is based on the video's filename, please avoid working on multiple files that have the same name. You can delete any workspace folders that you no longer need, or if you want to completely reset your work on a file.
Once you press the 'Start' button to load a file, it will begin extracting the video frames and then open the main application window.
To create a mask on a frame, simply left click to select areas that you want to be in the mask, and right click to select areas that should not be masked.
If you are not getting a good result after several clicks, it may be best to click "reset frame" and try again.
If you are masking anime images, change the "click segmentation model" to "Anime". This may or may not work better for certain clips.
It is sometimes not possible to make very fine changes to the mask. For best results, it is recommended to creating an accurate mask in an external application and then import it.
You can import masks created in external software by clicking the 'import masks' button. The mask should be a black and white image file, with white indicating the object to be masked. You can select either a single mask, or multiple masks.
The file names of each mask will be scanned for numbers to determine which frame number to load the mask into. For example, a file named "mask02.png" will be loaded at frame 2. Please avoid using any other numbers in the file names. Multiple numbers, such as "mask1_02.png" is ambiguous and may not get loaded to the frame that you intended.
If you load a single mask, and no number is detected, it will be loaded into the currently displayed frame.
Loading a single mask will automatically be committed to memory. Loading multiple masks will prompt you to ask if they should all be committed to memory.
After creating a mask on at least one frame, click "Full Propagate" to propagate the mask across the video.
The "processing quality" setting effects the accuracy of the propagation. Higher quality settings have a significant impact on the amount of VRAM used, and VRAM usage increases with the length of the clip. You may be able to get away with using higher settings for short clips.
Once you have propagated masks across all frames and updated the masks to your liking, you can export the result by clicking the "Export as video" button.
There are three export types available: Matte, Composite on Alpha, and Composite on Green Screen.
- Matte simply exports a black and white matte. This option provides the most flexibility when taking it back into another application for compositing.
- Composite on Alpha will export the video with the background removed and with an alpha channel. This lets you easily place it on top of a new background in an editing application.
- Composite on Green Screen will export the video against a solid green background. This option is not recommended, aside from being able to better visualize the mask or to quickly show it to another person.
The "refine edges" checkbox will soften the edges of the mask. It is generally recommended to enable this. However, this should only be used if you have run a full propagation immediately prior to exporting. This option utilizes soft masks that are only created or updated when each frame is propagated.
The refine edges option in the export video dialog should only be used after running a full propagation. Refine edges uses soft masks that are only generated/updated upon propagation. Also the soft mask for the very first frame will be low quality. A workaround for that is to then go to frame 1 and propagate backward.
Currently, propagating over a frame does not respect the manual changes that you made to the mask. For example, if you edit a frame's mask and commit it to permanent memory, and then run propagation over that frame, the propagation may make changes to that mask.
If you want to erase all masked frames and start over, just go to the first frame and press 'Reset frame', then press 'Reset all memory'. Finally, click 'full propagate' to remove the masks from all remaining frames.
For best results, create accurate masks in an external application and then import them. The interactive segmentation model used when you click on the frames is sometimes unable to handle small and detailed areas.
- Ctrl+Z: Undo the last click when masking a frame
- Left/Right: Previous/Next frame
- Space: Play video
- Backspace: Reset mask on current frame
- Plus/Minus: Zoom in/out on the minimap
- Enter: Commit the current frame to permanent memory
- F1: Open this help page
There is a configuration file located at cutie\config\gui_config.yaml
This file allows changing some of the advanced settings and defaults.