Skip to content

Releases

v3.0: cartoon landmark model v3 (DINOv3 features)

Choose a tag to compare

@zenonLEE zenonLEE released this 06 Nov 06:47

Cartoon face landmark model v3 (68 points): the v2 network with frozen DINOv3 ViT-B/16 features added to the input of its hourglass stack through a small adapter (1.6 M parameters), trained with more cartoon data (synthetic anime faces, new pseudo-labels) and distilled from the same design with DINOv3 ViT-L/16. It is the default model of the mdfld pip package 3.0.0; --model v2 keeps the smaller, faster v2.

  • File: mdfld_cartoon_v3.torchscript.pt (TorchScript, fp32), 419,214,809 bytes
  • SHA-256: dc5de1817397fb0f8eed113b971ea367e21bf7fb09ab22dff11181bf7cd0a337
  • Input: (N, 3, 256, 256) BGR face crops, pixel / 255 × 2 − 1; output: (N, 68, 2) in [−1, 1] (same interface as the v2 TorchScript file).

With the package's three-pass crop:

Test set v2 v3
CariFace test, 1,560 caricatures: NME (%) 5.03 4.64
CariFace test: faces with NME above 10% 3.6% 2.0%
Real faces, 313 held-out video frames (automatic labels): NME (%) 1.74 1.69
Synthetic anime, 1,462 held-out generated faces (layout labels): NME (%) 6.30 5.09
Anime illustrations, 980 held-out images: disagreement above 15% with an anime face detector 10.5% 6.1%

v3 is about 2 × slower on a GPU (71 vs 38 ms per face on an RTX 4090 D) and 4.6 × on a CPU. On first download the package prints a notice about the DINOv3 License. Regions, protocol, training data and limitations: README, Cartoon model v3.

License. Research use only. The v3 file contains DINOv3 weights and is distributed under the DINOv3 License (attached as LICENSE_DINOv3.md), in addition to the MDFLD terms of use.

v2.1: eyelid contours with SAM 3.1

Choose a tag to compare

@zenonLEE zenonLEE released this 03 Nov 04:11

v2.1 adds eyelid/eyelid_sam31.py, which turns v2 68-point landmarks into lash-free, smooth upper and lower eyelid curves (60 points each) using SAM 3.1 text prompts. It needs no training.

  • Weights: unchanged. Use mdfld_cartoon_v2.pkl from the v2.0 release.
  • Requirements: SAM 3 (facebookresearch/sam3) and the sam3.1_multiplex.pt checkpoint from facebook/sam3.1 (gated, SAM License), in a separate environment with a CUDA GPU.
  • Results on 17 in-house stills (median, 28 cartoon eyes): the share of the lash band inside the contour drops from 0.09 for the interpolated v2 landmark curve to 0.02, and roughness from 0.17 px to 0.10 px.
  • Docs: setup, usage, the output format and limitations are in the README section Eyelid contours (v2.1).

Also new: CITATION.cff, so GitHub shows "Cite this repository".

v2.0: cartoon landmark model v2

Choose a tag to compare

@zenonLEE zenonLEE released this 02 Nov 07:02

Cartoon face landmark model v2 (68 points), fine-tuned from the v1 model on cartoon and caricature faces plus a small share of in-house real-face video frames (details in the README).

For inference it is a drop-in replacement for v1: the weights are in checkpoint["net"], so pass the file with --pretrained_weight.

  • File: mdfld_cartoon_v2.pkl, 69,027,997 bytes
  • SHA-256: f061f616133fe0d2497093037af2a8ff6968fb898016188282fd0f914ae9d65e

CariFace test set: 1,560 caricatures with CariFace's ground truth. The images were not used for training, but their labels were used to tune the pseudo-labelling settings. NME is in % of the outer eye-corner distance; lower is better.

Crop (v1 → v2) All 68 Eyes Mouth Eye failure rate
demo.py crop, 1 pass 7.81 → 6.43 4.57 → 3.80 6.44 → 4.88 6.8% → 5.1%
Tight crop, 2 passes 8.31 → 7.63 3.82 → 3.17 6.30 → 4.73 3.3% → 1.7%

With the tight crop the eyebrows get worse (12.33 → 13.63). All face regions, the crops, the training data and the known limitations are in the README under Cartoon model v2. Research use only.