Skip to content

Awesome papers for markerless animal motion capture and 3D reconstruction.

Notifications You must be signed in to change notification settings

anl13/animal_papers

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Animal Paper Collection (Ongoing)

[2023.12.14] Here is a workshop involving animals and infants CV4Smalls Workshop! Feel free to check it out!

[2023.11.26] My own paper "MAMMAL" is published online in Nature Communications now!

[2023.06.10] The program of CV4Animals Workshop in CVPR 2023 is available!

[2022.08.08] Here is the Ph.D thesis of SEMIH GÜNEL (https://infoscience.epfl.ch/record/293821), who is the author of well-known DeepFly3D and LiftPose3D.

[2022.06.22] The 2nd CV4Animals Workshop in CVPR 2022 (https://www.cv4animals.com/) presents many excellent works on rodent 3D reconstruction!

[2022.01.02] To make it easy to track, I sort the papers using the timepoint I find them (not publication time), and add label badges to show paper features.

[2021.11.28] I recommend MMPose for 2D animal pose estimation. It has collected various kinds of datasets.

[2021.07.16] I recommend CV4Animal workshop in CVPR 2021 (https://www.cv4animals.com/) because it is a good collection of recent advances in animal pose estimation area!

Recently, markerless animal motion capture and 3D reconstruction attracts more and more attention in computer vision community. Inspired by remarkable techniques for markerless human motion capture, a few excellent literatures appear for animal modeling and reconstruction such as SMAL and DeepLabCut. However, there is still a long way before computer vision methods could capture natural motion of arbitary animals in industrial-grade.

Therefore, I contribute this repository to track every step towards the ultimate goal of high quality animal capture. If you want to add/remove an article, please send an email to Liang An(anliang at mail dot tsinghua dot edu dot cn). Thank all the authors for their contribution and support.


OpenAnimalTracks: A Dataset for Animal Track Recognition
Risa Shinoda, Kaede Shiohara
In ICIP 2024
EgoPet: Egomotion and Interaction Data from an Animal’s Perspective
Amir Bar, Arya Bakhtiar, Danny Tran, Antonio Loquercio, Jathushan Rajasegaran, Yann LeCun, Amir Globerson, Trevor Darrell
In ECCV 2024
PetFace: A Large-Scale Dataset and Benchmark for Animal Identification
Risa Shinoda, Kaede Shiohara
In ECCV 2024 (oral)
Ponymation: Learning Articulated 3D Animal Motions from Unlabeled Online Videos
Keqiang Sun, Dor Litvak, Yunzhi Zhang, Hongsheng Li, Jiajun Wu, Shangzhe Wu
In ECCV 2024
Characterizing the structure of mouse behavior using Motion Sequencing
Sherry Lin, Winthrop F. Gillis, Caleb Weinreb, Ayman Zeine, Samuel C. Jones, Emma M. Robinson, Jeffrey Markowitz, Sandeep Robert Datta
In Nature Protocols 2024
SUBTLE: An Unsupervised Platform with Temporal Link Embedding that Maps Animal Behavior
Jea Kwon, Sunpil Kim, Dong-Kyum Kim, Jinhyeong Joo, SoHyung Kim, Meeyoung Cha, C. Justin Lee
In IJCV 2024
AWOL: Analysis WithOut synthesis using Language
Silvia Zuffi, Michael J. Black
In ECCV 2024
Multi-view Tracking, Re-ID, and Social Network Analysis of a Flock of Visually Similar Birds in an Outdoor Aviary
Shiting Xiao, Yufu Wang, Ammon Perkes, Bernd Pfrommer, Marc Schmidt, Kostas Daniilidis, Marc Badger
In IJCV 2023
DeepShapeKit: accurate 4D shape reconstruction of swimming fish
Ruiheng Wu, Oliver Deussen, Liang Li
In IROS 2022
YOLO-Behaviour: A simple, flexible framework to automatically quantify animal behaviours from videos
Alex Hoi Hang Chan, Prasetia Putra, Harald Schupp, Johanna Köchling, Jana Straßheim, Britta Renner, Julia Schroeder, William D. Pearse, Shinichi Nakagawa, Terry Burke, Michael Griesser, Andrea Meltzer, Saverio Lubrano, Fumihiro Kano
In biorxiv 2024
High-throughput unsupervised quantification of patterns in the natural behavior of marmosets
William Menegas, Erin Corbett, Kimberly Beliard, Haoran Xu, Shivangi Parmar, Robert Desimone, Guoping Feng
In biorxiv 2024
Long-term tracking of social structure in groups of rats
M´at´e Nagy, Jacob D. Davidson, G´abor V´as´arhelyi, D´aniel Abel, Enik˝o Kubinyi, Ahmed El Hady, Tam´as Vicsek
In biorxiv 2024
Marigold: A machine learning-based web app for zebrafish pose tracking
Gregory Teicher, R. Madison Riffe, Wayne Barnaby, Gabrielle Martin, Benjamin E. Clayton, Josef G. Trapani, Gerald B. Downes
In biorxiv 2024
A Real-time Multi-Subject Three Dimensional Pose Tracking System for Analyzing Social Behaviors of Non-human Primates
Chaoqun Cheng, Zijian Huang, Ruiming Zhang, Guozheng Huang, Han Wang, Likai Tang, Xiaoqin Wang
In biorxiv 2024
PriMAT: A robust multi-animal tracking model for primates in the wild
Richard Vogg, Matthias Nuske, Marissa A. Weis, Timo L¨uddecke, Elif Karako¸c, Zurna Ahmed, Sofia M. Pereira, Suchinda Malaivijitnond, Suthirote Meesawat, Florentin W¨org¨otter, Peter M. Kappeler, Alexander Gail, Julia Ostner, Oliver Sch¨ulke, Claudia Fichtel, Alexander S. Ecker
In biorxiv 2024
Peering into the world of wild passerines with 3D-SOCS: synchronized video capture and posture estimation
Michael Chimento, Alex Hoi Hang Chan, Lucy M. Aplin, Fumihiro Kano
In biorxiv 2024
Quantifying social roles in multi-animal videos using subject-aware deep-learning
Kelly Goss, Lezio S. Bueno-Junior, Katherine Stangis, Théo Ardoin, Hanna Carmon, Jie Zhou, Rohan Satapathy, Isabelle Baker, Carolyn E. Jones-Tinsley, Miranda M. Lim, Brendon O. Watson, Cédric Sueur, Carrie R. Ferrario, Geoffrey G. Murphy, Bing Ye, Yujia Hu
In biorxiv 2024
Animal Avatars: Reconstructing Animatable 3D Animals from Casual Videos
Remy Sabathier, Niloy J. Mitra, David Novotny
In ECCV 2024
Three-dimensional body reconstruction enables quantification of liquid consumption in small invertebrates
Henrique Galante, Tomer J. Czaczkes, Massimo De Agrò
In biorxiv 2024
Lightning Pose: improved animal pose estimation via semi-supervised learning, Bayesian ensembling and cloud-native open-source tools
Dan Biderman, Matthew R. Whiteway, Cole Hurwitz, Nicholas Greenspan, Robert S. Lee, Ankit Vishnubhotla, Richard Warren, Federico Pedraja, Dillon Noone, Michael M. Schartner, Julia M. Huntenburg, Anup Khanal, Guido T. Meijer, Jean-Paul Noel, Alejandro Pan-Vazquez, Karolina Z. Socha, Anne E. Urai, The International Brain Laboratory, John P. Cunningham, Nathaniel B. Sawtell, Liam Paninski
In Nature Methods 2024
Keypoint-MoSeq: parsing behavior by linking point tracking to pose dynamics
Caleb Weinreb, Jonah E. Pearl, Sherry Lin, Mohammed Abdal Monium Osman, Libby Zhang, Sidharth Annapragada, Eli Conlin, Red Hoffmann, Sofia Makowska, Winthrop F. Gillis, Maya Jay, Ye Shaokai, Alexander Mathis, Mackenzie Weygandt Mathis, Talmo Pereira, Scott W. Linderman, Sandeep Robert Datta
In Nature Methods 2024
The Poses for Equine Research Dataset (PFERD)
Ci Li, Ylva Mellbin, Johanna Krogager, Senya Polikovsky, Martin Holmberg, Nima Ghorbani, Michael J. Black, Hedvig Kjellström, Silvia Zuffi, Elin Hernlund
In Scientific Data 2024
VAREN: Very Accurate and Realistic Equine Network
Silvia Zuffi, Ylva Mellbin, Ci Li, Markus Hoeschle, Hedvig Kjellström, Senya Polikovsky, Elin Hernlund, Michael J. Black
In CVPR 2024
TrAQ: a novel, versatile, semi-automated two-dimensional motor behavioural tracking software
Davide Di Censo, Ilaria Rosa, Brigida Ranieri, Tiziana DiLorenzo, Marcello Alecci, Tiziana Marilena Florio, Angelo Galante
In biorxiv 2024
A 3Dwhole-face movement analysis system to uncover underlying physiology in mice
Kyle Daruwalla, Irene Nozal Martin, Andrew Frankel, Diana Naglič, Zainab Ahmad, Xun Helen Hou
In biorxiv 2024
Deep Imputation for Skeleton Data (DISK) for Behavioral Science
France Rose, Monika Michaluk, Timon Blindauer, Bogna M. Ignatowska-Jankowska, Liam O’Shaughnessy, Greg J. Stephens, Talmo D. Pereira, Marylka Y. Uusisaari, Katarzyna Bozek
In biorxiv 2024
Visuo-frontal interactions during social learning in freely moving macaques
M. Franch, S. Yellapantula, A. Parajuli, N. Kharas, A. Wright, B. Aazhang, V. Dragoi
In Nature 2024
Neural signatures of natural behaviour in socializing macaques
Camille Testard, Sébastien Tremblay, Felipe Parodi, Ron W. DiTullio, Arianna Acevedo-Ithier, Kristin L. Gardiner, Konrad Kording, Michael L. Platt
In Nature 2024
replicAnt: a pipeline for generating annotated images of animals in complex environments using Unreal Engine
Fabian Plum, René Bulla, Hendrik K. Beck, Natalie Imirzian, David Labonte
In Nature Communications 2023
Investigating the use of odour and colour foraging cues by rosy-faced lovebirds (Agapornis roseicollis) using deep-learning based behavioural analysis
Winson King Wai Tsang, Emily Shui Kei Poon, Chris Newman, Christina D. Buesching, Simon Yung Wa Sin
In biorxiv 2024
Automated 3D analysis of social head-gaze behaviors in freely moving marmosets
Feng Xing, Alec G. Sheffield, Monika P. Jadi, Steve W.C Chang, Anirvan S. Nandy
In biorxiv 2024
A-SOiD, an active learning platform for expert-guided, data efficient discovery of behavior
Jens F. Tillmann, Alexander I. Hsu, Martin K. Schwarz, Eric A. Yttri
In Nature Methods 2024
Anti-drift pose tracker (ADPT): A transformer-based network for robust animal pose estimation cross-species
Guoling Tang, Yaning Han, Quanying Liu, Pengfei Wei
In biorxiv 2024
vmTracking: Virtual Markers Overcome Occlusion and Crowding in Multi-Animal Pose Tracking
Hirotsugu Azechi, Susumu Takahashi
In biorxiv 2024
ARBUR, a machine learning-based analysis system for relating behaviors and ultrasonic vocalizations of rats
Zhe Chen, Guanglu Jia, Qijie Zhou, Yulai Zhang, Zhenzhen Quan, Xuechao Chen, Toshio Fukuda, Qiang Huang, Qing Shi
In biorxiv 2024
WILDPOSE: A LONG-RANGE 3D WILDLIFE MOTION CAPTURE SYSTEM
Naoya Muramatsu, Sangyun Shin, Qianyi Deng, Andrew Markham, Amir Patel
In biorxiv 2024
Combining Unity with machine vision to create low latency, flexible, and simple virtual realities
Yuri Ogawa, Raymond Aoukar, Richard Leibbrandt, Jake S Manger, Zahra M Bagheri, Luke Turnbull, Chris Johnston, Pavan K Kaushik, Jan M Hemmi, Karin Nordström
In biorxiv 2024
MacAction: Realistic 3D macaque body animation based on multi-camera markerless motion capture
Lucas M. Martini, Anna Bogn´ar, Rufin Vogels, Martin A. Giese
In biorxiv 2024
DAMM for the detection and tracking of multiple animals within complex social and environmental settings
Gaurav Kaul, Jonathan McDevitt, Justin Johnson, Ada Eban-Rothschild
In biorxiv 2024
Computer Vision for Primate Behavior Analysis in the Wild
Richard Vogg, Timo Luddecke, Jonathan Henrich, Sharmita Dey, Matthias Nuske, Valentin Hassler, Derek Murphy, Julia Fischer, Julia Ostner, Oliver Schulke, Peter M. Kappeler, Claudia Fichte, Alexander Gail, Stefan Treue, Hansjorg Scherberger, Florentin Worg otter, Alexander S. Ecker
In arxiv 2024
PanAf20K: A Large Video Dataset for Wild Ape Detection and Behaviour Recognition
Otto Brookes et.al.
In IJCV 2024
Multi-animal 3D social pose estimation, identification and behaviour embedding with a few-shot learning framework
Yaning Han, Ke Chen, Yunke Wang, Wenhao Liu, Zhouwei Wang, Xiaojing Wang, Chuanliang Han, Jiahui Liao, Kang Huang, Shengyuan Cai, Yiting Huang, Nan Wang, Jinxiu Li, Yangwangzi Song, Jing Li, Guodong Wang, Liping Wang, Yaping Zhang, Pengfei Wei
In Nature Machine Intelligence 2024
Facemap: a framework for modeling neural activity based on orofacial tracking
Atika Syeda, Lin Zhong, Renee Tung, Will Long, Marius Pachitariu, Carsen Stringer
In Nature Neuroscience 2023
Three-dimensional surface motion capture of multiple freely moving pigs using MAMMAL
Liang An, Jilong Ren, Tao Yu, Tang Hai, Yichang Jia, Yebin Liu
In Nature Communications 2023
DeepLabCut-based Behavioural and Posture Analysis in a Cricket
Shota Hayakawa, Kosuke Kataoka, Masanobu Yamamoto, Toru Asahi, Takeshi Suzuki
In biorxiv 2023
Distinct developmental trajectories of autonomous behaviors and agency in rodents
C Mitelut, M Diez Castro, RE Peterson, M Gonçalves, MM Gamer, SRO Nilsson, TD Pereira, DH Sanes
In biorxiv 2023
BARN: Behavior-Aware Relation Network for multi-label behavior detection in socially housed macaques
Sen Yang, Zhi-Yuan Chen, Ke-Wei Liang, Cai-Jie Qin, Yang Yang, Wen-Xuan Fan, Chen-Lu Jie, Xi-Bo Ma
In Zoological Research 2023
CATER: Combined Animal Tracking & Environment Reconstruction
Lars Haalck, Michael Mangan, Antoine Wystrach, Leo Clement, Barbara Webb, Benjamin Risse
In Science Advances 2023
SMART-BARN: Scalable multimodal arena for real-time tracking behavior of animals in large numbers
Máté Nagy, Hemal Naik, Fumihiro Kano, Nora V. Carlson, Jens C. Koblitz, Martin Wikelski, Iain D. Couzin
In Science Advances 2023
Hierarchical organization of rhesus macaque behavior
Benjamin Voloh, Benjamin R. Eisenreich, David J-N. Maisson, R. Becket Ebitz, Hyun Soo Park, Benjamin Y. Hayden, Jan Zimmermann
In Oxford Open Neuroscience 2023
Open-source tools for behavioral video analysis: Setup, methods, and best practices
Kevin Luxem, Jennifer J Sun, Sean P Bradley, Keerthi Krishnan, Eric Yttri, Jan Zimmermann, Talmo D Pereira, Mark Laubach
In Elife 2023
Automated measurement of livestock body based on pose normalisation using statistical shape model
Xinying Luo, Yihu Hu, Zicheng Gao, Hao Guo, Yang Su
In Biosystems Engineering 2023
Establishing an AI-based evaluation system that quantifies social/pathophysiological behaviors of common marmosets
Takaaki Kaneko, Jumpei Matsumoto, Wanyi Lu, Xincheng Zhao, Louie Richard UenoNigh, Takao Oishi, Kei Kimura, Yukiko Otsuka, Andi Zheng, Kensuke Ikenaka, Kousuke Baba, Hideki Mochizuki, Hisao Nishijo, Ken-ichi Inoue, Masahiko Takada
In biorxiv 2023
Automatically annotated motion tracking identifies a distinct social behavioral profile following chronic social defeat stress
Joeri Bordes, Lucas Miranda, Maya Reinhardt, Sowmya Narayan, Jakob Hartmann, Emily L. Newman, Lea Maria Brix, Lotte van Doeselaar, Clara Engelhardt, Larissa Dillmann, Shiladitya Mitra, Kerry J. Ressler, Benno Pütz, Felix Agakov, Bertram Müller-Myhsok, Mathias V. Schmidt
In Nature Communications 2023
LoTE-Animal: A Long Time-span Dataset for Endangered Animal Behavior Understanding
Dan Liu, Jin Hou, Shaoli Huang, Jing Liu, Yuxin He, Bochuan Zheng, Jifeng Ning, Jindong Zhang
In ICCV 2023
Stimulatory effect of monoacylglycerol lipase inhibitor MJN110 on locomotion and step kinematics demonstrated by high-precision 3D motion capture in mice
Bogna M. Ignatowska-Jankowska, Aysen Gurkan Ozer, Alexander Kuck, Micah J. Niphakis, Daisuke Ogasawara, Benjamin F. Cravatt, Marylka Y. Uusisaari
In biorxiv 2023
NeuroMechFly 2.0, a framework for simulating embodied sensorimotor control in adult Drosophila
Sibo Wang-Chen, Victor Alfred Stimpfling, Pembe Gizem Ozdil, Louise Genoud, Femke Hurtak, Pavan Ramdya
In biorxiv 2023
Simultaneous recording of ultrasonic vocalizations and sniffing from socially interacting individual rats using a miniature microphone
Shanah Rachel John, Rishika Tiwari, Yizhaq Goussha, Rotem Amar, Alex Bizer, Shai Netser, Shlomo Wagner
In biorxiv 2023
Automated Maternal Behavior during Early life in Rodents (AMBER) pipeline
Hannah E. Lapp, Melissa G. Salazar, Frances A. Champagne
In biorxiv 2023
Three-dimensional markerless motion capture of multiple freely behaving monkeys for automated characterization of social behavior
Jumpei Matsumoto, Takaaki Kaneko, Kei Kimura, Salvador Blanco Negrete, Jia Guo, Naoko Suda-Hashimoto, Akihisa Kaneko, Mayumi Morimoto, Hiroshi Nishimaru, Tsuyoshi Setogawa, Yasuhiro Go, Tomohiro Shibata, Hisao Nishijo, Masahiko Takada, Ken-ichi Inoue
In biorxiv 2023
3D mouse pose from single‑view video and a new dataset
Bo Hu, Bryan Seybold, Shan Yang, Avneesh Sud, Yi Liu, Karla Barron, Paulyn Cha, Marcelo Cosino, Ellie Karlsson, Janessa Kite, Ganesh Kolumam, Joseph Preciado, José Zavala‑Solorio, Chunlian Zhang, Xiaomeng Zhang, Martin Voorbach, Ann E.Tovcimak, J.Graham Ruby, David A. Ross
In Scientific Reports 2023
Animal3D: A Comprehensive Dataset of 3D Animal Pose and Shape
Jiacong Xu, Yi Zhang, Jiawei Peng, Wufei Ma, Artur Jesslen, Pengliang Ji, Qixin Hu, Jiehua Zhang, Qihao Liu, Jiahao Wang, Wei Ji, Chen Wang, Xiaoding Yuan, Prakhar Kaushik, Guofeng Zhang, Jie Liu, Yushan Xie, Yawen Cui, Alan Yuille, Adam Kortylewski
In ICCV 2023
PPR: Physically Plausible Reconstruction from Monocular Videos
Gengshan Yang, Shuo Yang, John Z. Zhang, Zachary Manchester, Deva Ramanan
In ICCV 2023
AmadeusGPT: a natural language interface for interactive animal behavioral analysis
Shaokai Ye, Jessy Lauer, Mu Zhou, Alexander Mathis, Mackenzie Weygandt Mathis
In NeurIPS 2023
ARTIC3D: Learning Robust Articulated 3D Shapes from Noisy Web Image Collections
Chun-Han Yao, Amit Raj, Wei-Chih Hung, Yuanzhen Li, Michael Rubinstein, Ming-Hsuan Yang, Varun Jampani
In arxiv 2023
Behavioral decomposition reveals rich encoding structure employed across neocortex in rats
Bartul Mimica, Tuçe Tombaz, Claudia Battistin, Jingyi Guo Fuglstad, Benjamin A. Dunn, Jonathan R. Whitlock
In Nature Communications 2023
MABe22: A Multi-Species Multi-Task Benchmark for Learned Representations of Behavior
Jennifer J. Sun, Markus Marks, Andrew W. Ulmer, Dipam Chakraborty, Brian Geuther, Edward Hayes, Heng Jia, Vivek Kumar, Sebastian Oleszko, Zachary Partridge, Milan Peelman, Alice Robie, Catherine E. Schretter, Keith Sheppard, Chao Sun, Param Uttarwar, Julian M. Wagner, Erik Werner, Joseph Parker, Pietro Perona, Yisong Yue, Kristin Branson, Ann Kennedy
In ICML 2023
3D-POP - An automated annotation approach to facilitate markerless 2D-3D tracking of freely moving birds with marker-based motion capture
Hemal Naik, Alex Hoi Hang Chan, Junran Yang, Mathilde Delacoux, Iain D. Couzin, Fumihiro Kano, Mate Nagy
In CVPR 2023
SuperAnimal models pretrained for plug-and-play analysis of animal behavior
Shaokai Ye, Anastasiia Filippova, Jessy Lauer, Maxime Vidal, Steffen Schneider, Tian Qiu, Alexander Mathis, Mackenzie Weygandt Mathis
In arxiv 2023
CatFLW: Cat Facial Landmarks in the Wild Dataset
George Martvel, Nareed Farhat, Ilan Shimshoni, Anna Zamansky
In CV4Animals Workshop in CVPR 2023
Decompose to Generalize: Species-Generalized Animal Pose Estimation
Guangrui Li, Yifan Sun, Zongxin Yang, Yi Yang
In ICLR 2023
MammalNet: A Large-scale Video Benchmark for Mammal Recognition and Behavior Understanding
Jun Chen, Ming Hu, Darren J. Coker, Michael L. Berumen, Blair Costelloe, Sara Beery, Anna Rohrbach, Mohamed Elhoseiny
In CVPR 2023
BITE: Beyond Priors for Improved Three-D Dog Pose Estimation
Nadine Rüegg, Shashank Tripathi, Konrad Schindler, Michael J. Black, Silvia Zuffi
In CVPR 2023
MagicPony: Learning Articulated 3D Animals in the Wild
Shangzhe Wu, Ruining Li, Tomas Jakab, Christian Rupprecht, Andrea Vedaldi
In CVPR 2023
CLAMP: Prompt-based Contrastive Learning for Connecting Language and Animal Pose
Xu Zhang, Wen Wang, Zhe Chen, Yufei Xu, Jing Zhang, Dacheng Tao
In CVPR 2023
3D shape reconstruction of semi-transparent worms
Thomas P. Ilett, Omer Yuval, Thomas Ranner, Netta Cohen, David C. Hogg
In CVPR 2023
A robust and flexible deep-learning workflow for animal tracking
Abhishek Dutta, Natalia Pérez-Campanero, Graham K. Taylor, Andrew Zisserman, Cait Newport
In biorxiv 2023
Analysis of antennal responses to motion stimuli in the honey bee by automated tracking using DeepLabCut
Hiroki Kohno, Shuichi Kamata, Takeo Kubo
In biorxiv 2023
Learning Articulated Shape with Keypoint Pseudo-labels from Web Images
Anastasis Stathopoulos, Georgios Pavlakos, Ligong Han, Dimitris Metaxas
In CVPR 2023
Farm3D: Learning Articulated 3D Animals by Distilling 2D Diffusion
Tomas Jakab, Ruining Li, Shangzhe Wu, Christian Rupprecht, Andrea Vedaldi
In arxiv 2023
Parallelized computational 3D video microscopy of freely moving organisms at multiple gigapixels per second
Kevin C. Zhou, Mark Harfouche, Colin L. Cooke, Jaehee Park, Pavan C. Konda, Lucas Kreiss, Kanghyun Kim, Joakim Jönsson, Thomas Doman, Paul Reamey, Veton Saliu, Clare B. Cook, Maxwell Zheng, John P. Bechtel, Aurélien Bègue, Matthew McCarroll, Jennifer Bagwell, Gregor Horstmeyer, Michel Bagnat, Roarke Horstmeyer
In Nature Photonics 2023
Scalable Apparatus to Measure Posture and Locomotion (SAMPL): a high-throughput solution to study unconstrained vertical behavior in small animals
Yunlu Zhu, Franziska Auer, Hannah Gelnaw, Samantha N. Davis, Kyla R. Hamling, Christina E. May, Hassan Ahamed, Niels Ringstad, Katherine I. Nagel, David Schoppik
In biorxiv 2023
ScarceNet: Animal Pose Estimation with Scarce Annotations
Chen Li, Gim Hee Lee
In CVPR 2023
Lights, Camera, Mirrors, Action! Toolbox for 3D Analysis of High-rate Maneuvers Using a Single Camera and Planar Mirrors
Wajahat Hussain, Mahum Naveed, Asad Khan, Taimoor Hasan Khan, Muhammad Latif Anjum, Shahzad Rasool, Adnan Maqsood
In biorxiv 2023
Unsupervised decomposition of natural monkey behavior into a sequence of motion motifs
Koki Mimura, Jumpei Matsumoto, Daichi Mochihashi, Tomoaki Nakamura, Toshiyuki Hirabayashi, Makoto Higuchi, Takafumi Minamimoto
In biorxiv 2023
ArMo: An Articulated Mesh Approach for Mouse 3D Reconstruction
James P. Bohnslav, Mohammed Abdal Monium Osman, Akshay Jaggi, Sofia Soares, Caleb Weinreb, Sandeep Robert Datta, Christopher D. Harvey
In biorxiv 2023
Predicting long-term collective animal behavior with deep learning
Vaios Papaspyros, Ram´on Escobedo, Alexandre Alahi, Guy Theraulaz, Cl´ement Sire, Francesco Mondada
In biorxiv 2023
Three‑dimensional unsupervised probabilistic pose reconstruction (3D‑UPPER) for freely moving animals
Aghileh S. Ebrahimi, Patrycja Orlowska‑Feuer, Qian Huang, AntonioG. Zippo, Franck P. Martial, Rasmus S. Petersen, Riccardo Storchi
In Scientific Reports 2023
A review of 28 free animal-tracking software applications: current features and limitations
Veronica Panadeiro, Alvaro Rodriguez, Jason Henry, Donald Wlodkowic, Magnus Andersson
In Lab Animal 2021
JAX Animal Behavior System (JABS): A video-based phenotyping platform for the laboratory mouse
Glen Beane, Brian Q. Geuther, Thomas J. Sproule, Anshul Choudhary, Jarek Trapszo, Leinani Hession, Vivek Kohar, Vivek Kumar
In biorxiv 2023
An 8-cage imaging system for automated analyses of mouse behavior
Thaís Del Rosario Hernández, Narendra R. Joshi, Sayali V. Gore, Jill A. Kreiling, Robbert Creton
In biorxiv 2023
Quantifying the movement, behavior, and environmental context of group-living animals using drones and computer vision
Benjamin Koger, Adwait Deshpande, Jeffrey T. Kerby, Jacob M. Graving, Blair R. Costelloe, Iain D. Couzin
In Journal of Animal Ecology 2023
Gait-level analysis of mouse open field behavior using deep learning-based pose estimation
Keith Sheppard, Justin Gardin, Gautam Sabnis, Asaf Peer, Megan Darrell, Sean Deats, Brian Geuther, Cathleen M. Lutz, Vivek Kumar
In Cell Reports 2022
Markerless tracking of an entire honey bee colony
Katarzyna Bozek, Laetitia Hebert, Yoann Portugal, Alexander S. Mikheyev, Greg J. Stephens
In Nature Communications 2021
Capturing the continuous complexity of behaviour in Caenorhabditis elegans
Tosif Ahamed, Antonio C. Costa, Greg J. Stephens
In Nature Physics 2020
ConstrastivePose: A contrastive learning approach for self-supervised feature engineering for pose estimation and behavorial classification of interacting animals
Tianxun Zhou, Calvin Chee Hoe Cheah, Eunice Wei Mun Chin, Jie Chen, Hui Jia Farm, Eyleen Lay Keow Goh, Keng Hwee Chiam
In biorxiv 2022
Common Pets in 3D: Dynamic New-View Synthesis of Real-Life Deformable Categories
Samarth Sinha, Roman Shapovalov, Jeremy Reizenstein, Ignacio Rocco, Natalia Neverova, Andrea Vedaldi, David Novotny
In arxiv 2022
B-SOiD, an open-source unsupervised algorithm for identification and fast prediction of behaviors
Alexander I. Hsu, Eric A. Yttri
In Nature Communications 2022
Estimation of skeletal kinematics in freely moving rodents
Arne Monsees, Kay-Michael Voit, Damian J. Wallace, Juergen Sawinski, Edyta Charyasz, Klaus Scheffler, Jakob H. Macke, Jason N. D. Kerr
In Nature Methods 2022
OpenLabCluster: Active Learning Based Clustering and Classification of Animal Behaviors in Videos Based on Automatically Extracted Kinematic Body Keypoints
Jingyuan Li, Moishe Keselman, Eli Shlizerman
In biorxiv 2022
LASSIE: Learning Articulated Shapes from Sparse Image Ensemble via 3D Part Discovery
Chun-Han Yao, Wei-Chih Hung, Yuanzhen Li, Michael Rubinstein, Ming-Hsuan Yang, Varun Jampani
In NeurIPS 2022
Evaluation of mouse behavioral responses to nutritive versus nonnutritive sugar using a deep learning-based 3D real-time pose estimation system
Jineun Kim, Dae-gun Kim, Wongyo Jung, Greg S. B. Suh
In biorxiv 2022
Development of a new 3D tracking system for multiple marmosets under free-moving conditions
Terumi Yurimoto, Wakako Kumita, Kenya Sato, Rika Kikuchi, Yusuke Shibuki, Rino Hashimoto, Michiko Kamioka, Yumi Hayasegawa, Eiko Yamazaki, Yoko Kurotaki, Norio Goda, Junichi Kitakami, Tatsuya Fujita, Takashi Inoue, Erika Sasaki
In biorxiv 2022
LabGym: quantification of user-defined animal behaviors using learning-based holistic assessment
Yujia Hu, Carrie R. Ferrario, Alexander D. Maitland, Rita B. Ionides, Anjesh Ghimire, Brendon Watson, Kenichi Iwasaki, Hope White, Yitao Xi, Jie Zhou, Bing Ye
In Cell Reports Methods 2022
BehaviorDEPOT is a simple, flexible tool for automated behavioral detection based on markerless pose tracking
Christopher J Gabriel, Zachary Zeidler, Benita Jin, Changliang Guo, Caitlin M Goodpaster, Adrienne Q Kashay, Anna Wu, Molly Delaney, Jovian Cheung, Lauren E DiFazio, Melissa J Sharpe, Daniel Aharoni, Scott A Wilke, Laura A DeNardo
In eLife 2022
Cross-Skeleton Interaction Graph Aggregation Network for Representation Learning of Mouse Social Behaviour
Feixiang Zhou, Xinyu Yang, Fang Chen, Long Chen, Zheheng Jiang, Hui Zhu, Reiko Heckel, Haikuan Wang, Minrui Fei, Huiyu Zhou
In arxiv 2022
Animal pose estimation: A closer look at the state-of-the-art, existing gaps and opportunities
Le Jiang, Caleb Lee, Divyang Teotia, Sarah Ostadabbas
In Computer Vision and Image Understanding (CVIU) 2022
3D pose estimation enables virtual head fixation in freely moving rats
Artur Schneider, Christian Zimmermann, Mansour Alyahyay, Florian Steenbergen, Thomas Brox, Ilka Diester
In Neuron 2022
Natural behavior is the language of the brain
Cory T. Miller, David Gire, Kim Hoke, Alexander C. Huk, Darcy Kelley, David A. Leopold, Matthew C. Smear, Frederic Theunissen, Michael Yartsev, Cristopher M. Niell
In Current Biology 2022
Deep-learning based identification, tracking, pose estimation, and behavior classification of interacting primates and mice in complex environments
Markus Marks, Jin Qiuhan, Oliver Sturman, Lukas von Ziegler, Sepp Kollmorgen, Wolfger von der Behrens, Valerio Mante, Johannes Bohacek, Mehmet Fatih Yanik
In Nature Machine Intelligence 2022
EXPLORE: A novel deep learning-based analysis method for exploration behaviour in object recognition tests
Victor Iba˜nez, Laurens Bohlen, Francesca Manuell, Isabelle Mansuy, Fritjof Helmchen, Anna-Sophia Wahl
In biorxiv 2022
TRex, a fast multi-animal tracking system with markerless identification, 2D body posture estimation and visual field reconstruction
Tristan Walter, Iain D Couzin
In eLife 2021
Unsupervised behaviour analysis and magnification (uBAM) using deep learning
Biagio Brattoli, Uta Büchler, Michael Dorkenwald, Philipp Reiser, Linard Filli, Fritjof Helmchen, Anna-Sophia Wahl, Björn Ommer
In Nature Machine Intelligence 2021
Leaving Flatland: Advances in 3D behavioral measurement
Jesse D. Marshall, Tianqing Li, Joshua H. Wu, Timothy W. Dunn
In Current Opinion in Neurobiology 2022
Improved Markerless 3D Animal Pose Estimation Using Temporal Semi-Supervision
Tianqing Li, Kyle S. Severson, Fan Wang, Timothy W. Dunn
In CV4Animal workshop at CVPR 2022
A whole-body musculoskeletal model of the mouse
Shravan Tata Ramalingasetty, Simon M. Danner, Jonathan Arreguit, Sergey N. Markin, Dimitri Rodarie, Claudia Kathe, Grégoire Courtine, Ilya A. Rybak, Auke Ijspeert
In IEEE Access 2021
AVATAR: AI Vision Analysis for Three-dimensional Action in Real-time
Dae-Gun Kim, Anna Shin, Yong-Cheol Jeong, Seahyung Park, Daesoo Kim
In biorxiv 2022
TAVA: Template-free Animatable Volumetric Actors
Ruilong Li, Julian Tanke, Minh Vo, Michael Zollhoefer, Jurgen Gall, Angjoo Kanazawa, Christoph Lassner
In ECCV 2022
Selfee, Self-supervised Features Extraction of animal behaviors
Yinjun Jia, Shuaishuai Li, Xuan Guo, Bo Lei, Junqiang Hu, Xiao-Hong Xu, Wei Zhang
In Elife 2022
NeuroMechFly, a neuromechanical model of adult Drosophila melanogaster
Victor Lobato Rios, Pembe Gizem Ozdil, Shravan Tata Ramalingasetty, Jonathan Arreguit, Auke Jan Ijspeert, Pavan Ramdya
In Nature Methods 2022
SemiMultiPose: A Semi-supervised Multi-animal Pose Estimation Framework
Ari Blau, Christoph Gebhardt, Andres Bendesky, Liam Paninski, Anqi Wu
In arxiv 2022
Animal Kingdom: A Large and Diverse Dataset for Animal Behavior Understanding
Xun Long Ng, Kian Eng Ong, Qichen Zheng, Yun Ni, Si Yong Yeo, Jun Liu
In CVPR 2022 (oral)
Multi-animal pose estimation, identification and tracking with DeepLabCut
Jessy Lauer, Mu Zhou, Shaokai Ye, William Menegas, Steffen Schneider, Tanmay Nath, Mohammed Mostafizur Rahman, Valentina Di Santo, Daniel Soberanes, Guoping Feng, Venkatesh N. Murthy, George Lauder, Catherine Dulac, Mackenzie Weygandt Mathis, Alexander Mathis
In Nature Methods 2022
SLEAP: A deep learning system for multi-animal pose tracking
Talmo D. Pereira, Nathaniel Tabris, Arie Matsliah, David M. Turner, Junyu Li, Shruthi Ravindranath, Eleni S. Papadoyannis, Edna Normand, David S. Deutsch, Z. Yan Wang, Grace McKenzie-Smith, Catalin C. Mitelut, Marielisa Diez Castro, John D'Uva, Mikhail Kislin, Dan H. Sanes, Sarah D. Kocher, Samuel S.-H. Wang, Annegret L. Falkner, Joshua W. Shaevitz, Mala Murthy
In Nature Methods 2022
BARC: Learning to Regress 3D Dog Shape from Images by Exploiting Breed Information
Nadine Ruegg, Silvia Zuffi, Konrad Schindler, Michael J. Black
In CVPR 2022
Artemis: Articulated Neural Pets with Appearance and Motion Synthesis
Haimin Luo, Teng Xu, Yuheng Jiang, Chenglin Zhou, Qiwei Qiu, Yingliang Zhang, Wei Yang, Lan Xu, Jingyi Yu
In TOG (Siggraph) 2022
Automatic mapping of multiplexed social receptive fields by deep learning and GPU-accelerated 3D videography
Christian L. Ebbesen, Robert C. Froemke
In Nature Communications 2022
BANMo: Building Animatable 3D Neural Models from Many Casual Videos
Gengshan Yang, Minh Vo, Natalia Neverova, Deva Ramanan, Andrea Vedaldi, Hanbyul Joo
In CVPR 2022
Overcoming the Domain Gap in Neural Action Representations
Semih Gunel, Florian Aymanns, Sina Honari, Pavan Ramdya, Pascal Fua
In arxiv 2021
Incremental Learning for Animal Pose Estimation using RBF k-DPP
Gaurav Kumar Nayak, Het Shah, Anirban Chakraborty
In BMVC 2021
Muti-view Mouse Social Behaviour Recognition with Deep Graphic Model
Zheheng Jiang, Feixiang Zhou, Aite Zhao, Xin Li, Ling Li, Dacheng Tao, Xuelong Li, Huiyu Zhou
In TIP 2021
MouseVenue3D: A Markerless Three-Dimension Behavioral Tracking System for Matching Two-Photon Brain Imaging in Free-Moving Mice
Yaning Han, Kang Huang, Ke Chen, Hongli Pan, Furong Ju, Yueyue Long, Gao Gao, Runlong Wu, Aimin Wang, Liping Wang, Pengfei Wei
In Neuroscience Bulletin 2021
Multi-camera real-time threedimensional tracking of multiple flying animals
Andrew D. Straw, Kristin Branson, Titus R. Neumann, Michael H. Dickinson
In J. R. Soc. Interface 2011
The Mouse Action Recognition System (MARS) software pipeline for automated analysis of social behaviors in mice
Cristina Segalin, Jalani Williams, Tomomi Karigo, May Hui, Moriel Zelikowsky, Jennifer J Sun, Pietro Perona, David J Anderson, Ann Kennedy
In eLife 2021
Self-Supervised Keypoint Discovery in Behavioral Videos
Jennifer J. Sun, Serim Ryou, Roni Goldshmid, Brandon Weissbourd, John Dabiri, David J. Anderson, Ann Kennedy, Yisong Yue, Pietro Perona
In arxiv 2021
The Multi-Agent Behavior Dataset: Mouse Dyadic Social Interactions
Jennifer J. Sun, Tomomi Karigo, Dipam Chakraborty, Sharada P. Mohanty, David J. Anderson, Pietro Perona, Yisong Yue, Ann Kennedy
In NeurIPS (Dataset & Benchmarks) 2021
Task Programming: Learning Data Efficient Behavior Representations
Jennifer J. Sun, Ann Kennedy, Eric Zhan, David J. Anderson, Yisong Yue, Pietro Perona
In CVPR 2021 (Best Student Paper Award)
Coarse-to-fine Animal Pose and Shape Estimation
Chen Li, Gim Hee Lee
In NeurIPS 2021
Unified 3D Mesh Recovery of Humans and Animals by Learning Animal Exercise
Kim Youwang, Kim Ji-Yeon, Kyungdon Joo, Tae-Hyun Oh
In BMVC 2021
Anipose: a toolkit for robust markerless 3D pose estimation
Pierre Karashchuk, Katie L. Rupp, Evyn S. Dickinson, SarahWalling-Bell, Elischa Sanders, Eiman Azim, Bingni W. Brunton, John C. Tuthill
In Cell Reports (Resource) 2021
Multimodal-based Scene-Aware Framework for Aquatic Animal Segmentation
Minh-Quan Le, Trung-Nghia Le, Tam V. Nguyen, Isao Echizen, Minh-Triet Tran
In arxiv 2021
Learning interaction rules from multi-animal trajectories via augmented behavioral models
Keisuke Fujii, Naoya Takeishi, Kazushi Tsutsui, Emyo Fujioka, Nozomi Nishiumi, Ryoya Tanaka, Mika Fukushiro, Kaoru Ide, Hiroyoshi Kohno, Ken Yoda, Susumu Takahashi, Shizuko Hiryu, Yoshinobu Kawahara
In NeurIPS 2021
The PAIR-R24M Dataset for Multi-animal 3D Pose Estimation
Jesse Marshall, Ugne Klibaite, Amanda Gellis, Diego Aldarondo, Bence O¨lveczky, Tim Dunn
In NeurIPS 2021
Across-animal odor decoding by probabilistic manifold alignment
Pedro Herrero-Vidal, Dmitry Rinberg, Cristina Savin
In NeurIPS 2021
4DComplete: Non-Rigid Motion Estimation Beyond the Observable Surface
Yang Li, Hikari Takehara, Takafumi Taketomi, Bo Zheng, Matthias Nießner
In ICCV 2021
DensePose 3D: Lifting Canonical Surface Maps of Articulated Objects to the Third Dimension
Roman Shapovalov, David Novotny, Benjamin Graham, Patrick Labatut, Andrea Vedaldi
In ICCV 2021
Body language signals for rodent social communication
Christian L. Ebbesen, Robert C. Froemke
In Current Opinion in Neurobiology 2021
coding of facial expressions of pain in the laboratory mouse
Dale J Langford, Andrea L Bailey, Mona Lisa Chanda, Sarah E Clarke, Tanya E Drummond, Stephanie Echols, Sarah Glick, Joelle Ingrao, Tammy Klassen-Ross, Michael L LaCroix-Fralish, Lynn Matsumiya, Robert E Sorge, Susana G Sotocinal, John M Tabaka, David Wong, Arn M J M van den Maagdenberg, Michel D Ferrari, Kenneth D Craig, Jeffrey S Mogil
In Nature Methods (brief communications) 2010
AP-10K: A Benchmark for Animal Pose Estimation in the Wild
Hang Yu, Yufei Xu, Jing Zhang, Wei Zhao, Ziyu Guan, Dacheng Tao
In NeurIPS 2021
DeepImageJ: A user-friendly environment to run deep learning models in ImageJ
Estibaliz Gómez-de-Mariscal, Carlos García-López-de-Haro, Wei Ouyang, Laurène Donati, Emma Lundberg, Michael Unser, Arrate Muñoz-Barrutia, Daniel Sage
In Nature Methods (brief communication) 2021
idtracker.ai: tracking all individuals in small or large collectives of unmarked animals
Francisco Romero-Ferrero, Mattia G. Bergomi, Robert C. Hinz, Francisco J. H. Heras, Gonzalo G. de Polavieja
In Nature Methods (Brief Communication) 2019
OpenMonkeyChallenge: Dataset and Benchmark Challenges for Pose Tracking of Non-human Primates
Yuan Yao, Abhiraj Mohan, Eliza Bliss-Moreau, Kristine Coleman, Sienna M. Freeman, Christopher J. Machado, Jessica Raper, Jan Zimmermann, Benjamin Y. Hayden, Hyun Soo Park
In IJCV 2022
A Novel Dataset for Keypoint Detection of Quadruped Animals from Images
Prianka Banik, Lin Li, Xishuang Dong
In arxiv 2021
Discovering Relationships between Object Categories via Universal Canonical Maps
Natalia Neverova, Artsiom Sanakoyeu, Patrick Labatut, David Novotny, Andrea Vedaldi
In CVPR 2021
LiftPose3D, a deep learning-based approach for transforming two-dimensional to three-dimensional poses in laboratory animals
Adam Gosztolai, Semih Gunel, Marco Pietro Abrate, Daniel Morales, Victor Lobato Rios, Helge Rhodin, Pascal Fua, Pavan Ramdya
In Nature Methods 2021
Geometric deep learning enables 3D kinematic profiling across species and environments
Timothy W. Dunn, Jesse D. Marshall, Kyle S. Severson, Diego E. Aldarondo, David G. C. Hildebrand, Selmaan N. Chettih, William L. Wang, Amanda J. Gellis, David E. Carlson, Dmitriy Aronov, Winrich A. Freiwald, Fan Wang, Bence P. Ölveczky
In Nature Methods 2021
Birds of a Feather: Capturing Avian Shape Models from Images
Yufu Wang, Nikos Kolotouros, Kostas Daniilidis, Marc Badger
In CVPR 2021
ThruTracker: Open-Source Software for 2-D and 3-D Animal Video Tracking
Aaron J. Corcoran, Michael R. Schirmacher, Eric Black, Tyson L. Hedrick
In biorxiv 2021
A Hierarchical 3D-motion Learning Framework for Animal Spontaneous Behavior Mapping
Kang Huang, Yaning Han, Ke Chen, Hongli Pan, Wenling Yi, Xiaoxi Li, Siyuan Liu, Liping Wang, Pengfei Wei
In Nature Communications 2021
Big behavior: challenges and opportunities in a new era of deep behavior profiling
Lukas von Ziegler, Oliver Sturman, Johannes Bohacek
In Neuropsychopharmacology 2021
A three-dimensional virtual mouse generates synthetic training data for behavioral analysis
Luis A. Bolaños, Dongsheng Xiao, Nancy L. Ford, Jeff M. LeDue, Pankaj K. Gupta, Carlos Doebeli, Hao Hu, Helge Rhodin, Timothy H. Murphy
In Nature Methods (Brief Communication) 2021 (cover)
DeepLabStream enables closed-loop behavioral experiments using deep learning-based markerless, real-time posture detection
Jens F. Schweihoff, Matvey Loshakov, Irina Pavlova, Laura Kück, Laura A. Ewell, Martin K. Schwarz
In Communications Biology 2021
From Synthetic to Real: Unsupervised Domain Adaptation for Animal Pose Estimation
Chen Li, Gim Hee Lee
In CVPR 2021 (oral)
LASR: Learning Articulated Shape Reconstruction from a Monocular Video
Gengshan Yang, Deqing Sun, Varun Jampani, Daniel Vlasic, Forrester Cole, Huiwen Chang, Deva Ramanan, William T. Freeman, Ce Liu
In CVPR 2021
Real-time, low-latency closed-loop feedback using markerless posture tracking
Gary Kane, Gonçalo Lopes, Jonny L. Saunders, Alexander Mathis, Mackenzie W. Mathis
In eLife 2020 (Featured by Nature Methods)
Dynamics of gaze control during prey capture in freely moving mice
Angie M Michaiel, Elliott TT Abe, Cristopher M Niell
In eLife 2020
Continuous Whole-Body 3D Kinematic Recordings across the Rodent Behavioral Repertoire
Jesse D. Marshall, Diego E. Aldarondo, Timothy W. Dunn, William L. Wang, Gordon J. Berman, Bence P. O¨lveczky
In Neuron 2020
Robust mouse tracking in complex environments using neural networks
Brian Q. Geuther, Sean P. Deats, Kai J. Fox, Steve A. Murray, Robert E. Braun, Jacqueline K. White, Elissa J. Chesler, Cathleen M. Lutz, Vivek Kumar
In Communications Biology 2019
AlphaTracker: A Multi-Animal Tracking and Behavioral Analysis Tool
Zexin Chen, Ruihan Zhang, Yu Eva Zhang, Haowen Zhou, Hao-Shu Fang, Rachel R. Rock, Aneesh Bal, Nancy Padilla-Coreano, Laurel Keyes, Kay M. Tye, Cewu Lu
In biorxiv 2020
Deep Graph Pose: a semi-supervised deep graphical model for improved animal pose tracking
Anqi Wu, E. Kelly Buchanan, Matthew Whiteway, Michael Schartner, Guido Meijer, Jean-Paul Noel, Erica Rodriguez, Claire Everett, Amy Norovich, Evan Schaffer, Neeli Mishra, C. Daniel Salzman, Dora Angelaki, Andrés Bendesky, The International Brain Laboratory, John Cunningham, Liam Paninski
In NeurIPS 2020
Simple Behavioral Analysis (SimBA)-an open source toolkit for computer classification of complex social behaviors in experimental animals
Simon RO Nilsson, Nastacia L. Goodwin, Jia Jie Choong, Sophia Hwang, Hayden R Wright, Zane C Norville, Xiaoyu Tong, Dayu Lin, Brandon S. Bentzley, Neir Eshel, Ryan J McLaughlin, Sam A. Golden
In biorxiv 2020
SLEAP: Multi-animal pose tracking
Talmo D. Pereira, Nathaniel Tabris, Junyu Li, Shruthi Ravindranath, Eleni S. Papadoyannis, Z. Yan Wang, David M. Turner, Grace McKenzie-Smith, Sarah D. Kocher, Annegret L. Falkner, Joshua W. Shaevitz, Mala Murthy
In biorxiv 2020
FreiPose: A Deep Learning Framework for Precise Animal Motion Capture in 3D Spaces
Christian Zimmermann, Artur Schneider, Mansour Alyahyay, Thomas Brox, Ilka Diester
In biorxiv 2020
Multiview Supervision By Registration
Yilun Zhang, Hyun Soo Park
In WACV 2020
Facial expressions of emotion states and their neuronal correlates in mice
Nejc Dolensek, Daniel A. Gehrlach, Alexandra S. Klein, Nadine Gogolla
In Science 2020
ZooBuilder: 2D and 3D Pose Estimation for Quadrupeds Using Synthetic Data
Abassin Sourou Fangbemi, Yi Fei Lu, Mao Yuan Xu, Xiao Wu Luo, Alexis Rolland, Chedy Raissi
In SCA (ACM Siggraph/Eurographics Symposium on Computer Animation) 2020
A Primer on Motion Capture with Deep Learning: Principles, Pitfalls and Perspectives
Alexander Mathis, Steffen Schneider, Jessy Lauer, Mackenzie W. Mathis
In Neuron 2020
Quantifying behavior to understand the brain
Talmo D. Pereira, Joshua W. Shaevitz, Mala Murthy
In Nature Neuroscience 2020 (review article)
EthoLoop: automated closed-loop neuroethology in naturalistic environments
Ali Nourizonoz, Robert Zimmermann, Chun Lum Andy Ho, Sebastien Pellat, Yannick Ormen, Clément Prévost-Solié, Gilles Reymond, Fabien Pifferi, Fabienne Aujard, Anthony Herrel, Daniel Huber
In Nature Methods 2020 (cover)
Automated Markerless Pose Estimation in Freely Moving Macaques using OpenMonkeyStudio
Praneet C. Bala, Benjamin R. Eisenreich, Seng Bum Michael Yoo, Benjamin Y. Hayden, Hyun Soo Park, Jan Zimmermann
In Nature Communications 2020
3D Bird Reconstruction: a Dataset, Model, and Shape Recovery from a Single View
Marc Badger, Yufu Wang, Adarsh Modh, Ammon Perkes, Nikos Kolotouros, Bernd Pfrommer, Marc F. Schmidt, Kostas Daniilidis
In ECCV 2020
Who Left the Dogs Out? 3D Animal Reconstruction with Expectation Maximization in the Loop
Benjamin Biggs, Oliver Boyne, James Charles, Andrew Fitzgibbon, Roberto Cipolla
In ECCV 2020
Shape and Viewpoint without Keypoints
Shubham Goel, Angjoo Kanazawa, Jitendra Malik
In ECCV 2020
Deep Cross-species Feature Learning for Animal Face Recognition via Residual Interspecies Equivariant Network
Xiao Shi, Chenxue Yang, Xue Xia, Xiujuan Chai
In ECCV 2020
RGBD-Dog: Predicting Canine Pose from RGBD Sensors
Sinead Kearney, Wenbin Li, Martin Parsons, Kwang In Kim, Darren Cosker
In CVPR 2020
AnimalWeb: A Large-Scale Hierarchical Dataset of Annotated Animal Faces
Muhammad Haris Khan, John McDonagh, Salman Khan, Muhammad Shahabuddin, Aditya Arora, Fahad Shahbaz Khan, Ling Shao, Georgios Tzimiropoulos
In CVPR 2020
Articulation Aware Canonical Surface Mapping
Nilesh Kulkarni, Abhinav Gupta, David F. Fouhey, Shubham Tulsiani
In CVPR 2020
Deformation-aware Unpaired Image Translation for Pose Estimation on Laboratory Animals
Siyuan Li, Semih Gunel, Mirela Ostrek, Pavan Ramdya, Pascal Fua, Helge Rhodin
In CVPR 2020
Learning from Synthetic Animals
Jiteng Mu, Weichao Qiu, Gregory Hager, Alan Yuille
In CVPR 2020 (oral)
3D-ZeF: A 3D Zebrafish Tracking Benchmark Dataset
Malte Pedersen, Joakim Bruslund Haurum, Stefan Hein Bengtson, Thomas B. Moeslund
In CVPR 2020
Transferring Dense Pose to Proximal Animal Classes
Artsiom Sanakoyeu, Vasil Khalidov, Maureen S. McCarthy, Andrea Vedaldi, Natalia Neverova
In CVPR 2020
Unsupervised Learning of Probably Symmetric Deformable 3D Objects from Images in the Wild
Shangzhe Wu, Christian Rupprecht, Andrea Vedaldi
In CVPR 2020 (Best Paper Award)
DeepPoseKit, a software toolkit for fast and robust animal pose estimation using deep learning
Jacob M. Graving, Daniel Chae, Hemal Naik, Liang Li, Benjamin Koger, Blair R. Costelloe, Iain D. Couzin
In eLife 2019
Using DeepLabCut for 3D markerless pose estimation across species and behaviors
Tanmay Nath, Alexander Mathis, An Chi Chen, Amir Patel, Matthias Bethge, Mackenzie Weygandt Mathis
In Nature Protocols 2019 (cover)
Fast animal pose estimation using deep neural networks
Talmo D. Pereira, Diego E. Aldarondo, Lindsay Willmore, Mikhail Kislin, Samuel S.-H. Wang, Mala Murthy, Joshua W. Shaevitz
In Nature Methods 2019
Real-time analysis of the behaviour of groups of mice via a depth-sensing camera and machine learning
Fabrice de Chaumont, Elodie Ey, Nicolas Torquet, Thibault Lagache, Stéphane Dallongeville, Albane Imbert, Thierry Legou, Anne-Marie Le Sourd, Philippe Faure, Thomas Bourgeron, Jean-Christophe Olivo-Marin
In Nature Biomedical Engineering 2019
Cross-Domain Adaptation for Animal Pose Estimation
Jinkun Cao, Hongyang Tang, Hao-Shu Fang, Xiaoyong Shen, Cewu Lu, Yu-Wing Tai
In ICCV 2019 (oral)
Canonical Surface Mapping via Geometric Cycle Consistency
Nilesh Kulkarni, Abhinav Gupta, Shubham Tulsiani
In ICCV 2019
MONET: Multiview Semi-supervised Keypoint Detection via Epipolar Divergence
Yuan Yao, Yasamin Jafarian, Hyun Soo Park
In ICCV 2019
Three-D Safari: Learning to Estimate Zebra Pose, Shape, and Texture from Images "In the Wild"
Silvia Zuffi, Angjoo Kanazawa, Tanya Berger-Wolf, Michael J. Black
In ICCV 2019
DeepLabCut: markerless pose estimation of user-defined body parts with deep learning
Alexander Mathis, Pranav Mamidanna, Kevin M. Cury, Taiga Abe, Venkatesh N. Murthy, Mackenzie Weygandt Mathis, Matthias Bethge
In Nature Neuroscience 2018
Learning Category-Specific Mesh Reconstruction from Image Collections
Angjoo Kanazawa, Shubham Tulsiani, Alexei A. Efros, Jitendra Malik
In ECCV 2018
Lions and Tigers and Bears: Capturing Non-Rigid, 3D, Articulated Shape from Images
Silvia Zuffi, Angjoo Kanazawa, Michael J. Black
In CVPR 2018 (spotlight)
Creatures great and SMAL: Recovering the shape and motion of animals from video
Benjamin Biggs, Thomas Roddick, Andrew Fitzgibbon, Roberto Cipolla
In ACCV 2018 (oral)
An unsupervised method for quantifying the behavior of paired animals
Ugne Klibaite, Gordon J Berman, Jessica Cande, David L Stern, Joshua W Shaevitz
In Physical Biology 2017
Single-View 3D Reconstruction of Animals
Angjoo Kanazawa
In Ph.D Thesis 2017
3D Menagerie: Modeling the 3D shape and pose of animals
Silvia Zuffi, Angjoo Kanazawa, David W. Jacobs, Michael J. Black
In CVPR 2017 (spotlight)
3D Pose Estimation of Flying Animals in Multi-view Video Datasets
Mikhail Breslav
In Ph.D Thesis 2016
Learning 3D Deformation of Animals from 2D Images
Angjoo Kanazawa, Shahar Kovalsky, Ronen Basri, David W. Jacobs
In Eurographics 2016 (Günter Enderle Best Paper Award)
idTracker: Tracking individuals in a group by automatic identification of unmarked animals
Alfonso Pérez-Escudero, Julián Vicente-Page, Robert C Hinz, Sara Arganda, Gonzalo G de Polavieja1
In Nature Methods 2014

Last updated in Oct 2024

This repository is inspired by Cat Papers with some code borrowed from it. Paper and project signs are taken from http://kesen.realtimerendering.com/.

About

Awesome papers for markerless animal motion capture and 3D reconstruction.

Topics

Resources

Stars

Watchers

Forks

Languages