Skip to content


Repository files navigation

Learning Time-Critical Responses for Interactive Character Control



This code implements the paper Learning Time-Critical Responses for Interactive Character Control. This system implements teacher-student framework to learn time-critically responsive policies, which guarantee the time-to-completion between user inputs and their associated responses regardless of the size and composition of the motion databases. This code is written in java and Python, based on Tensorflow2.


Kyungho Lee, Sehee Min, Sunmin Lee, and Jehee Lee. 2021. Learning Time-Critical Responses for Interactive Character Control. ACM Trans. Graph. 40, 4, 147. (SIGGRAPH 2021)

Project page:



How to install

This code is implemented with Java and Python, and was developed using Eclipse on Windows. A Windows 64-bit environment is required to run the code.


Install JDK 1.8

Java SE Development Kit 8 Downloads

Install Eclipse

Install Eclipse IDE for Java Developers

Install Python 3.6

Install pydev to Eclipse

Install cuda and cudnn 10.0

CUDA Toolkit 10.0 Archive


Install Visual C++ Redistributable for VS2012

Laplacian Motion Editing(PmQmJNI.dll) is implemented in C++, and VS2012 is required to run it.

Visual C++ Redistributable for Visual Studio 2012 Update 4

Install JEP(Java Embedded Python)

Java Embedded Python

This library requires a part of the Visual Studio installation. I don't know exactly which ones are needed, but I'm guessing .net framework 3.5, VC++ 2015.3 v14.00(v140). Installing Visual Studio 2017 or later may be helpful.

Install Tensoflow 1.14.0

pip install tensorflow-gpu==1.14.0

Install this repository

We recommend downloading through Git in Eclipse environment.

  1. Open Git Perspective in Elcipse
  2. Paste repository url and clone repository ( '' )
  3. Select all projects in Working Tree
  4. Right click and select Import Projects, and Import existing Eclipse projects.

Or you can just download the repository as Zip file and extract it, and import it using File->Import->General->Existing Projects into Workspace in Eclipse.

Install third party library

This code uses Interactive Character Animation by Learning Multi-Objective Control for learning the student policy.

Download required third pary library files( and extract it to mrl.motion.critical folder.


The entire data used in the paper cannot be published due to copyright issues. This repository contains only minimal motion dataset for algorithm validation. SNU Motion Database was used for martial arts movements, CMU Motion Database was used for locomotion.

How to run


All of the instructions below are assumed to be executed based on Eclipse. Executable java files are grouped in package of project mrl.motion.critical.

  • You can directly open source file with Ctrl+Shift+R
  • You can run the currently open source file with Ctrl+F11.
  • You can configure program arguments in Run->Run Configurations menu.

Pre-trained student policy

You can see the pre-trained network by running Pre-trained network file is located at mrl.python.neural\train\martial_arts_sp_da

  • 1, 2 : walk, run
  • 3,4,5,6 : martial arts actions
  • q,w,e,r,t : control critical response time

How to train

  1. Data Annotation & Configuration
    • You can check motion data list and annotation information by executing
  2. Model Configuration
    • Action list, critical response time of each action, user input model and error metric is defined at
  3. Preprocessing
    • You can precompute data table for pruning by executing
    • The data file will be located at mrl.motion.critical\output\dp_cache
  4. Training teacher policy
    • You can train teacher policy by executing
    • The result will be located at mrl.motion.critical\train_rl
  5. Training data for student policy
    • You can generate training data for student policy by executing
    • The result will be located at mrl.python.neural\train
  6. Training student policy
    • You can train student policy by executing mrl.python.neural\
    • You need to set program arguments in Run->Run Configurations menu.
      • arguments format : <new/load>
      • ex) martial_arts_sp new 0.0001
  7. Running student policy
    • You can see the trained student policy by running
    • This class will be load student policy located at mrl.python.neural\train.


Learning Time-Critical Responses for Interactive Character Control







No releases published


No packages published