Skip to content

RomeroBarata/hierarchical_action_prediction

master
Switch branches/tags

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 
 
 
 
 
 
 
 
 
 
 

This is the code and data for the BMVC'20 paper Learning to Abstract and Predict Human Actions.

Download Data

Please download the Hierarchical Breakfast from the link below and put the folder HierarchicalBreakfast folder inside a data directory in this current directory (i.e. ./data/HierarchicalBreakfast/...).

Link: Hierarchical Breakfast.

For the original Breakfast Actions dataset annotation, and other related material (e.g. videos), please access Breakfast Actions.

Environment Setup

First please create an appropriate environment using conda:

conda env create -f environment.yml

conda activate fpua

Test Pre-Trained Models

To evaluate pre-trained models run the test.py script. A few examples:

Evaluate HERA

CUDA_VISIBLE_DEVICES=0 python -W ignore test.py hera_cv --pretrained_root ./pretrained --pretrained_suffix hera/hs-16-16_20e_bs512_act-elu-sigmoid_h2_lr1e-03_tlr1e-03_es-16-16_tanh_tsalways_mtll_use-hmgruv2_mv3_aips_wfa_isq35_osq50_nc5_atleast20_with-val.tar --fine_labels_root_path ./data/HierarchicalBreakfast/labels/fine --coarse_labels_root_path ./data/HierarchicalBreakfast/labels/coarse --fine_action_to_id ./data/HierarchicalBreakfast/dictionaries/fine_action_to_id.txt --coarse_action_to_id ./data/HierarchicalBreakfast/dictionaries/coarse_action_to_id.txt --observed_fraction 0.2 --ignore_silence_action silence

Evaluate Dummy Predictor on the fine level of the Hierarchical Breakfast

CUDA_VISIBLE_DEVICES=0 python -W ignore test.py dummy_cv --labels_root_path ./data/HierarchicalBreakfast/labels/fine --action_to_id ./data/HierarchicalBreakfast/dictionaries/fine_action_to_id.txt --observed_fraction 0.2 --unobserved_fraction 0.5 --ignore silence

Evaluate Independent-Single-RNN on coarse level of the Hierarchical Breakfast

CUDA_VISIBLE_DEVICES=0 python -W ignore test.py baselines_cv --pretrained_root ./pretrained --pretrained_suffix baselines/hs16_75e_bs12_act-sigmoid_h1_lr1e-03_es16_tanh_mtll_wfa_bt0coarse_tm_isq35_osq35_with-val.tar --fine_labels_root_path ./data/HierarchicalBreakfast/labels/fine --coarse_labels_root_path ./data/HierarchicalBreakfast/labels/coarse --fine_action_to_id ./data/HierarchicalBreakfast/dictionaries/fine_action_to_id.txt --coarse_action_to_id ./data/HierarchicalBreakfast/dictionaries/coarse_action_to_id.txt --observed_fraction 0.2 --ignore_silence_action silence

Train a Model

To train a model run the train.py script (use -h flag for options). See below an example on how to train the provided pre-trained HERA model (for full cross-validation you need to run the same command on the other training splits as well):

CUDA_VISIBLE_DEVICES=0 python train.py hera --training_data ./data/HierarchicalBreakfast/training_arrays/hera/split02-03-04T_isq35_osq50_nc5_atleast20_nosilence_withfinalaction.npz --validation_data ./data/HierarchicalBreakfast/training_arrays/hera/split02-03-04V_isq35_osq50_nc5_atleast20_nosilence_withfinalaction.npz --epochs 20 --multi_task_loss_learner --always_include_parent_state --log_dir ./pretrained/split02-03-04/hera

The training arrays are provided here for convenience and were extracted from the ./data/HierarchicalBreakfast/labels using the process_data.py script.

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages