在image-level中object_file和verb_file是用来干嘛的

[whqwill]

在image-level中object_file和verb_file是用来干嘛的

[liangxuy]

These two files are used in function prior_info of https://github.com/DirtyHarryLYL/HAKE-Action/blob/Image-level-HAKE-Action/scripts/obj_mask.py to boost the performance.
Take object.csv as an example, it is an matrix of size (9,658, 600). 9,658 is the size of testing set and 600 is the number of HOI categories. The value is non-zero only when the HOI index of the related object is detected in this image, and the value is the object detection score. For example, if the value of index <ride bicycle> is non-zero, then bicycle must be detected in this image.

[whqwill]

But these two files are used directly for testing? I am confused. So how to get these two files? You must use some code to generate these two files.

[whqwill]

BTW, what is the meaning of "related", how to get the related object? why are they related?

[whqwill]

然后就是 hp_list 以及 pvp_ankle2、pvp_knee2等等这些标注是做什么用的？

[liangxuy]

But these two files are used directly for testing? I am confused. So how to get these two files? You must use some code to generate these two files.

BTW, what is the meaning of "related", how to get the related object? why are they related?

The object.csv is only based on object detection results. To make it clear, given an image, if the object bicycle is detected, then in the 600-d array, the values of bicycle-related HOIs (such as carry bicycle, ride bicycle, park bicycle, ...) are non-zero.
The verb.csv is based on the results of verb classification. We directly expand 117-d verb results to 600-d arrays to serve as a kind of prior knowledge.

[liangxuy]

然后就是 hp_list 以及 pvp_ankle2、pvp_knee2等等这些标注是做什么用的？

Pvp_ankle2, pvp_knee2, ... are the labelings of the part-states of a person's 10 body parts. hp_list is a 10-d array corresponds to a person's 10 body parts. If the body part has no-interaction with the object, the vaule is 0, else the value is 1.

[liangxuy]

Please see our paper PaStaNet: Toward Human Activity Knowledge Engine for better understanding.

[whqwill]

然后就是 hp_list 以及 pvp_ankle2、pvp_knee2等等这些标注是做什么用的？

Pvp_ankle2, pvp_knee2, ... are the labelings of the part-states of a person's 10 body parts. hp_list is a 10-d array corresponds to a person's 10 body parts. If the body part has no-interaction with the object, the vaule is 0, else the value is 1.

why dose ankle have 6 labels and hand have 23 labels? Does that mean hands can have 23 different states? Which is not mentioned in the paper. Also, I understand the mearning of hp_list now, but there are three persons and only one hp_list?

[liangxuy]

This is the setting of our dataset, and we provide the full list of our part-state definitions under here. If we have multiple persons, then we use "OR" operation to fuse the three hp_lists.