Removing the requirement groups to be exactly 2.

[fire]

Hi, I am trying to remove the requirement that there are only A and B groups of skeletons.

I want there to be A, B, C, D, any number of groups.

Currently each group must have the same skeleton.

Problem 1: Support > 2 groups.

Problem 2: Support picking any pair from any group and in forwards or backwards order.

[PeizhuoLi]

Hi, in our current model, a new network is needed for a new skeleton topology and this obstacle seems difficult to bypass. However, if there is a dataset for each group, it is possible to train multiple networks simultaneously in a common latent space.

[fire]

import itertools as it
group_a = [
    "BerkeleyMHAD_skl_s05",
    "BerkeleyMHAD_skl_s06", 
    "BerkeleyMHAD_skl_s07",
    "BerkeleyMHAD_skl_s08", 
]
group_b = [
    "Female1",
    "Female2",
]
group_c = [
    "Test",
    "Test2"
]
group_d = [
    "Cat",
    "Cat2"
]
all_groups = [group_a, group_b, group_c, group_d]
characters = [list(x) for x in it.combinations(all_groups, 2)]
print(characters)
# [[['BerkeleyMHAD_skl_s05', 'BerkeleyMHAD_skl_s06', 'BerkeleyMHAD_skl_s07', 'BerkeleyMHAD_skl_s08'], ['Female1', 'Female2']], [['BerkeleyMHAD_skl_s05', 'BerkeleyMHAD_skl_s06', 'BerkeleyMHAD_skl_s07', 'BerkeleyMHAD_skl_s08'], ['Test', 'Test2']], [['BerkeleyMHAD_skl_s05', 'BerkeleyMHAD_skl_s06', 'BerkeleyMHAD_skl_s07', 'BerkeleyMHAD_skl_s08'], ['Cat', 'Cat2']], [['Female1', 'Female2'], ['Test', 'Test2']], [['Female1', 'Female2'], ['Cat', 'Cat2']], [['Test', 'Test2'], ['Cat', 'Cat2']]]

This is what I planned for multiple models, but wasn't sure. Can you tell me more about the common latent space?

[PeizhuoLi]

In our original implementation, we use two auto encoders. They share a common latent space, i.e., it's possible to decode the latent code from Encoder_A with Decoder_B. This latent space can be generalize to more than two auto encoders, i.e., to make it possible to decode the latent code from Encoder_i with Decoder_j. For training, one very simple solution is to train all the pairs simultaneously, but training all the pairs by random sampling should also work.

[fire]

1. Does order matter? Is Encoder_A <-> Decoder_B?

2. If order doesn't matter I would have to generate these encoder pairs:

import itertools as it
all_groups = ["a", "b", "c"]
characters = [list(x) for x in it.combinations(all_groups, 2)]
print(characters)
# [['a', 'b'], ['a', 'c'], ['b', 'c']]

• encoder_a <-> encoder_b
• encoder_a <-> encoder_c
• encoder_b <-> encoder_c

I'm still having trouble understanding this.

[PeizhuoLi]

Personally I think you might need to train all pairs, like you are doing now. But it's an open problem. For example, if A<->B, A<->C and A<-> D are all well trained, since they share a common latent space, it seems all the pairs then shall work.

[fire]

This was what I used.

import itertools as it
all_groups = ["group_a", "group_b", "group_c", "group_d"]
characters = [list(x) for x in it.permuations(all_groups, 2)]
print(characters)

The idea is I can go any group to any other group both forwards and backwards.

I had trouble getting the right results, but that's an evaluation / engineering problem and not a theory one.