HMM for sign language recognition

[kahrabji08]

Hi there

Am trying to use mlpack for my MSc research in sign language recognition. I know the theory of HMM and I tested the command line programs provided for HMM by mlpack. I am using cyberGloves(which provides 23 values) and a tracker(which provides 6 values for position and orientation) for the data collection.So my feature vector is of width of 29. I am just not sure about few things for which I hope to get your help:

1. number of models: Am doing continuous recognition so I got my data by recording 40 sentences coming from 80 words. I think I have to build an HMM model for each word, this means 80 models, Am I right?
2. Training Data : to prepare the training data i should put all the training examples of one word in one data file and use it for training. For example if the first recorded sentence is " I LOVE FOOTBALL" and the 2nd sentence was " I LOVE MY MOTHER" then I should take the sensor data correspond to the word "LOVE" from the first sentence and put it in a new file, then add to that file the sensor data correspond to the same word "LOVE" from the second sentence and so on for the 40 sentences. And then Start training. Am I right?
3. What should be the type of my HMM? gaussian or gmm? if it's the later how could I know the number of gaussians in each GMM?
4. How should I label the data? and How to choose the number of States?
5. Recognition: I want to perform sentence recognition. So in order to do that, for each feature vector should I first find the most probable model using hmm_loglik then use hmm_viterbi based on that model to find the hidden state(the word)?

Thanks for your help and support,
Mohamed

[rcurtin]

Hi,

I think your question is more appropriate on StackOverflow than here -- generally issues here are for mlpack-specific support, but your questions are general to HMM usage. Each of your questions are typical questions for HMM usage, and many of the parameters you're wondering about could be selected by cross-validating and selecting the model with highest log-likelihood. If you have any problems with the software specifically, open an issue here and we can help out.

Thanks,
Ryan