Marker movement tracking

[Rancho-zhao]

Hello! I have successfully used your codes to generate depth maps from tactile images. Next, I want to try marker movement tracking by myself, but I don't know how to build the dataset. If I use the supervised learning method, how can I label the marker movement in the tactile images of my dataset?

[vocdex]

Hello! Glad to hear it this repo was useful.
I just realized I haven't pushed the code for marker tracking. Until I do so, you can take a look at this repo from Gelsight:

1. Optical flow method
   It works for DIGIT as well. You have to change the configs (trial&error) in **setting.py. My code is the same but for DIGIT.
   2.Neural Network method
   More recent, robust version of marker tracking. I had trouble with generating dataset for this model, as well.
   I'll let you know if I have updates with this.

[Rancho-zhao]

Thank you for your reply! Your suggestions are very useful! Actually, I have used the neural network method in the repo you mentioned, but there are some problems I find. For example, the flow arrows are all in the upper left corner of the image when using the trained model to test. The reason I think may be the “generate_data” module . I'm trying to solve these problems.

[vocdex]

I guess you have to train a new model for DIGIT. Their provided generic model didn't work for our DIGIT, as well. And for optical flow method, it would run best if you cover the markers with complete black since it detects the markers based on HSV color histogram. Good luck!