MediaPipe KNIFT knift_labelmap.txt

[hbyjw]

How to get the correct tab order？

In the knift_labelmap.txt I thought it was

1-XLP
2-AJS
3-LHQWJN
4-ZZM

but it turned out to be

4-ZZM
3-LHQWJN
1-XLP
2-AJS

[sureshdagooglecom]

Hi @hbyjw ,
can you please elaborate this issue to investigate further?

[hbyjw]

Hi @sureshdagooglecom

I used my own image to build template_matching.aar knift_index.pb mobile_cpu.binarypb but I don't know how to fill in knift_labelmap.txt correctly, when I run it in android studio, I don't know how to make it correct Modified knift_labelmap.txt.

[PrinceP]

@hbyjw In the template creation directory what is the sequence of image files?

bazel-bin/mediapipe/examples/desktop/template_matching/template_matching_tflite \ --calculator_graph_config_file=mediapipe/graphs/template_matching/index_building.pbtxt \ --input_side_packets="file_directory=<template image directory>,file_suffix=png,output_index_filename=<output index filename>"

That particular sequence should be taken in knift_label.txt file.

[hbyjw]

@PrinceP
bazel-bin/mediapipe/examples/desktop/template_matching/template_matching_tflite \ --calculator_graph_config_file=mediapipe/graphs/template_matching/index_building.pbtxt \ --input_side_packets="file_directory=/Users/yu/HO/mediapipe-0.8.9/images,file_suffix=png,output_index_filename=knift_index.0307-1.pb"

template creation directory

knift_label.txt

1. If I put it in android studio according to the settings in knift_label.txt, the order displayed in the phone is wrong

2. I would like to know if the knife_label.txt is created automatically, or if I create it myself. If it was me, where should I put the knift_label.txt when I create the knife_index.pb

[hbyjw]

@PrinceP

#3133 (comment)

The correct name of the item in the picture is 3-LianHuaQingWenJIaoNang

[PrinceP]

@hbyjw

1. The knift_label file has to be given by the user only.

Step 3: Replace mediapipe/models/knift_index.pb with the index file you generated, and update mediapipe/models/knift_labelmap.txt with your own template names.

2. The index is made here. Can you put some debug statements in calculator and check the sequence of indexes taken. https://github.com/google/mediapipe/blob/b899d17f185f6bcbf3f5947d3e134f8ce1e69407/mediapipe/calculators/video/box_detector_calculator.cc#L51

[hbyjw]

@PrinceP

I don't know much about C++, but I'll give it a try

[yegenzhi]

Is image #3 the only wrong detection? if you shoot the image #1 xianglianpian, i wonder what is detection result from the demo?

[hbyjw]

@yegenzhi

knift_labelmap.txt 顺序

1-XiangLianPian
2-AnJiSuan
3-LianHuaQingWenJIaoNang    
4-ZZM

[yegenzhi]

that's indeed an interesting bug.
when you run index_building.pbtxt, can you look out for the logs from this file?
https://github.com/google/mediapipe/blob/e6c19885c6d3c6f410c730952aeed2852790d306/mediapipe/calculators/util/local_file_pattern_contents_calculator.cc#L60
you can just copy paste the logs here.

[hbyjw]

Where can I see the log, I don't see it in the terminal.

[yegenzhi]

ok.
i believe if you add the following line to LN54 of this file, this may fix your issue.

std::sort(filenames_.begin(), filenames_.end());

can you please try?

[hbyjw]

#3133 (comment)

I wrote the record to txt

[yegenzhi]

you can add " --alsologtostderr" to your command to make the logs show in terminal.

yes. the bug is because the match file api at LN50 does not guarantee the matched result in exact order for some OS.
we will fix this in the next release.

Thanks for the catch.

[hbyjw]

Okay thank you.

I will add the code and try again.

[hbyjw]

#3133 (comment)

After adding the code, the correct order is displayed.

I have this in both hyper-v ubuntu 20.04 and mac os (12.0.1)

[jianingwei]

Is this a bug? Should we fix it in our next release?

[google-ml-butler]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you.

[google-ml-butler]

Closing as stale. Please reopen if you'd like to work on this further.

[google-ml-butler]

Are you satisfied with the resolution of your issue?
Yes
No