【PaddlePaddle Hackathon 4】No.205 Notebook RFC

[Liyulingyue]

基于PaddleOCR和PaddleDetection进行工频场强计读数识别

本ISSUE是赛题【PaddlePaddle Hackathon 4】No.205的方案设计。

方案目标

工频场强计是用于测量交流电工作频率，以及交流电产生的电场和磁场强度（即高压辐射）的仪器。该仪器多为手持式，类似于电表读数，对工频场强计图片进行识别能够代替人工抄表，提高工作效率。本方案的目标是识别工频场图像中的工频电磁场数值和单位、以及下方X\Y\Z的数值，结构化输出结果: [ {"Info_Probe":""}, {"Freq_Set":""}, {"Freq_Main":""}, {"Val_Total":""},{"Val_X":""}, {"Val_Y":""}, {"Val_Z":""}, {"Unit":""}, {"Field":""} ] 。下图展示了两种工频场强计及对应的结构化输出结果。

方案介绍

该项目的推理部分的方案策略为通过PaddleDetection锁定和截取场强计区域，通过PaddleOCR检测区域内文字，并填充对应输出结构体。具体识别流程如下：

1. 通过PaddleDetection中的PPYoloE检测场强计屏幕的位置，并截取屏幕
2. 使用PaddleOCR检测截取区域内的文字
3. 根据文字信息（如文字的内容/文字框的大小等）确定文字对应的属性条目。例如最大的文字框对应的信息为输出结构体中的Val_Total信息。

流程图如下所示：

使用到的推理模型

• PaddleDetection
  • PPyoloE+：使用该模型是为了基于预训练模型在小数据集上获取更好的泛化性，如有需要可以更替检测模型
• PaddleOCR
  • ch_PP-OCRv3_xx

开发进展

完成基于PaddlePaddle的检测到结构化输出的全部流程。

[andrei-kochin]

@OpenVINO-dev-contest could you please help with that?

[OpenVINO-dev-contest]

@OpenVINO-dev-contest could you please help with that?

sure

[OpenVINO-dev-contest]

Hi @Liyulingyue Great thanks for your RFC application. We would like to create a new notebook instance for general purpose digital meter reader. Could you help to submit a PR and upload this instance to meter reader

[Liyulingyue]

嗨，非常感谢您的 RFC 应用程序。我们希望为通用数字抄表器创建一个新的笔记本实例。您能否帮助提交 PR 并将此实例上传到抄表器

sure

[HicariHuang]

Hi,
Is the issue still ongoing? How can I join the project to contribute and assist in development and testing?
Hicari

[Liyulingyue]

Hi, Is the issue still ongoing? How can I join the project to contribute and assist in development and testing? Hicari

Sorry for replying late. This project has been going on for too long and has not been updated. If you are willing to participate, test, and provide feedback, I would be very happy.

[HicariHuang]

Sorry for replying late. This project has been going on for too long and has not been updated. If you are willing to participate, test, and provide feedback, I would be very happy.

How is the current status of this issue? I can help with test, debug, and discuss!

[Liyulingyue]

How is the current status of this issue? I can help with test, debug, and discuss!

This issue was originally proposed for a competition, but now the competition has ended. For some reasons, the PR I submitted has not been included, so this issue has not been closed.

Although this PR has not been merged for a long time, if you are interested in this topic, you can try running ipynb to understand the current progress. If you think there are some areas that can be optimized, I am very willing to discuss and optimize with you.

[HicariHuang]

This issue was originally proposed for a competition, but now the competition has ended. For some reasons, the PR I submitted has not been included, so this issue has not been closed.

Although this PR has not been merged for a long time, if you are interested in this topic, you can try running ipynb to understand the current progress. If you think there are some areas that can be optimized, I am very willing to discuss and optimize with you.

What is the current recognition rate?

[Liyulingyue]

What is the current recognition rate?

I have not conducted large-scale experiments, perhaps very high, even exceeding 95%. The recognition rate of this project depends on the base model. In addition, the application scenario of this project does not require truly universal text recognition, but rather structured in specific areas. Therefore, we can post-process the recognition results based on pre information, resulting in better actual results.

Here is benchmark of paddleocr which is my base model: https://github.com/PaddlePaddle/PaddleOCR/blob/release/2.7/doc/doc_ch/benchmark.md

[HicariHuang]

I have not conducted large-scale experiments, perhaps very high, even exceeding 95%. The recognition rate of this project depends on the base model. In addition, the application scenario of this project does not require truly universal text recognition, but rather structured in specific areas. Therefore, we can post-process the recognition results based on pre information, resulting in better actual results.

Here is benchmark of paddleocr which is my base model: https://github.com/PaddlePaddle/PaddleOCR/blob/release/2.7/doc/doc_ch/benchmark.md

I found that this program requires manually input of four POINTS(Left top、right top、right bottom、left bottom) to specify the recognition area. Is there a possibility to collaborate and improve this process so that it can automatically detect the screen area without entering the coordinates ?

[Liyulingyue]

I found that this program requires manually input of four POINTS(Left top、right top、right bottom、left bottom) to specify the recognition area. Is there a possibility to collaborate and improve this process so that it can automatically detect the screen area without entering the coordinates ?

There are two solutions to this problem:

1. Train a Detection model to detect borders. I have implemented a version before, you can refer to it https://aistudio.baidu.com/projectdetail/5852670 However, this approach requires annotation of approximately 20 images and 30 minutes of training
2. Using CV2 for edge detection, such as Canny and minAreaRect, I haven't implemented this version yet.

Of course, I would prefer a completely untrained solution for detecting borders. If you have any ideas, feel free to discuss them together.

[HicariHuang]

There are two solutions to this problem:

1. Train a Detection model to detect borders. I have implemented a version before, you can refer to it https://aistudio.baidu.com/projectdetail/5852670 However, this approach requires annotation of approximately 20 images and 30 minutes of training
2. Using CV2 for edge detection, such as Canny and minAreaRect, I haven't implemented this version yet.

Of course, I would prefer a completely untrained solution for detecting borders. If you have any ideas, feel free to discuss them together.

Attempting to extract the edges of the image using Canny edge detection, then using the cv2.findContours function to find contours, and finally extracting the largest contour to obtain the four corner points. Since different images may require adjusting the parameters of the Canny edge detection, this is not a universal method applicable to all images. If I find a better method, I will discuss it with you.