Sharing Training Details

[YusenZhang826]

Hi Pan,

Your paper "Inter-GPS: Interpretable Geometry Problem Solving with Formal Language and Symbolic Reasoning" is awesome.

However, when reproducing results of this work, I have one problem. I trained the symbol detection model with the data you provided, but the model could not perform as well as the box_result you released. Could you please share more training details?

Thank you very much and looking forward to your reply.

[lupantech]

Hi Yusen,

Thank you for following our work! The released code sets the default arguments that we used for training the symbol detection model. The symbol detection results are saved in box_results and not quantitatively evaluated in our experiment. So I was wondering how you evaluated your detection results and what they looked like?

Best,
Pan

[YusenZhang826]

Hi Pan,

Thanks for your response. I trained the detection model following your default arguments, and evaluated the results on the test set using the function csv_eval.evaluate in your code. The evaluation results are shown in the figure evaluation.png, and the detection results on the test set are saved in box_results_reproduce.zip.box_results_reproduce.zip

And the accuracy of Symbolic Solver task is only around 26% when we replaced the origin box_results with box_results_reproduce(keeping other arguments fixed). Could you please let me know where the problem is.

Looking forward for your reply.

[lupantech]

Hi Yusen,

Sorry for the late response! I trained the detection model and I obtained similar detection results as yours. So the detection model works well. 

I was wondering if you ran the text recognition tool MathpixOCR and generated the diagram logic forms again before you tested the symbolic solver. If you generate new symbol detection results, you must also repeat these two steps. MathpixOCR we used is not free and costs about $20 for our test dataset. So, if possible, you can replace MathpixOCR with other open-source tools or models.

Pan