which paper is implementd by this repository?"what you get"..or "coarse to fine"

[ganliqiang]

the paper link is not the "what you get is what you see ",the link is the paper"Image-to-Markup Generation with Coarse-to-Fine Attention", i want to know which paper i should read to know the principle.thanks

[da03]

Hi @ganliqiang, "image-to-markup generation with coarse-to-fine attention" is just an updated version of "what you get is what you see". However, coarse-to-fine attention is not implemented in this repo, so reading the earlier version of the paper is enough for understanding this implementation.

However, note that there is a difference: in this repo the attention is over columns of features (so each feature in the column is given the same weight), while in the paper we used a more powerful attention which attends over individual features. Which one is better (attending over columns or individual features) both depends on the problem (in normal OCR probably column-wise attention is enough if the images are not distorted, while for more complicated structure like math formula or tables we should attend to individual positions). It also depends on the amount of training data we have: since attending to individual features is more powerful, we'd expect more training data to make it work.