Skip to content

This project aims to reconstruct the offline character images back to online handwritten trajectories using an encoder-decoder with attention and GMM

License

Notifications You must be signed in to change notification settings

ntuanhung/im2ink

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

13 Commits
 
 
 
 
 
 
 
 

Repository files navigation

im2ink

Overview

This project aims to reconstruct the offline character images back to online handwritten trajectories using an encoder-decoder with attention and GMM.

For questions or more details, please contact us via email addresses: ntuanhung@gmail.com or nakagawa (at) cc.tuat.ac.jp

Demo

In this part, we provide samples as animated movies rather than static figures in the paper. Moreover, we show here more samples than in the paper.

1. Recovered samples (Figures 3, 4 and 5 in the paper)

In the following figures, the original trajectories are colored by black and the recovered trajectories are colored by red. For figures generated by the attention layer, the focused regions on original trajectories at every step are shown by brighter color.

Next, the samples are successfully recovered either with or without the attention layer.

Figure 3. Successfully recovered examples.
With attention layer Without attention layer
Samples in paper paper_fig3a_1_animated paper_fig3a_1_static paper_fig3b_1_animated paper_fig3b_1_static
paper_fig3a_2_animated paper_fig3a_2_static paper_fig3b_2_animated paper_fig3b_2_static
paper_fig3a_3_animated paper_fig3a_3_static paper_fig3b_3_animated paper_fig3b_3_static
Other samples other_fig3a_1_animated other_fig3a_1_static other_fig3b_1_animated other_fig3b_1_static
other_fig3a_2_animated other_fig3a_2_static other_fig3b_2_animated other_fig3b_2_static
other_fig3a_3_animated other_fig3a_3_static other_fig3b_3_animated other_fig3b_3_static

The samples were successfully recovered with the attention layer, but unsuccessfully recovered without the attention layer.

Figure 4. Successfully recovered examples with attention.
With attention layer Without attention layer
Samples in paper paper_fig4a_1_animated paper_fig4a_1_static paper_fig4b_1_animated paper_fig4b_1_static
paper_fig4a_2_animated paper_fig4a_2_static paper_fig4b_2_animated paper_fig4b_2_static
paper_fig4a_3_animated paper_fig4a_3_static paper_fig4b_3_animated paper_fig4b_3_static
Other samples other_fig4a_1_animated other_fig4a_1_static other_fig4b_1_animated other_fig4b_1_static
other_fig4a_2_animated other_fig4a_2_static other_fig4b_2_animated other_fig4b_2_static
other_fig4a_3_animated other_fig4a_3_static other_fig4b_3_animated other_fig4b_3_static

The samples are unsuccessfully recovered either with or without the attention layer.

Figure 5. Unsuccessfully recovered examples.
With attention layer Without attention layer
Samples in paper paper_fig5a_1_animated paper_fig5a_1_static paper_fig5b_1_animated paper_fig5b_1_static
paper_fig5a_2_animated paper_fig5a_2_static paper_fig5b_2_animated paper_fig5b_2_static
paper_fig5a_3_animated paper_fig5a_3_static paper_fig5b_3_animated paper_fig5b_3_static

2. Unnatural recovered samples (Figure 6(a) in the paper)

Although the following samples are successfully recovered, their online trajectories consist of more points than the original ones, as shown in Figure 6 (a).

Figure 6 (a). Sample recovered with unusual pen/touch speed.
Original trajectories Recovered trajectories
Sample in paper paper_fig6a_1_origin paper_fig6a_1_recover paper_fig6a_1_static
Other samples other_fig6a_1_origin other_fig6a_1_recover paper_fig6a_1_static
other_fig6a_2_origin other_fig6a_2_recover paper_fig6a_2_static

3. Incompletely recovered samples (Figure 6(b) in the paper)

As shown in Figure 6 (b), there are incompletely recovered samples when attention was stuck.

Figure 6 (b). Sample whose recovery was stuck.
Original trajectories Recovered trajectories
Sample in paper paper_fig6b_1_origin paper_fig6b_1_recover paper_fig6b_1_static
Other samples other_fig6b_1_origin other_fig6b_1_recover paper_fig6b_1_static
other_fig6b_2_origin other_fig6b_2_recover paper_fig6b_2_static

About

This project aims to reconstruct the offline character images back to online handwritten trajectories using an encoder-decoder with attention and GMM

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published