Image grab not supported in Linux

[lukas-blecher]

Take a snapshot of the clipboard image, if any. Only macOS and Windows are currently supported.
https://pillow.readthedocs.io/en/stable/reference/ImageGrab.html#PIL.ImageGrab.grabclipboard

pix2tex.py uses this method but it is not supported in Linux.
Need to find a workaround.

[weiwenlan]

Hey, I am trying to test this model, I did all progress under the Linux. To work around the clipboard problem.
I just use the Image.open() to work this out.
However, I tested 2 pics in png format and the model threw out ('latex error', b'/bin/sh: 1: xelatex: not found\n', b'') for 2 times.
I am wondering where I did it wrong.

[weiwenlan]

@lukas-blecher I have retested that with small resolution like(210*170) and this turns out. However I still got the same result with latex error on xelatex.

[lukas-blecher]

Do I understand you correctly, that you edited the pix2tex.py file so you can parse an image path and open it in the place where I get the Image from the clipboard?
XeLaTex is not needed to run the model. However it is called if you set the flag --show or args.show=True. The idea was that I call the tex2pil function to render the predicted LaTeX code.
Does it still throw the error if you set the flag to false? You can visualize the predicted code quickly using e.g. katex.org.

[weiwenlan]

@lukas-blecher yeah, just modify the input of the image, it could read local images. I just find another tip that if you just add image.resize() to zoom every image to like x * 60, the output would be stabilized. Just as you mentioned, the model worked better under the low resolution.
Thanks a lot to let me know what is the xelatex doing here!

[lukas-blecher]

Yesterday I added a somewhat more sophisticated image resize method (not sure if you saw it) where I predict the image size beforehand using a image classification network (c610cca).
And just now I also added the possibility to load an image from the file, just like you did (c50dbb3).
Just wanted to let you know.

[weiwenlan]

Thanks a lot to let me know!

[lukas-blecher]

Possibly fixed by using an GUI, shown in this fork: https://github.com/katie-lim/LaTeX-OCR
because it is not using the grabclipboard function but integrates the snipping tool into the UI.

Really love it. Thank you @katie-lim.

[katie-lim]

Possibly fixed by using an GUI, shown in this fork: https://github.com/katie-lim/LaTeX-OCR
because it is not using the grabclipboard function but integrates the snipping tool into the UI.

Really love it. Thank you @katie-lim.

No problem! Just wanted a more convenient way to use the model.

[lukas-blecher]

GUI added #14.