Decrease inference time of 30% when not using CRF for seq tagging

[pommedeterresautee]

When CRF is not used, data were slow down by a for loop.
In this little refactoring, this part have been vectorized.
I kept everything on Pytorch Tensor.
As always, I am interested in your timings on V100 :-)

On my 2080 TI, without refactoring on French data with a model trained without CRF, before I got 63 seconds, and after modification I get 43 seconds.

[alanakbik]

@pommedeterresautee big thanks for improving this! I ran some experiments on a v100 with two sequence taggers that don't use a CRF: the 'frame' and 'de-pos' models:

• For 'frame' on the Ontonotes corpus, an evaluation run went from ~98 to ~87 seconds and a predict over the whole dataset went from ~10.5 minutes to ~7.6 minutes.

• For 'de-pos' on the German UD HDT corpus, an an evaluation run went from ~159 to ~137 seconds and a predict over the whole dataset went from ~12.8 minutes to ~9.4 minutes.

So we are seeing really nice speed improvements :)

[alanakbik]

👍

[adizdari]

👍